TY - JOUR
T1 - A Distributed Workflow Management System with Case Study of Real-life Scientific Applications on Grids
AU - Wu, Qishi
AU - Zhu, Mengxia
AU - Gu, Yi
AU - Brown, Patrick
AU - Lu, Xukang
AU - Lin, Wuyin
AU - Liu, Yangang
PY - 2012/9
Y1 - 2012/9
N2 - Next-generation scientific applications feature complex workflows comprised of many computing modules with intricate inter-module dependencies. Supporting such scientific workflows in wide-area networks especially Grids and optimizing their performance are crucial to the success of collaborative scientific discovery. We develop a Scientific Workflow Automation and Management Platform (SWAMP), which enables scientists to conveniently assemble, execute, monitor, control, and steer computing workflows in distributed environments via a unified web-based user interface. The SWAMP architecture is built entirely on a seamless composition of web services: the functionalities of its own are provided and its interactions with other tools or systems are enabled through web services for easy access over standard Internet protocols while being independent of different platforms and programming languages. SWAMP also incorporates a class of efficient workflow mapping schemes to achieve optimal end-to-end performance based on rigorous performance modeling and algorithm design. The performance superiority of SWAMP over existing workflow mapping schemes is justified by extensive simulations, and the system efficacy is illustrated by large-scale experiments on real-life scientific workflows for climate modeling through effective system implementation, deployment, and testing on the Open Science Grid.
AB - Next-generation scientific applications feature complex workflows comprised of many computing modules with intricate inter-module dependencies. Supporting such scientific workflows in wide-area networks especially Grids and optimizing their performance are crucial to the success of collaborative scientific discovery. We develop a Scientific Workflow Automation and Management Platform (SWAMP), which enables scientists to conveniently assemble, execute, monitor, control, and steer computing workflows in distributed environments via a unified web-based user interface. The SWAMP architecture is built entirely on a seamless composition of web services: the functionalities of its own are provided and its interactions with other tools or systems are enabled through web services for easy access over standard Internet protocols while being independent of different platforms and programming languages. SWAMP also incorporates a class of efficient workflow mapping schemes to achieve optimal end-to-end performance based on rigorous performance modeling and algorithm design. The performance superiority of SWAMP over existing workflow mapping schemes is justified by extensive simulations, and the system efficacy is illustrated by large-scale experiments on real-life scientific workflows for climate modeling through effective system implementation, deployment, and testing on the Open Science Grid.
KW - Climate modeling
KW - Distributed computing
KW - Open Science Grid
KW - Scientific workflow
UR - http://www.scopus.com/inward/record.url?scp=84866300443&partnerID=8YFLogxK
U2 - 10.1007/s10723-012-9222-7
DO - 10.1007/s10723-012-9222-7
M3 - Article
AN - SCOPUS:84866300443
SN - 1570-7873
VL - 10
SP - 367
EP - 393
JO - Journal of Grid Computing
JF - Journal of Grid Computing
IS - 3
ER -