A Distributed Workflow Management System with Case Study of Real-life Scientific Applications on Grids

Qishi Wu, Michelle Zhu, Yi Gu, Patrick Brown, Xukang Lu, Wuyin Lin, Yangang Liu

Research output: Contribution to journalArticle

14 Citations (Scopus)

Abstract

Next-generation scientific applications feature complex workflows comprised of many computing modules with intricate inter-module dependencies. Supporting such scientific workflows in wide-area networks especially Grids and optimizing their performance are crucial to the success of collaborative scientific discovery. We develop a Scientific Workflow Automation and Management Platform (SWAMP), which enables scientists to conveniently assemble, execute, monitor, control, and steer computing workflows in distributed environments via a unified web-based user interface. The SWAMP architecture is built entirely on a seamless composition of web services: the functionalities of its own are provided and its interactions with other tools or systems are enabled through web services for easy access over standard Internet protocols while being independent of different platforms and programming languages. SWAMP also incorporates a class of efficient workflow mapping schemes to achieve optimal end-to-end performance based on rigorous performance modeling and algorithm design. The performance superiority of SWAMP over existing workflow mapping schemes is justified by extensive simulations, and the system efficacy is illustrated by large-scale experiments on real-life scientific workflows for climate modeling through effective system implementation, deployment, and testing on the Open Science Grid.

Original languageEnglish
Pages (from-to)367-393
Number of pages27
JournalJournal of Grid Computing
Volume10
Issue number3
DOIs
StatePublished - 16 Aug 2012

Fingerprint

Automation
Web services
Wide area networks
Internet protocols
Computer programming languages
User interfaces
Testing
Chemical analysis
Experiments

Keywords

  • Climate modeling
  • Distributed computing
  • Open Science Grid
  • Scientific workflow

Cite this

Wu, Qishi ; Zhu, Michelle ; Gu, Yi ; Brown, Patrick ; Lu, Xukang ; Lin, Wuyin ; Liu, Yangang. / A Distributed Workflow Management System with Case Study of Real-life Scientific Applications on Grids. In: Journal of Grid Computing. 2012 ; Vol. 10, No. 3. pp. 367-393.
@article{e9bc96a863134224b89e2c4ed74b4025,
title = "A Distributed Workflow Management System with Case Study of Real-life Scientific Applications on Grids",
abstract = "Next-generation scientific applications feature complex workflows comprised of many computing modules with intricate inter-module dependencies. Supporting such scientific workflows in wide-area networks especially Grids and optimizing their performance are crucial to the success of collaborative scientific discovery. We develop a Scientific Workflow Automation and Management Platform (SWAMP), which enables scientists to conveniently assemble, execute, monitor, control, and steer computing workflows in distributed environments via a unified web-based user interface. The SWAMP architecture is built entirely on a seamless composition of web services: the functionalities of its own are provided and its interactions with other tools or systems are enabled through web services for easy access over standard Internet protocols while being independent of different platforms and programming languages. SWAMP also incorporates a class of efficient workflow mapping schemes to achieve optimal end-to-end performance based on rigorous performance modeling and algorithm design. The performance superiority of SWAMP over existing workflow mapping schemes is justified by extensive simulations, and the system efficacy is illustrated by large-scale experiments on real-life scientific workflows for climate modeling through effective system implementation, deployment, and testing on the Open Science Grid.",
keywords = "Climate modeling, Distributed computing, Open Science Grid, Scientific workflow",
author = "Qishi Wu and Michelle Zhu and Yi Gu and Patrick Brown and Xukang Lu and Wuyin Lin and Yangang Liu",
year = "2012",
month = "8",
day = "16",
doi = "10.1007/s10723-012-9222-7",
language = "English",
volume = "10",
pages = "367--393",
journal = "Journal of Grid Computing",
issn = "1570-7873",
publisher = "Springer Netherlands",
number = "3",

}

A Distributed Workflow Management System with Case Study of Real-life Scientific Applications on Grids. / Wu, Qishi; Zhu, Michelle; Gu, Yi; Brown, Patrick; Lu, Xukang; Lin, Wuyin; Liu, Yangang.

In: Journal of Grid Computing, Vol. 10, No. 3, 16.08.2012, p. 367-393.

Research output: Contribution to journalArticle

TY - JOUR

T1 - A Distributed Workflow Management System with Case Study of Real-life Scientific Applications on Grids

AU - Wu, Qishi

AU - Zhu, Michelle

AU - Gu, Yi

AU - Brown, Patrick

AU - Lu, Xukang

AU - Lin, Wuyin

AU - Liu, Yangang

PY - 2012/8/16

Y1 - 2012/8/16

N2 - Next-generation scientific applications feature complex workflows comprised of many computing modules with intricate inter-module dependencies. Supporting such scientific workflows in wide-area networks especially Grids and optimizing their performance are crucial to the success of collaborative scientific discovery. We develop a Scientific Workflow Automation and Management Platform (SWAMP), which enables scientists to conveniently assemble, execute, monitor, control, and steer computing workflows in distributed environments via a unified web-based user interface. The SWAMP architecture is built entirely on a seamless composition of web services: the functionalities of its own are provided and its interactions with other tools or systems are enabled through web services for easy access over standard Internet protocols while being independent of different platforms and programming languages. SWAMP also incorporates a class of efficient workflow mapping schemes to achieve optimal end-to-end performance based on rigorous performance modeling and algorithm design. The performance superiority of SWAMP over existing workflow mapping schemes is justified by extensive simulations, and the system efficacy is illustrated by large-scale experiments on real-life scientific workflows for climate modeling through effective system implementation, deployment, and testing on the Open Science Grid.

AB - Next-generation scientific applications feature complex workflows comprised of many computing modules with intricate inter-module dependencies. Supporting such scientific workflows in wide-area networks especially Grids and optimizing their performance are crucial to the success of collaborative scientific discovery. We develop a Scientific Workflow Automation and Management Platform (SWAMP), which enables scientists to conveniently assemble, execute, monitor, control, and steer computing workflows in distributed environments via a unified web-based user interface. The SWAMP architecture is built entirely on a seamless composition of web services: the functionalities of its own are provided and its interactions with other tools or systems are enabled through web services for easy access over standard Internet protocols while being independent of different platforms and programming languages. SWAMP also incorporates a class of efficient workflow mapping schemes to achieve optimal end-to-end performance based on rigorous performance modeling and algorithm design. The performance superiority of SWAMP over existing workflow mapping schemes is justified by extensive simulations, and the system efficacy is illustrated by large-scale experiments on real-life scientific workflows for climate modeling through effective system implementation, deployment, and testing on the Open Science Grid.

KW - Climate modeling

KW - Distributed computing

KW - Open Science Grid

KW - Scientific workflow

UR - http://www.scopus.com/inward/record.url?scp=84866300443&partnerID=8YFLogxK

U2 - 10.1007/s10723-012-9222-7

DO - 10.1007/s10723-012-9222-7

M3 - Article

AN - SCOPUS:84866300443

VL - 10

SP - 367

EP - 393

JO - Journal of Grid Computing

JF - Journal of Grid Computing

SN - 1570-7873

IS - 3

ER -