A distributed workflow management system with case study of real-life scientific applications

Qishi Wu, Mengxia Zhu, Yi Gu, Xukang Lu, Patrick Brown, Michael A. Reuter, Stephen D. Miller

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

Abstract

Supporting large-scale scientific workflows in distributed network environments and optimizing their performances are crucial to the success of collaborative scientific discovery. We develop a generic scientific workflow platform, referred to as SciFlow, which constitutes a flexible framework to facilitate the distributed execution and management of scientific workflows and incorporates a class of workflow mapping schemes to achieve optimal end-to-end performances. The functionalities of SciFlow are provided and its interactions with other tools or systems are enabled through web services for easy access over standard Internet protocols while being independent of different platforms and programming languages. The performance superiority of SciFlow over existing workflow mapping schemes and management systems is illustrated by extensive simulations and is further verified by large-scale experiments on real-life scientific workflows through effective system implementation and deployment in distributed network environments.

Original languageEnglish
Title of host publication30th IEEE International Performance Computing and Communications Conference, IPCCC 2011
DOIs
StatePublished - 1 Dec 2011
Event30th IEEE International Performance, Computing and Communications Conference, IPCCC 2011 - Orlando, FL, United States
Duration: 17 Nov 201119 Nov 2011

Publication series

NameConference Proceedings of the IEEE International Performance, Computing, and Communications Conference

Other

Other30th IEEE International Performance, Computing and Communications Conference, IPCCC 2011
CountryUnited States
CityOrlando, FL
Period17/11/1119/11/11

Keywords

  • distributed computing
  • performance modeling
  • scientific workflow
  • workflow system

Fingerprint Dive into the research topics of 'A distributed workflow management system with case study of real-life scientific applications'. Together they form a unique fingerprint.

Cite this