A cost-effective scheduling algorithm for scientific workflows in clouds

Michelle Zhu, Qishi Wu, Yang Zhao

Research output: Chapter in Book/Report/Conference proceedingConference contributionResearchpeer-review

26 Citations (Scopus)

Abstract

Cloud computing enables the delivery of computing, software, storage, and data access through web browsers as a metered service. In addition to commercial applications, an increasing number of large-scale workflow-based scientific applications are being supported by cloud computing. In order to meet the rapidly growing and dynamic computing demands of scientific users, the cloud service provider needs to employ efficient and cost-effective job schedulers to guarantee workflow completion time as well as improve resource utilization for high throughput. Based on rigorous cost models, we formulate a delay-constrained optimization problem to maximize resource utilization and propose a two-step workflow scheduling algorithm to minimize the cloud overhead within a user-specified execution time bound. The extensive simulation results illustrate that our approach consistently achieves lower computing overhead or higher resource utilization than existing methods within the execution time bound. Our approach also significantly reduces the total execution time by strategically selecting appropriate mapping nodes for prioritized modules.

Original languageEnglish
Title of host publication2012 IEEE 31st International Performance Computing and Communications Conference, IPCCC 2012
Pages256-265
Number of pages10
DOIs
StatePublished - 1 Dec 2012
Event2012 IEEE 31st International Performance Computing and Communications Conference, IPCCC 2012 - Austin, TX, United States
Duration: 1 Dec 20123 Dec 2012

Other

Other2012 IEEE 31st International Performance Computing and Communications Conference, IPCCC 2012
CountryUnited States
CityAustin, TX
Period1/12/123/12/12

Fingerprint

Cloud computing
Scheduling algorithms
Web browsers
Constrained optimization
Costs
Throughput

Keywords

  • cloud computing
  • Scientific workflow
  • workflow scheduling

Cite this

Zhu, M., Wu, Q., & Zhao, Y. (2012). A cost-effective scheduling algorithm for scientific workflows in clouds. In 2012 IEEE 31st International Performance Computing and Communications Conference, IPCCC 2012 (pp. 256-265). [6407766] https://doi.org/10.1109/PCCC.2012.6407766
Zhu, Michelle ; Wu, Qishi ; Zhao, Yang. / A cost-effective scheduling algorithm for scientific workflows in clouds. 2012 IEEE 31st International Performance Computing and Communications Conference, IPCCC 2012. 2012. pp. 256-265
@inproceedings{03c14fabce3240858694f8a49becd6cc,
title = "A cost-effective scheduling algorithm for scientific workflows in clouds",
abstract = "Cloud computing enables the delivery of computing, software, storage, and data access through web browsers as a metered service. In addition to commercial applications, an increasing number of large-scale workflow-based scientific applications are being supported by cloud computing. In order to meet the rapidly growing and dynamic computing demands of scientific users, the cloud service provider needs to employ efficient and cost-effective job schedulers to guarantee workflow completion time as well as improve resource utilization for high throughput. Based on rigorous cost models, we formulate a delay-constrained optimization problem to maximize resource utilization and propose a two-step workflow scheduling algorithm to minimize the cloud overhead within a user-specified execution time bound. The extensive simulation results illustrate that our approach consistently achieves lower computing overhead or higher resource utilization than existing methods within the execution time bound. Our approach also significantly reduces the total execution time by strategically selecting appropriate mapping nodes for prioritized modules.",
keywords = "cloud computing, Scientific workflow, workflow scheduling",
author = "Michelle Zhu and Qishi Wu and Yang Zhao",
year = "2012",
month = "12",
day = "1",
doi = "10.1109/PCCC.2012.6407766",
language = "English",
isbn = "9781467348812",
pages = "256--265",
booktitle = "2012 IEEE 31st International Performance Computing and Communications Conference, IPCCC 2012",

}

Zhu, M, Wu, Q & Zhao, Y 2012, A cost-effective scheduling algorithm for scientific workflows in clouds. in 2012 IEEE 31st International Performance Computing and Communications Conference, IPCCC 2012., 6407766, pp. 256-265, 2012 IEEE 31st International Performance Computing and Communications Conference, IPCCC 2012, Austin, TX, United States, 1/12/12. https://doi.org/10.1109/PCCC.2012.6407766

A cost-effective scheduling algorithm for scientific workflows in clouds. / Zhu, Michelle; Wu, Qishi; Zhao, Yang.

2012 IEEE 31st International Performance Computing and Communications Conference, IPCCC 2012. 2012. p. 256-265 6407766.

Research output: Chapter in Book/Report/Conference proceedingConference contributionResearchpeer-review

TY - GEN

T1 - A cost-effective scheduling algorithm for scientific workflows in clouds

AU - Zhu, Michelle

AU - Wu, Qishi

AU - Zhao, Yang

PY - 2012/12/1

Y1 - 2012/12/1

N2 - Cloud computing enables the delivery of computing, software, storage, and data access through web browsers as a metered service. In addition to commercial applications, an increasing number of large-scale workflow-based scientific applications are being supported by cloud computing. In order to meet the rapidly growing and dynamic computing demands of scientific users, the cloud service provider needs to employ efficient and cost-effective job schedulers to guarantee workflow completion time as well as improve resource utilization for high throughput. Based on rigorous cost models, we formulate a delay-constrained optimization problem to maximize resource utilization and propose a two-step workflow scheduling algorithm to minimize the cloud overhead within a user-specified execution time bound. The extensive simulation results illustrate that our approach consistently achieves lower computing overhead or higher resource utilization than existing methods within the execution time bound. Our approach also significantly reduces the total execution time by strategically selecting appropriate mapping nodes for prioritized modules.

AB - Cloud computing enables the delivery of computing, software, storage, and data access through web browsers as a metered service. In addition to commercial applications, an increasing number of large-scale workflow-based scientific applications are being supported by cloud computing. In order to meet the rapidly growing and dynamic computing demands of scientific users, the cloud service provider needs to employ efficient and cost-effective job schedulers to guarantee workflow completion time as well as improve resource utilization for high throughput. Based on rigorous cost models, we formulate a delay-constrained optimization problem to maximize resource utilization and propose a two-step workflow scheduling algorithm to minimize the cloud overhead within a user-specified execution time bound. The extensive simulation results illustrate that our approach consistently achieves lower computing overhead or higher resource utilization than existing methods within the execution time bound. Our approach also significantly reduces the total execution time by strategically selecting appropriate mapping nodes for prioritized modules.

KW - cloud computing

KW - Scientific workflow

KW - workflow scheduling

UR - http://www.scopus.com/inward/record.url?scp=84874288724&partnerID=8YFLogxK

U2 - 10.1109/PCCC.2012.6407766

DO - 10.1109/PCCC.2012.6407766

M3 - Conference contribution

SN - 9781467348812

SP - 256

EP - 265

BT - 2012 IEEE 31st International Performance Computing and Communications Conference, IPCCC 2012

ER -

Zhu M, Wu Q, Zhao Y. A cost-effective scheduling algorithm for scientific workflows in clouds. In 2012 IEEE 31st International Performance Computing and Communications Conference, IPCCC 2012. 2012. p. 256-265. 6407766 https://doi.org/10.1109/PCCC.2012.6407766