Scheduling data processing flows under budget constraint on the cloud

Fei Cao, Dabin Ding, Dunren Che, Michelle M. Zhu, Wen Chi Hou

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Cloud computing is emerging as a promising paradigm for large-scale data-intensive queries modeled as complex Directed Acyclic Graph (DAG)-structured dataflows with arbitrary data operators as nodes and producer-consumer interactions as directed edges. The optimization problem of scheduling dataflows on the Cloud is a very complex and challenging task which is similar to query optimization. Optimization must satisfy a variety of objectives and constraints, while taking into account the particular characteristics of the underlying Cloud environment. In addition to achieving minimum completion time, the commercialization of Clouds requires policies to take users' economic concerns as well. In this paper, we formulate scheduling of dataflows onto Cloud resources under the objective of minimizing the completion time under certain budget constraint. A heuristic scheduling algorithm, Layer-oriented Resource Allocation within Budget constraint (LRA-B) is proposed and evaluated. Experiments are conducted on numerous dataflows and Cloud environment configurations, and the overall results are quite promising and indicate the effectiveness of our algorithm.

Original languageEnglish
Title of host publicationProceedings of the 2013 Research in Adaptive and Convergent Systems, RACS 2013
Pages69-74
Number of pages6
DOIs
StatePublished - 1 Dec 2013
Event2013 Research in Adaptive and Convergent Systems, RACS 2013 - Montreal, QC, Canada
Duration: 1 Oct 20134 Oct 2013

Publication series

NameProceedings of the 2013 Research in Adaptive and Convergent Systems, RACS 2013

Other

Other2013 Research in Adaptive and Convergent Systems, RACS 2013
CountryCanada
CityMontreal, QC
Period1/10/134/10/13

Keywords

  • budget constraint
  • cloud computing
  • completion time
  • dataflows
  • scheduling

Fingerprint Dive into the research topics of 'Scheduling data processing flows under budget constraint on the cloud'. Together they form a unique fingerprint.

  • Cite this

    Cao, F., Ding, D., Che, D., Zhu, M. M., & Hou, W. C. (2013). Scheduling data processing flows under budget constraint on the cloud. In Proceedings of the 2013 Research in Adaptive and Convergent Systems, RACS 2013 (pp. 69-74). (Proceedings of the 2013 Research in Adaptive and Convergent Systems, RACS 2013). https://doi.org/10.1145/2513228.2513250