Bandwidth Scheduling for Big Data Transfer with Deadline Constraint between Data Centers

Aiqin Hou, Chase Q. Wu, Dingyi Fang, Liudong Zuo, Michelle Zhu, Xiaoyang Zhang, Ruimin Qiao, Xiaoyan Yin

Research output: Chapter in Book/Report/Conference proceedingConference contribution

2 Scopus citations

Abstract

An increasing number of applications in scientific and other domains have moved or are in active transition to clouds, and the demand for the movement of big data between geographically distributed cloud-based data centers is rapidly growing. Many modern backbone networks leverage logically centralized controllers based on software-defined networking (SDN) to provide advance bandwidth reservation for data transfer requests. How to fully utilize the bandwidth resources of the links connecting data centers with guaranteed QoS for each user request is an important problem for cloud service providers. Most existing work focuses on bandwidth scheduling for a single request for data transfer or multiple requests using the same service model. In this work, we construct rigorous cost models to quantify user satisfaction degree and formulate a generic problem of bandwidth scheduling for multiple deadline-constrained data transfer requests of different types to maximize the request scheduling success ratio while minimizing the data transfer completion time of each request. We prove this problem to be NP-complete and design a heuristic solution. Extensive simulation results show that our scheduling scheme significantly outperforms existing methods in terms of user satisfaction degree and scheduling success ratio.

Original languageEnglish
Title of host publicationProceedings of INDIS 2018
Subtitle of host publicationInnovating the Network for Data-Intensive Science, Held in conjunction with SC 2018: The International Conference for High Performance Computing, Networking, Storage and Analysis
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages55-63
Number of pages9
ISBN (Electronic)9781728101941
DOIs
Publication statusPublished - 21 Feb 2019
Event2018 IEEE/ACM Innovating the Network for Data-Intensive Science, INDIS 2018 - Dallas, United States
Duration: 11 Nov 2018 → …

Publication series

NameProceedings of INDIS 2018: Innovating the Network for Data-Intensive Science, Held in conjunction with SC 2018: The International Conference for High Performance Computing, Networking, Storage and Analysis

Conference

Conference2018 IEEE/ACM Innovating the Network for Data-Intensive Science, INDIS 2018
CountryUnited States
CityDallas
Period11/11/18 → …

    Fingerprint

Keywords

  • Bandwidth-scheduling
  • Big-data
  • Data-center
  • High-performance-networks
  • Software-defined-networking

Cite this

Hou, A., Wu, C. Q., Fang, D., Zuo, L., Zhu, M., Zhang, X., ... Yin, X. (2019). Bandwidth Scheduling for Big Data Transfer with Deadline Constraint between Data Centers. In Proceedings of INDIS 2018: Innovating the Network for Data-Intensive Science, Held in conjunction with SC 2018: The International Conference for High Performance Computing, Networking, Storage and Analysis (pp. 55-63). [8648791] (Proceedings of INDIS 2018: Innovating the Network for Data-Intensive Science, Held in conjunction with SC 2018: The International Conference for High Performance Computing, Networking, Storage and Analysis). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/INDIS.2018.00009