QoS provisioning for various types of deadline-constrained bulk data transfers between data centers

Aiqin Hou, Chase Q. Wu, Ruimin Qiao, Liudong Zuo, Michelle M. Zhu, Dingyi Fang, Weike Nie, Feng Chen

Research output: Contribution to journalArticle

Abstract

An increasing number of applications in scientific and other domains have moved or are in active transition to clouds, and the demand for big data transfers between geographically distributed cloud-based data centers is rapidly growing. Many modern backbone networks leverage logically centralized controllers based on software-defined networking (SDN) to provide advance bandwidth reservation for data transfer requests. How to fully utilize the bandwidth resources of the links connecting data centers with guaranteed quality of service for each user request is an important problem for cloud service providers. Most existing work focuses on bandwidth scheduling for a single request for data transfer or multiple requests using the same service model. In this work, we construct rigorous cost models to quantify user satisfaction degree, and formulate a generic problem of bandwidth scheduling for multiple deadline-constrained data transfer requests of different types to maximize the request scheduling success ratio while minimizing the data transfer completion time of each request. We prove this problem to be not only NP-complete but also non-approximable, and hence design a heuristic algorithm. For performance evaluation, we establish a proof-of-concept emulated SDN testbed and also generate large-scale simulation networks. Both experimental and simulation results show that the proposed scheduling scheme significantly outperforms existing methods in terms of user satisfaction degree and scheduling success ratio.

Original languageEnglish
Pages (from-to)162-174
Number of pages13
JournalFuture Generation Computer Systems
Volume105
DOIs
StatePublished - Apr 2020

Fingerprint

Data transfer
Quality of service
Scheduling
Bandwidth
Heuristic algorithms
Testbeds
Telecommunication links
Controllers
Costs
Software defined networking

Keywords

  • Bandwidth scheduling
  • Big data
  • Data center
  • High-performance networks
  • Software-defined networking

Cite this

Hou, Aiqin ; Wu, Chase Q. ; Qiao, Ruimin ; Zuo, Liudong ; Zhu, Michelle M. ; Fang, Dingyi ; Nie, Weike ; Chen, Feng. / QoS provisioning for various types of deadline-constrained bulk data transfers between data centers. In: Future Generation Computer Systems. 2020 ; Vol. 105. pp. 162-174.
@article{9525718cc78144db8b4f652b4bc7b536,
title = "QoS provisioning for various types of deadline-constrained bulk data transfers between data centers",
abstract = "An increasing number of applications in scientific and other domains have moved or are in active transition to clouds, and the demand for big data transfers between geographically distributed cloud-based data centers is rapidly growing. Many modern backbone networks leverage logically centralized controllers based on software-defined networking (SDN) to provide advance bandwidth reservation for data transfer requests. How to fully utilize the bandwidth resources of the links connecting data centers with guaranteed quality of service for each user request is an important problem for cloud service providers. Most existing work focuses on bandwidth scheduling for a single request for data transfer or multiple requests using the same service model. In this work, we construct rigorous cost models to quantify user satisfaction degree, and formulate a generic problem of bandwidth scheduling for multiple deadline-constrained data transfer requests of different types to maximize the request scheduling success ratio while minimizing the data transfer completion time of each request. We prove this problem to be not only NP-complete but also non-approximable, and hence design a heuristic algorithm. For performance evaluation, we establish a proof-of-concept emulated SDN testbed and also generate large-scale simulation networks. Both experimental and simulation results show that the proposed scheduling scheme significantly outperforms existing methods in terms of user satisfaction degree and scheduling success ratio.",
keywords = "Bandwidth scheduling, Big data, Data center, High-performance networks, Software-defined networking",
author = "Aiqin Hou and Wu, {Chase Q.} and Ruimin Qiao and Liudong Zuo and Zhu, {Michelle M.} and Dingyi Fang and Weike Nie and Feng Chen",
year = "2020",
month = "4",
doi = "10.1016/j.future.2019.11.039",
language = "English",
volume = "105",
pages = "162--174",
journal = "Future Generation Computer Systems",
issn = "0167-739X",
publisher = "Elsevier",

}

QoS provisioning for various types of deadline-constrained bulk data transfers between data centers. / Hou, Aiqin; Wu, Chase Q.; Qiao, Ruimin; Zuo, Liudong; Zhu, Michelle M.; Fang, Dingyi; Nie, Weike; Chen, Feng.

In: Future Generation Computer Systems, Vol. 105, 04.2020, p. 162-174.

Research output: Contribution to journalArticle

TY - JOUR

T1 - QoS provisioning for various types of deadline-constrained bulk data transfers between data centers

AU - Hou, Aiqin

AU - Wu, Chase Q.

AU - Qiao, Ruimin

AU - Zuo, Liudong

AU - Zhu, Michelle M.

AU - Fang, Dingyi

AU - Nie, Weike

AU - Chen, Feng

PY - 2020/4

Y1 - 2020/4

N2 - An increasing number of applications in scientific and other domains have moved or are in active transition to clouds, and the demand for big data transfers between geographically distributed cloud-based data centers is rapidly growing. Many modern backbone networks leverage logically centralized controllers based on software-defined networking (SDN) to provide advance bandwidth reservation for data transfer requests. How to fully utilize the bandwidth resources of the links connecting data centers with guaranteed quality of service for each user request is an important problem for cloud service providers. Most existing work focuses on bandwidth scheduling for a single request for data transfer or multiple requests using the same service model. In this work, we construct rigorous cost models to quantify user satisfaction degree, and formulate a generic problem of bandwidth scheduling for multiple deadline-constrained data transfer requests of different types to maximize the request scheduling success ratio while minimizing the data transfer completion time of each request. We prove this problem to be not only NP-complete but also non-approximable, and hence design a heuristic algorithm. For performance evaluation, we establish a proof-of-concept emulated SDN testbed and also generate large-scale simulation networks. Both experimental and simulation results show that the proposed scheduling scheme significantly outperforms existing methods in terms of user satisfaction degree and scheduling success ratio.

AB - An increasing number of applications in scientific and other domains have moved or are in active transition to clouds, and the demand for big data transfers between geographically distributed cloud-based data centers is rapidly growing. Many modern backbone networks leverage logically centralized controllers based on software-defined networking (SDN) to provide advance bandwidth reservation for data transfer requests. How to fully utilize the bandwidth resources of the links connecting data centers with guaranteed quality of service for each user request is an important problem for cloud service providers. Most existing work focuses on bandwidth scheduling for a single request for data transfer or multiple requests using the same service model. In this work, we construct rigorous cost models to quantify user satisfaction degree, and formulate a generic problem of bandwidth scheduling for multiple deadline-constrained data transfer requests of different types to maximize the request scheduling success ratio while minimizing the data transfer completion time of each request. We prove this problem to be not only NP-complete but also non-approximable, and hence design a heuristic algorithm. For performance evaluation, we establish a proof-of-concept emulated SDN testbed and also generate large-scale simulation networks. Both experimental and simulation results show that the proposed scheduling scheme significantly outperforms existing methods in terms of user satisfaction degree and scheduling success ratio.

KW - Bandwidth scheduling

KW - Big data

KW - Data center

KW - High-performance networks

KW - Software-defined networking

UR - http://www.scopus.com/inward/record.url?scp=85075971010&partnerID=8YFLogxK

U2 - 10.1016/j.future.2019.11.039

DO - 10.1016/j.future.2019.11.039

M3 - Article

AN - SCOPUS:85075971010

VL - 105

SP - 162

EP - 174

JO - Future Generation Computer Systems

JF - Future Generation Computer Systems

SN - 0167-739X

ER -