Concurrent Bandwidth Reservation Strategies for Big Data Transfers in High-Performance Networks

Liudong Zuo, Michelle Zhu

Research output: Contribution to journalArticle

16 Citations (Scopus)

Abstract

Because of the deployment of large-scale experimental and computational scientific applications, big data is being generated on a daily basis. Such large volumes of data usually need to be transferred from the data generating center to remotely located scientific sites for collaborative data analysis in a timely manner. Bandwidth reservation along paths provisioned by dedicated high-performance networks (HPNs) has proved to be a fast, reliable, and predictable way to satisfy the transfer requirements of massive time-sensitive data. In this paper, we study the problem of scheduling multiple bandwidth reservation requests (BRRs) concurrently within an HPN while achieving their best average transfer performance. Two common data transfer performance parameters are considered: the Earliest Completion Time (ECT) and the Shortest Duration (SD). Since not all BRRs in one batch can oftentimes be successfully scheduled, the problem of scheduling all BRRs in one batch while achieving their best average ECT and SD are converted into the problem of scheduling as many BRRs as possible while achieving the average ECT and SD of scheduled BRRs, respectively. The aforementioned two problems are proved to be NP-complete problems. Two fast and efficient heuristic algorithms with polynomial-time complexity are proposed. Extensive simulation experiments are conducted to compare their performance with two proposed naive algorithms in various performance metrics. Performance superiority of these two fast and efficient algorithms is verified.

Original languageEnglish
Article number7103032
Pages (from-to)232-247
Number of pages16
JournalIEEE Transactions on Network and Service Management
Volume12
Issue number2
DOIs
StatePublished - 1 Jun 2015

Fingerprint

Data transfer
Network performance
Bandwidth
Scheduling
Heuristic algorithms
Big data
Computational complexity
Polynomials
Experiments

Keywords

  • Bandwidth reservation
  • QoS
  • big data
  • dynamic provisioning
  • high-performance networks

Cite this

@article{5e7627d05ece464bb3535be2b85241b2,
title = "Concurrent Bandwidth Reservation Strategies for Big Data Transfers in High-Performance Networks",
abstract = "Because of the deployment of large-scale experimental and computational scientific applications, big data is being generated on a daily basis. Such large volumes of data usually need to be transferred from the data generating center to remotely located scientific sites for collaborative data analysis in a timely manner. Bandwidth reservation along paths provisioned by dedicated high-performance networks (HPNs) has proved to be a fast, reliable, and predictable way to satisfy the transfer requirements of massive time-sensitive data. In this paper, we study the problem of scheduling multiple bandwidth reservation requests (BRRs) concurrently within an HPN while achieving their best average transfer performance. Two common data transfer performance parameters are considered: the Earliest Completion Time (ECT) and the Shortest Duration (SD). Since not all BRRs in one batch can oftentimes be successfully scheduled, the problem of scheduling all BRRs in one batch while achieving their best average ECT and SD are converted into the problem of scheduling as many BRRs as possible while achieving the average ECT and SD of scheduled BRRs, respectively. The aforementioned two problems are proved to be NP-complete problems. Two fast and efficient heuristic algorithms with polynomial-time complexity are proposed. Extensive simulation experiments are conducted to compare their performance with two proposed naive algorithms in various performance metrics. Performance superiority of these two fast and efficient algorithms is verified.",
keywords = "Bandwidth reservation, QoS, big data, dynamic provisioning, high-performance networks",
author = "Liudong Zuo and Michelle Zhu",
year = "2015",
month = "6",
day = "1",
doi = "10.1109/TNSM.2015.2430358",
language = "English",
volume = "12",
pages = "232--247",
journal = "IEEE Transactions on Network and Service Management",
issn = "1932-4537",
publisher = "Institute of Electrical and Electronics Engineers Inc.",
number = "2",

}

Concurrent Bandwidth Reservation Strategies for Big Data Transfers in High-Performance Networks. / Zuo, Liudong; Zhu, Michelle.

In: IEEE Transactions on Network and Service Management, Vol. 12, No. 2, 7103032, 01.06.2015, p. 232-247.

Research output: Contribution to journalArticle

TY - JOUR

T1 - Concurrent Bandwidth Reservation Strategies for Big Data Transfers in High-Performance Networks

AU - Zuo, Liudong

AU - Zhu, Michelle

PY - 2015/6/1

Y1 - 2015/6/1

N2 - Because of the deployment of large-scale experimental and computational scientific applications, big data is being generated on a daily basis. Such large volumes of data usually need to be transferred from the data generating center to remotely located scientific sites for collaborative data analysis in a timely manner. Bandwidth reservation along paths provisioned by dedicated high-performance networks (HPNs) has proved to be a fast, reliable, and predictable way to satisfy the transfer requirements of massive time-sensitive data. In this paper, we study the problem of scheduling multiple bandwidth reservation requests (BRRs) concurrently within an HPN while achieving their best average transfer performance. Two common data transfer performance parameters are considered: the Earliest Completion Time (ECT) and the Shortest Duration (SD). Since not all BRRs in one batch can oftentimes be successfully scheduled, the problem of scheduling all BRRs in one batch while achieving their best average ECT and SD are converted into the problem of scheduling as many BRRs as possible while achieving the average ECT and SD of scheduled BRRs, respectively. The aforementioned two problems are proved to be NP-complete problems. Two fast and efficient heuristic algorithms with polynomial-time complexity are proposed. Extensive simulation experiments are conducted to compare their performance with two proposed naive algorithms in various performance metrics. Performance superiority of these two fast and efficient algorithms is verified.

AB - Because of the deployment of large-scale experimental and computational scientific applications, big data is being generated on a daily basis. Such large volumes of data usually need to be transferred from the data generating center to remotely located scientific sites for collaborative data analysis in a timely manner. Bandwidth reservation along paths provisioned by dedicated high-performance networks (HPNs) has proved to be a fast, reliable, and predictable way to satisfy the transfer requirements of massive time-sensitive data. In this paper, we study the problem of scheduling multiple bandwidth reservation requests (BRRs) concurrently within an HPN while achieving their best average transfer performance. Two common data transfer performance parameters are considered: the Earliest Completion Time (ECT) and the Shortest Duration (SD). Since not all BRRs in one batch can oftentimes be successfully scheduled, the problem of scheduling all BRRs in one batch while achieving their best average ECT and SD are converted into the problem of scheduling as many BRRs as possible while achieving the average ECT and SD of scheduled BRRs, respectively. The aforementioned two problems are proved to be NP-complete problems. Two fast and efficient heuristic algorithms with polynomial-time complexity are proposed. Extensive simulation experiments are conducted to compare their performance with two proposed naive algorithms in various performance metrics. Performance superiority of these two fast and efficient algorithms is verified.

KW - Bandwidth reservation

KW - QoS

KW - big data

KW - dynamic provisioning

KW - high-performance networks

UR - http://www.scopus.com/inward/record.url?scp=84933533476&partnerID=8YFLogxK

U2 - 10.1109/TNSM.2015.2430358

DO - 10.1109/TNSM.2015.2430358

M3 - Article

AN - SCOPUS:84933533476

VL - 12

SP - 232

EP - 247

JO - IEEE Transactions on Network and Service Management

JF - IEEE Transactions on Network and Service Management

SN - 1932-4537

IS - 2

M1 - 7103032

ER -