Bandwidth provision strategies for reliable data movements in dedicated networks

Liudong Zuo, Michelle Zhu

Research output: Chapter in Book/Report/Conference proceedingConference contributionResearchpeer-review

2 Citations (Scopus)

Abstract

Sheer volumes of data are being generated in extreme-scale distributed scientific applications, and need to be transferred remotely in fast, predictable and reliable way for data storage and analysis purpose. Reserving bandwidth along selected paths in high-performance networks (HPNs) has proved to be an effective way to satisfy the high-demanding performance requirements of such data transfer. However, node and link failures within the HPNs potentially degrade the quality of data transfer. In this paper, we focus on the scheduling of two generic types of bandwidth reservation requests concerning data transfer reliability: (i) to achieve the highest data transfer reliability under a given data transfer deadline, and (ii) to achieve the earliest data transfer completion time while satisfying a given data transfer reliability requirement. Poisson distribution is used to model the node and failures within the HPNs, and two periodic bandwidth reservation algorithms with rigorous optimality proofs are proposed.

Original languageEnglish
Title of host publicationProceedings - 2016 IEEE International Conference on Big Data, Big Data 2016
EditorsRonay Ak, George Karypis, Yinglong Xia, Xiaohua Tony Hu, Philip S. Yu, James Joshi, Lyle Ungar, Ling Liu, Aki-Hiro Sato, Toyotaro Suzumura, Sudarsan Rachuri, Rama Govindaraju, Weijia Xu
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages3069-3078
Number of pages10
ISBN (Electronic)9781467390040
DOIs
StatePublished - 1 Jan 2016
Event4th IEEE International Conference on Big Data, Big Data 2016 - Washington, United States
Duration: 5 Dec 20168 Dec 2016

Other

Other4th IEEE International Conference on Big Data, Big Data 2016
CountryUnited States
CityWashington
Period5/12/168/12/16

Fingerprint

Data transfer
Bandwidth
Network performance
Poisson distribution
Scheduling
Data storage equipment

Keywords

  • Bandwidth reservation
  • bandwidth scheduling
  • fault tolerance
  • high-performance networks

Cite this

Zuo, L., & Zhu, M. (2016). Bandwidth provision strategies for reliable data movements in dedicated networks. In R. Ak, G. Karypis, Y. Xia, X. T. Hu, P. S. Yu, J. Joshi, L. Ungar, L. Liu, A-H. Sato, T. Suzumura, S. Rachuri, R. Govindaraju, ... W. Xu (Eds.), Proceedings - 2016 IEEE International Conference on Big Data, Big Data 2016 (pp. 3069-3078). [7840961] Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/BigData.2016.7840961
Zuo, Liudong ; Zhu, Michelle. / Bandwidth provision strategies for reliable data movements in dedicated networks. Proceedings - 2016 IEEE International Conference on Big Data, Big Data 2016. editor / Ronay Ak ; George Karypis ; Yinglong Xia ; Xiaohua Tony Hu ; Philip S. Yu ; James Joshi ; Lyle Ungar ; Ling Liu ; Aki-Hiro Sato ; Toyotaro Suzumura ; Sudarsan Rachuri ; Rama Govindaraju ; Weijia Xu. Institute of Electrical and Electronics Engineers Inc., 2016. pp. 3069-3078
@inproceedings{07c953e2bd1c4b91aca71839902ab8c5,
title = "Bandwidth provision strategies for reliable data movements in dedicated networks",
abstract = "Sheer volumes of data are being generated in extreme-scale distributed scientific applications, and need to be transferred remotely in fast, predictable and reliable way for data storage and analysis purpose. Reserving bandwidth along selected paths in high-performance networks (HPNs) has proved to be an effective way to satisfy the high-demanding performance requirements of such data transfer. However, node and link failures within the HPNs potentially degrade the quality of data transfer. In this paper, we focus on the scheduling of two generic types of bandwidth reservation requests concerning data transfer reliability: (i) to achieve the highest data transfer reliability under a given data transfer deadline, and (ii) to achieve the earliest data transfer completion time while satisfying a given data transfer reliability requirement. Poisson distribution is used to model the node and failures within the HPNs, and two periodic bandwidth reservation algorithms with rigorous optimality proofs are proposed.",
keywords = "Bandwidth reservation, bandwidth scheduling, fault tolerance, high-performance networks",
author = "Liudong Zuo and Michelle Zhu",
year = "2016",
month = "1",
day = "1",
doi = "10.1109/BigData.2016.7840961",
language = "English",
pages = "3069--3078",
editor = "Ronay Ak and George Karypis and Yinglong Xia and Hu, {Xiaohua Tony} and Yu, {Philip S.} and James Joshi and Lyle Ungar and Ling Liu and Aki-Hiro Sato and Toyotaro Suzumura and Sudarsan Rachuri and Rama Govindaraju and Weijia Xu",
booktitle = "Proceedings - 2016 IEEE International Conference on Big Data, Big Data 2016",
publisher = "Institute of Electrical and Electronics Engineers Inc.",

}

Zuo, L & Zhu, M 2016, Bandwidth provision strategies for reliable data movements in dedicated networks. in R Ak, G Karypis, Y Xia, XT Hu, PS Yu, J Joshi, L Ungar, L Liu, A-H Sato, T Suzumura, S Rachuri, R Govindaraju & W Xu (eds), Proceedings - 2016 IEEE International Conference on Big Data, Big Data 2016., 7840961, Institute of Electrical and Electronics Engineers Inc., pp. 3069-3078, 4th IEEE International Conference on Big Data, Big Data 2016, Washington, United States, 5/12/16. https://doi.org/10.1109/BigData.2016.7840961

Bandwidth provision strategies for reliable data movements in dedicated networks. / Zuo, Liudong; Zhu, Michelle.

Proceedings - 2016 IEEE International Conference on Big Data, Big Data 2016. ed. / Ronay Ak; George Karypis; Yinglong Xia; Xiaohua Tony Hu; Philip S. Yu; James Joshi; Lyle Ungar; Ling Liu; Aki-Hiro Sato; Toyotaro Suzumura; Sudarsan Rachuri; Rama Govindaraju; Weijia Xu. Institute of Electrical and Electronics Engineers Inc., 2016. p. 3069-3078 7840961.

Research output: Chapter in Book/Report/Conference proceedingConference contributionResearchpeer-review

TY - GEN

T1 - Bandwidth provision strategies for reliable data movements in dedicated networks

AU - Zuo, Liudong

AU - Zhu, Michelle

PY - 2016/1/1

Y1 - 2016/1/1

N2 - Sheer volumes of data are being generated in extreme-scale distributed scientific applications, and need to be transferred remotely in fast, predictable and reliable way for data storage and analysis purpose. Reserving bandwidth along selected paths in high-performance networks (HPNs) has proved to be an effective way to satisfy the high-demanding performance requirements of such data transfer. However, node and link failures within the HPNs potentially degrade the quality of data transfer. In this paper, we focus on the scheduling of two generic types of bandwidth reservation requests concerning data transfer reliability: (i) to achieve the highest data transfer reliability under a given data transfer deadline, and (ii) to achieve the earliest data transfer completion time while satisfying a given data transfer reliability requirement. Poisson distribution is used to model the node and failures within the HPNs, and two periodic bandwidth reservation algorithms with rigorous optimality proofs are proposed.

AB - Sheer volumes of data are being generated in extreme-scale distributed scientific applications, and need to be transferred remotely in fast, predictable and reliable way for data storage and analysis purpose. Reserving bandwidth along selected paths in high-performance networks (HPNs) has proved to be an effective way to satisfy the high-demanding performance requirements of such data transfer. However, node and link failures within the HPNs potentially degrade the quality of data transfer. In this paper, we focus on the scheduling of two generic types of bandwidth reservation requests concerning data transfer reliability: (i) to achieve the highest data transfer reliability under a given data transfer deadline, and (ii) to achieve the earliest data transfer completion time while satisfying a given data transfer reliability requirement. Poisson distribution is used to model the node and failures within the HPNs, and two periodic bandwidth reservation algorithms with rigorous optimality proofs are proposed.

KW - Bandwidth reservation

KW - bandwidth scheduling

KW - fault tolerance

KW - high-performance networks

UR - http://www.scopus.com/inward/record.url?scp=85015224050&partnerID=8YFLogxK

U2 - 10.1109/BigData.2016.7840961

DO - 10.1109/BigData.2016.7840961

M3 - Conference contribution

SP - 3069

EP - 3078

BT - Proceedings - 2016 IEEE International Conference on Big Data, Big Data 2016

A2 - Ak, Ronay

A2 - Karypis, George

A2 - Xia, Yinglong

A2 - Hu, Xiaohua Tony

A2 - Yu, Philip S.

A2 - Joshi, James

A2 - Ungar, Lyle

A2 - Liu, Ling

A2 - Sato, Aki-Hiro

A2 - Suzumura, Toyotaro

A2 - Rachuri, Sudarsan

A2 - Govindaraju, Rama

A2 - Xu, Weijia

PB - Institute of Electrical and Electronics Engineers Inc.

ER -

Zuo L, Zhu M. Bandwidth provision strategies for reliable data movements in dedicated networks. In Ak R, Karypis G, Xia Y, Hu XT, Yu PS, Joshi J, Ungar L, Liu L, Sato A-H, Suzumura T, Rachuri S, Govindaraju R, Xu W, editors, Proceedings - 2016 IEEE International Conference on Big Data, Big Data 2016. Institute of Electrical and Electronics Engineers Inc. 2016. p. 3069-3078. 7840961 https://doi.org/10.1109/BigData.2016.7840961