Modeling and optimizing transport-support workflows in high-performance networks

Daqing Yun, Qishi Wu, Patrick Brown, Michelle Zhu

Research output: Chapter in Book/Report/Conference proceedingConference contribution

2 Citations (Scopus)

Abstract

High-performance networking technologies and services are being rapidly developed and deployed across the nation and around the globe to support the transfer of large data sets generated by next-generation scientific applications for collaborative data processing, analysis, and storage. However, these networking technologies and services have not been fully utilized mainly because their use often requires considerable domain knowledge and many application users are even not aware of their existence. The main goal of our work is to provide end users an integrated solution to discovering system and network resources and composing end-to-end paths for large data transfer. By leveraging the resource discovery capability previously developed in Network-Aware Data Movement Advisor (NADMA), we propose novel profiling and modeling approaches to characterize various types of resources that are available in end systems, edge segments, and backbone networks, taking into consideration a comprehensive set of performance metrics and network parameters in different phases including device deployment, circuit setup, and data transfer. Based on these profiles and models, we formulate a class of transport-support workflow optimization problems where an appropriate set of technologies and services are selected to compose the best transport-support workflow to meet the user's data transfer request in terms of various performance requirements. We conduct wide-area network experiments to validate the cost models and illustrate the efficacy of the proposed workflow-based transport solution.

Original languageEnglish
Title of host publicationProceedings of the 37th Annual IEEE Conference on Local Computer Networks, LCN 2012
Pages384-391
Number of pages8
DOIs
StatePublished - 1 Dec 2012
Event37th Annual IEEE Conference on Local Computer Networks, LCN 2012 - Clearwater, FL, United States
Duration: 22 Oct 201225 Oct 2012

Publication series

NameProceedings - Conference on Local Computer Networks, LCN

Other

Other37th Annual IEEE Conference on Local Computer Networks, LCN 2012
CountryUnited States
CityClearwater, FL
Period22/10/1225/10/12

Fingerprint

Data transfer
Network performance
Wide area networks
Networks (circuits)
Costs
Experiments

Keywords

  • Service model
  • high-performance networking
  • performance modeling
  • workflow

Cite this

Yun, D., Wu, Q., Brown, P., & Zhu, M. (2012). Modeling and optimizing transport-support workflows in high-performance networks. In Proceedings of the 37th Annual IEEE Conference on Local Computer Networks, LCN 2012 (pp. 384-391). [6423652] (Proceedings - Conference on Local Computer Networks, LCN). https://doi.org/10.1109/LCN.2012.6423652
Yun, Daqing ; Wu, Qishi ; Brown, Patrick ; Zhu, Michelle. / Modeling and optimizing transport-support workflows in high-performance networks. Proceedings of the 37th Annual IEEE Conference on Local Computer Networks, LCN 2012. 2012. pp. 384-391 (Proceedings - Conference on Local Computer Networks, LCN).
@inproceedings{8d147766ba61417ba22875042466129e,
title = "Modeling and optimizing transport-support workflows in high-performance networks",
abstract = "High-performance networking technologies and services are being rapidly developed and deployed across the nation and around the globe to support the transfer of large data sets generated by next-generation scientific applications for collaborative data processing, analysis, and storage. However, these networking technologies and services have not been fully utilized mainly because their use often requires considerable domain knowledge and many application users are even not aware of their existence. The main goal of our work is to provide end users an integrated solution to discovering system and network resources and composing end-to-end paths for large data transfer. By leveraging the resource discovery capability previously developed in Network-Aware Data Movement Advisor (NADMA), we propose novel profiling and modeling approaches to characterize various types of resources that are available in end systems, edge segments, and backbone networks, taking into consideration a comprehensive set of performance metrics and network parameters in different phases including device deployment, circuit setup, and data transfer. Based on these profiles and models, we formulate a class of transport-support workflow optimization problems where an appropriate set of technologies and services are selected to compose the best transport-support workflow to meet the user's data transfer request in terms of various performance requirements. We conduct wide-area network experiments to validate the cost models and illustrate the efficacy of the proposed workflow-based transport solution.",
keywords = "Service model, high-performance networking, performance modeling, workflow",
author = "Daqing Yun and Qishi Wu and Patrick Brown and Michelle Zhu",
year = "2012",
month = "12",
day = "1",
doi = "10.1109/LCN.2012.6423652",
language = "English",
isbn = "9781467315647",
series = "Proceedings - Conference on Local Computer Networks, LCN",
pages = "384--391",
booktitle = "Proceedings of the 37th Annual IEEE Conference on Local Computer Networks, LCN 2012",

}

Yun, D, Wu, Q, Brown, P & Zhu, M 2012, Modeling and optimizing transport-support workflows in high-performance networks. in Proceedings of the 37th Annual IEEE Conference on Local Computer Networks, LCN 2012., 6423652, Proceedings - Conference on Local Computer Networks, LCN, pp. 384-391, 37th Annual IEEE Conference on Local Computer Networks, LCN 2012, Clearwater, FL, United States, 22/10/12. https://doi.org/10.1109/LCN.2012.6423652

Modeling and optimizing transport-support workflows in high-performance networks. / Yun, Daqing; Wu, Qishi; Brown, Patrick; Zhu, Michelle.

Proceedings of the 37th Annual IEEE Conference on Local Computer Networks, LCN 2012. 2012. p. 384-391 6423652 (Proceedings - Conference on Local Computer Networks, LCN).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

TY - GEN

T1 - Modeling and optimizing transport-support workflows in high-performance networks

AU - Yun, Daqing

AU - Wu, Qishi

AU - Brown, Patrick

AU - Zhu, Michelle

PY - 2012/12/1

Y1 - 2012/12/1

N2 - High-performance networking technologies and services are being rapidly developed and deployed across the nation and around the globe to support the transfer of large data sets generated by next-generation scientific applications for collaborative data processing, analysis, and storage. However, these networking technologies and services have not been fully utilized mainly because their use often requires considerable domain knowledge and many application users are even not aware of their existence. The main goal of our work is to provide end users an integrated solution to discovering system and network resources and composing end-to-end paths for large data transfer. By leveraging the resource discovery capability previously developed in Network-Aware Data Movement Advisor (NADMA), we propose novel profiling and modeling approaches to characterize various types of resources that are available in end systems, edge segments, and backbone networks, taking into consideration a comprehensive set of performance metrics and network parameters in different phases including device deployment, circuit setup, and data transfer. Based on these profiles and models, we formulate a class of transport-support workflow optimization problems where an appropriate set of technologies and services are selected to compose the best transport-support workflow to meet the user's data transfer request in terms of various performance requirements. We conduct wide-area network experiments to validate the cost models and illustrate the efficacy of the proposed workflow-based transport solution.

AB - High-performance networking technologies and services are being rapidly developed and deployed across the nation and around the globe to support the transfer of large data sets generated by next-generation scientific applications for collaborative data processing, analysis, and storage. However, these networking technologies and services have not been fully utilized mainly because their use often requires considerable domain knowledge and many application users are even not aware of their existence. The main goal of our work is to provide end users an integrated solution to discovering system and network resources and composing end-to-end paths for large data transfer. By leveraging the resource discovery capability previously developed in Network-Aware Data Movement Advisor (NADMA), we propose novel profiling and modeling approaches to characterize various types of resources that are available in end systems, edge segments, and backbone networks, taking into consideration a comprehensive set of performance metrics and network parameters in different phases including device deployment, circuit setup, and data transfer. Based on these profiles and models, we formulate a class of transport-support workflow optimization problems where an appropriate set of technologies and services are selected to compose the best transport-support workflow to meet the user's data transfer request in terms of various performance requirements. We conduct wide-area network experiments to validate the cost models and illustrate the efficacy of the proposed workflow-based transport solution.

KW - Service model

KW - high-performance networking

KW - performance modeling

KW - workflow

UR - http://www.scopus.com/inward/record.url?scp=84874334494&partnerID=8YFLogxK

U2 - 10.1109/LCN.2012.6423652

DO - 10.1109/LCN.2012.6423652

M3 - Conference contribution

AN - SCOPUS:84874334494

SN - 9781467315647

T3 - Proceedings - Conference on Local Computer Networks, LCN

SP - 384

EP - 391

BT - Proceedings of the 37th Annual IEEE Conference on Local Computer Networks, LCN 2012

ER -

Yun D, Wu Q, Brown P, Zhu M. Modeling and optimizing transport-support workflows in high-performance networks. In Proceedings of the 37th Annual IEEE Conference on Local Computer Networks, LCN 2012. 2012. p. 384-391. 6423652. (Proceedings - Conference on Local Computer Networks, LCN). https://doi.org/10.1109/LCN.2012.6423652