Network-aware data movement advisor

Patrick Brown, Mengxia Zhu, Qishi Wu, Xukang Lu

Research output: Chapter in Book/Report/Conference proceedingConference contribution

4 Scopus citations

Abstract

Next-generation eScience applications often generate large amounts of simulation or experimental data that must be shared and managed by collaborative organizations. Advanced networking technologies and services have been rapidly developed and deployed to facilitate the massive data transport necessary for such data sharing and collaboration. However, these technologies and services have not been fully utilized by application users mainly because their use typically requires significant domain knowledge and in many cases even their existence is not made aware to the public. We design and develop a Network-aware Data Movement Advisor (NADMA) utility to enable automated discovery of network and system resources and advise the user of efficient strategies for fast and successful data transfer. NADMA is primarily a client-end program that interacts with existing data/space management and discovery services such as Storage Resource Management, transport methods such as GridFTP, and network resource provisioning systems such as TeraPaths and OSCARS. NADMA acts as a route planner in a typical vehicle navigation system to provide the user a set of feasible route options along with performance estimations as well as specific steps and commands to authorize and execute data transfer. We demonstrate the efficacy of NADMA in several use cases based on its implementation and deployment in wide-area networks.

Original languageEnglish
Title of host publicationNDM'11 - Proceedings of the 2011 International Workshop on Network-Aware Data Management, Co-located with SC'11
Pages31-39
Number of pages9
DOIs
StatePublished - 1 Dec 2011
EventInternational Workshop on Network-Aware Data Management, NDM'11, Held in Conjunction with the International Conference for High Performance Computing, Networking, Storage and Analysis, SC'11 - Seattle, WA, United States
Duration: 14 Nov 201114 Nov 2011

Publication series

NameNDM'11 - Proceedings of the 2011 International Workshop on Network-Aware Data Management, Co-located with SC'11

Other

OtherInternational Workshop on Network-Aware Data Management, NDM'11, Held in Conjunction with the International Conference for High Performance Computing, Networking, Storage and Analysis, SC'11
CountryUnited States
CitySeattle, WA
Period14/11/1114/11/11

Keywords

  • Data transfer
  • High-performance network

Fingerprint Dive into the research topics of 'Network-aware data movement advisor'. Together they form a unique fingerprint.

  • Cite this

    Brown, P., Zhu, M., Wu, Q., & Lu, X. (2011). Network-aware data movement advisor. In NDM'11 - Proceedings of the 2011 International Workshop on Network-Aware Data Management, Co-located with SC'11 (pp. 31-39). (NDM'11 - Proceedings of the 2011 International Workshop on Network-Aware Data Management, Co-located with SC'11). https://doi.org/10.1145/2110217.2110222