Exploring the optimal strategy for large-scale data movement in high-performance networks

Patrick Brown, Mengxia Zhu, Qishi Wu, Daqing Yun, Jason Zurawski

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Advanced networking technologies and services have been rapidly developed and deployed to facilitate bulk data transfer so as to support next-generation eScience applications. However, these technologies and services have not been fully utilized due to the knowledge lack of scientific domain experts. By leveraging the functionalities of an existing data movement advising utility, we propose a new Workflow-based Intelligent Network Data Movement Advisor (WINDMA) with end-to-end performance optimization. WINDMA provides a web interface and interacts with existing data/space management and discovery services such as Storage Resource Management, transport methods such as GridFTP and GlobusOnline, and network resource provisioning brokers such as ION and OSCARS. Efficacy of WINDMA has been demonstrated in several use cases based on its implementation and deployment in wide-area networks.

Original languageEnglish
Title of host publication2012 IEEE 31st International Performance Computing and Communications Conference, IPCCC 2012
Pages181-182
Number of pages2
DOIs
StatePublished - 1 Dec 2012
Event2012 IEEE 31st International Performance Computing and Communications Conference, IPCCC 2012 - Austin, TX, United States
Duration: 1 Dec 20123 Dec 2012

Other

Other2012 IEEE 31st International Performance Computing and Communications Conference, IPCCC 2012
CountryUnited States
CityAustin, TX
Period1/12/123/12/12

Keywords

  • eScience applications
  • High-performance networks
  • large data transfer

Fingerprint Dive into the research topics of 'Exploring the optimal strategy for large-scale data movement in high-performance networks'. Together they form a unique fingerprint.

  • Cite this

    Brown, P., Zhu, M., Wu, Q., Yun, D., & Zurawski, J. (2012). Exploring the optimal strategy for large-scale data movement in high-performance networks. In 2012 IEEE 31st International Performance Computing and Communications Conference, IPCCC 2012 (pp. 181-182). [6407676] https://doi.org/10.1109/PCCC.2012.6407676