In an era of data explosion and analysis, researchers across the globe are trying to convert massive quantities of complex data into useful knowledge using Computational and Data-Enabled Science and Engineering (CDS&E) applications. CDS&E applications are gaining traction as an important dimension of Science, Technology, Engineering, and Mathematics research. These applications require powerful processors with fast interconnects, extreme large scale, and elastic resources. While high-end High Performance Computing (HPC) resources provide the necessary requirements, the complexity of such systems has grown exponentially. Furthermore, due to their high cost, limited availability, and high demand, the queue wait times to run applications on these systems is in the order of months. All of the above challenges prevent their adoption as a mainstream solution. On the other hand, Cloud computing is emerging as a dominant computing paradigm that offers many advantages. Consequently, early adopters have looked into using Clouds to solve the HPC model challenges. Initially, CDS&E applications were run on commodity Clouds, but this was found to be appropriate only for certain classes of applications. Other approaches explored complementing HPC resources with Clouds but failed to address all challenges in the HPC environment. Cloud providers also tried to provide HPC as a Cloud using small HPC clusters connected to form a larger Cloud but were hindered by small scale and limited performance. These approaches fall short of providing the high performance necessary for CDS&E applications. In this document, we propose a new approach to achieve the notion of HPC as a Service. This approach targets existing high-end HPC resources and investigates how a Cloud abstraction can be applied to provide a simple interface and support real-world applications. In particular, the application of Clouds to supercomputers is discussed, tested, and validated on an IBM Blue Gene/P supercomputer. The proposed framework transforms Blue Gene/P into an elastic cloud by bridging multiple systems to create HPC federated Clouds, supporting dynamic provisioning and efficient utilization, and maximizing ease-of-use through an as a Service abstraction. In order to effectively illustrate the benefits of such a concept, the proposed framework is demonstrated using a real-world ensemble application.
Subject (authority = RUETD)
Topic
Electrical and Computer Engineering
RelatedItem (type = host)
TitleInfo
Title
Rutgers University Electronic Theses and Dissertations
Rutgers University. Graduate School - New Brunswick
AssociatedObject
Type
License
Name
Author Agreement License
Detail
I hereby grant to the Rutgers University Libraries and to my school the non-exclusive right to archive, reproduce and distribute my thesis or dissertation, in whole or in part, and/or my abstract, in whole or in part, in and from an electronic format, subject to the release date subsequently stipulated in this submittal form and approved by my school. I represent and stipulate that the thesis or dissertation and its abstract are my original work, that they do not infringe or violate any rights of others, and that I make these grants as the sole owner of the rights to my thesis or dissertation and its abstract. I represent that I have obtained written permissions, when necessary, from the owner(s) of each third party copyrighted matter to be included in my thesis or dissertation and will supply copies of such upon request by my school. I acknowledge that RU ETD and my school will not distribute my thesis or dissertation or its abstract if, in their reasonable judgment, they believe all such rights have not been secured. I acknowledge that I retain ownership rights to the copyright of my work. I also retain the right to use all or part of this thesis or dissertation in future works, such as articles or books.