Journey of provisioning HPC with Infrastructure-as-Code approach

NeSI is currently in the process of provisioning its new HPC infrastructure. This marks the third major infrastructure provisioning process NeSI is embarking on. This presentation will focus on two of the key approaches taken and share the experience and learnings from the process. The first approach is around provisioning of the HPC and its core services using the Infrastructure-as-Code (IaC) approach. NeSI has an ongoing collaboration with AgResearch to build and operate an eResearch infrastructure. This was done on top of NeSI s Flexible HPC, a platform designed to support flexibility and scale. Many of the learnings from this collaboration was applied to NeSI s own infrastructure provisioning process, and much of the IaC based approach helped NeSI in taking the processes from the code base and modifying them to fit the needs. The second approach is around NeSI s data migration into the new infrastructure. NeSI has adopted new technologies, WEKA and Versity, to address the needs around ever growing data. NeSI has applied thinking of data in temperature to allow smoother migration as well as ongoing HPC operations. This presentation will address many of the challenges around data migration and the solutions applied.