D4.5 HPC API and documentation

This document describes HPC-API development for LUMI AI Factory. An HPC-API (High Performance Computing Application Programming Interface) is used to programmatically manage and interact with HPC environments. It functions for job management, like submitting and cancelling jobs on a supercomputer and querying their status, as well as handling data on the system. The HPC-APIs deployed to LUMI are used by access various software components that are being developed for the LUMI AI Factory. Modern AI (Artificial Intelligence) development often relies on an extensive underlying software environment to support the machine learning workloads running in the system. This document describes the deployment of a new HPC-API for LUMI. The aim is to set up an HPC-API that can be used with MLOps tools and containerized workflows to serve users of LUMI AI Factory in AI development tasks. The technology selected for this HPC-API is FirecREST developed by Swiss National Supercomputing Centre (CSCS). This document also describes HEAppE as an alternative approach. HEAppE, developed by IT4Innovations National Supercomputing Center (IT4I), is an open-source software framework that provides a secure and user-friendly interface to HPC resources. HEAppE is already used as part of some CSC services as well as isolated, dedicated HPC-API for some scientific projects.