D4.3 Inference Service documentation

LUMI AI Factory's objectives are advancing the development of AI solutions, including offerings like inference service. We expect to see fast-paced growth in computational needs in this area. This document describes two inference platforms: Aitta and EXA4MIND. Aitta inference platform is developed by CSC, and EXA4MIND developed by the EXA4MIND project consortium (IT4I and METU). The inference services will be provided by LUMI AI Factory. Inference refers to the process of using an AI model to generate predictions, produce outputs or perform specific tasks based on new input data. Inference enables real-world applications of Large Language Models (LLMs) and other AI models, allowing users to interact with the model by providing prompts — text inputs that guide the model’s responses. Aitta inference platform is a service to host AI models on supercomputers. With Aitta, you can interact with LLMs efficiently using simple API (Application Programming Interface) calls or simple Web UI (Web browser User Interface). Users will be able to run their self-trained models, use curated portfolio of models. Easy integration to existing model repositories will be provided. EXA4MIND inference service is developed on efficient leveraging of HPC resources in a project initiative that is building a platform for extreme data (i.e. data that is at the upper or lower limits of expectations that should be accepted by the system), enabling advanced analytics on supercomputers. EXA4MIND inference service solution complements the Aitta inference service with the main difference in the task queue implementation. This opens the possibility for further Aitta extensions and optimization based on the incoming requirements.