A Certification Scheme for Large Language Models-Based Applications

The advent of Large Language Models (LLMs) is revolutionizing the design and deployment of modern applications, enabling intelligent, adaptive, and context-aware services across a wide range of domains. AI-driven services now coexist with legacy systems, microservices, and nanoservices, forming complex and evolving environments. While LLMs offer unprecedented capabilities, they also introduce new risks related to security, privacy, and ethics, especially due to their probabilistic nature and reliance on vast, often opaque, training sets. This paradigm shift calls for robust mechanisms to assess the non-functional behavior of LLM-based applications. In the last decades, assurance has emerged as a key strategy to assess the non-functional properties of complex distributed systems and applications. However, traditional assurance methods prove inadequate to assess LLM-based applications, while existing assurance approaches tailored to LLMs are still in their infancy. In this paper, we propose a multi-dimensional certification scheme for LLM-based applications that initially captures the broader context and dynamic behavior introduced by LLMs. To this aim, after proposing a taxonomy of LLM-based applications and discussing the current gaps in LLM assessment, we develop a certification process that accounts for the peculiarities of the different types of applications in the taxonomy. The certification process builds on a hypergraph that represents a specific behavior supporting the property to be certified and guides the corresponding evidence collection process. We experimentally evaluate our scheme using an LLM-based application that implements an assessment process for cloud systems.