The Engineering of AI Evaluation and Scoring: Overview and Insights

The rapid evolution of Artificial Intelligence (AI) requires a rethinking of practices and better anticipation of impacts, as highlighted by several new regulations. Different communities, including engineers, auditors, business leaders, standardization experts, and ethicists, are developing AI evaluation frameworks with scoring systems to progressively operationalize good practices for AI. The development of AI evaluation frameworks should be closely tied to disciplines such as systems and software engineering, requirements engineering, quality management, risk analysis, verification and validation, and decision support. This paper formalizes the development of AI evaluation frameworks as six non-linear activities to better leverage these disciplines. It is intended to be of particular interest to communities engaged in AI evaluation but not familiar with systems engineering. It draws on the authors' experience in defining end-to-end methodologies for AI trustworthiness and designing evaluation frameworks. The activities presented are: (1) Managing the evaluation framework as a product in its own right; (2) Managing the evaluation framework as a generic specification for different objects; (3) Formalizing progressive good practices, possibly in the form of requirement levels; (4) Clustering these requirements into understandable dimensions; (5) Defining a global scoring scheme; and (6) Tailoring the framework to account for industry-specific considerations and risks. This formalization aims to enhance the effectiveness of AI evaluation frameworks.