AI metrics and policymaking: assumptions and challenges in the shaping of AI

Abstract This paper explores the interplay between AI metrics and policymaking by examining the conceptual and methodological frameworks of global AI metrics and their alignment with National Artificial Intelligence Strategies (NAIS). Through topic modeling and qualitative content analysis, key thematic areas in NAIS are identified. The findings suggest a misalignment between the technical and economic focus of global AI metrics and the broader societal and ethical priorities emphasized in NAIS. This highlights the need to recalibrate AI evaluation frameworks to include ethical and other social considerations, aligning AI advancements with the United Nations Sustainable Development Goals (SDGs) for an inclusive, ethical, and sustainable future.