Empirical Validation of a Human‐AI Collaboration Assessment Framework

ABSTRACT Human‐AI collaboration is a defining characteristic of AI‐augmented systems and increasingly critical for organizational performance. However, small and medium‐sized enterprises (SMEs) lack practical, validated methods to assess how effectively humans and artificial intelligence (AI) function together in their organization and to guide improvements in collaborative maturity. This paper extends and empirically validates the Human‐AI Collaboration Maturity Model (HAIC‐MM) as a systems engineering assessment framework designed to reflect the operational realities and resource constraints of SMEs. The study introduces a quantitative scoring methodology that incorporates novel balance‐based scoring and dependency‐aware capability progression to capture interdependencies between human and AI subsystems and their influence on system‐level performance and behavior. The framework is operationalized through a web‐based platform and evaluated through deployment with ten SME practitioners across diverse industries. Results indicate high practitioner satisfaction with assessment usability, strong alignment between outputs and organizational realities, and clear practical value of the generated reports. The novel balance‐based scoring approach effectively revealed misalignments between human and AI capabilities that conventional averaging methods obscure, while dependency logic prevented artificially inflated maturity classifications. Overall, HAIC‐MM is shown to be a usable, interpretable, and practically valuable tool for assessing and strengthening human‐AI collaboration in SMEs. By empirically validating the model and introducing a novel structured scoring approach, this work advances systems engineering methods for evaluating and managing human‐AI collaboration, supporting system integration and lifecycle monitoring of AI‐augmented systems.