Evaluating the Effectiveness of AI-Supported Digital Training: Implications for Organizational Learning and Decision-Making

In contemporary organizations, digital learning environments and AI-supported instructional modalities play an increasingly important role in workforce upskilling and operational efficiency. Despite growing investments in video-based learning and AI-generated instructional agents, empirical evidence on their effectiveness remains inconclusive. This study examines whether different digital learning modalities influence skill acquisition, task performance, retention, and user perceptions in a simulated work-related context. An experimental study was conducted with 65 participants assigned to one of three learning conditions: static instructional material, video-based instruction with human narration, and video-based instruction with an AI-generated avatar. Performance was assessed through a pretest–posttest design, a practical task simulating a typical data-processing activity, and a delayed retention test after seven days. Participants also evaluated the learning experience in terms of clarity, engagement, and overall effectiveness. The results revealed no statistically significant differences between instructional modalities in knowledge acquisition, task performance, or retention. Similarly, no statistically significant differences were observed in participants’ self-reported ratings. However, qualitative findings suggested that some participants perceived the AI-generated avatar as somewhat distracting, despite generally positive evaluations of the video-based formats. These findings did not provide evidence that more technologically advanced and resource-intensive learning formats led to superior performance outcomes in the present sample. The findings highlight the importance of instructional design quality over technological complexity and point to a potential mismatch between user preferences and actual performance. From a management perspective, the results raise relevant questions regarding the cost-effectiveness of AI-supported learning solutions and provide evidence-based insights for decision-making in organizational learning and digital transformation strategies.