Assessment of AI Influence on an IT Company Performance: Methodologies and Challenges

Artificial Intelligence (AI) has gained popularity as a transformative force for businesses in different domains. Despite its rapid adoption, particularly within IT companies, one still faces a challenge when it comes to measuring AI's influence on organizational performance. To a certain extent, AI affects both macro-level metrics (e.g., ROI or overall employee productivity), and specific operational Key Performance Indicators (KPIs), including employee satisfaction and turnover rates, customer satisfaction and net promotion scores, marketing effectiveness, and so on. Moreover, AI influence often alters over time since AI-based solutions often require an 'adaptation' period, which includes technical implementation, additional process changes, staff training, etc. This article explores potential ways to assess AI's impact on IT company performance, considering both the challenges of broad influence and time-dependent results. The study underscores the importance of integrating quantitative metrics with qualitative feedback mechanisms to capture a holistic view of AI's impact, especially in direct user interaction scenarios. Through the case study of a mid-sized IT (game development) company, the article demonstrates how AI-powered marketing asset creation improved campaign efficiency, increased testing capacity, and boosted financial performance over six months. This real-world application showcases an example of an incremental analysis to evaluate AI's contributions to both macro and micro-level metrics to understand if the AI application paid off. The findings emphasize that measuring AI's influence requires a complex approach, which would consider both the high-level organizational metrics, specific operational KPIs, and user insights. Future research should explore ways to isolate AI-related contributions in complex real-economy environments and monitor long-term AI performance trends.