Dynamic domain analysis for predicting concept drift in engineering AI-enabled software

ABSTRACT Purpose This research addresses the challenge of concept drift in AI-enabled software, particularly within autonomous vehicle systems where concept drift in object recognition (like pedestrian detection) can lead to misclassifications and safety risks. This study introduces a proactive framework to detect early signs of domain-specific concept drift by leveraging domain analysis and natural language processing techniques. This method is designed to help maintain the relevance of domain knowledge and prevent potential failures in AI systems due to evolving concept definitions. Design/methodology/approach The proposed framework integrates natural language processing and image analysis to continuously update and monitor key domain concepts against evolving external data sources, such as social media and news. By identifying terms and features closely associated with core concepts, the system anticipates and flags significant changes. This was tested in the automotive domain on the pedestrian concept, where the framework was evaluated for its capacity to detect shifts in the recognition of pedestrians, particularly during events like Halloween and specific car accidents. Findings The framework demonstrated an ability to detect shifts in the domain concept of pedestrians, as evidenced by contextual changes around major events. While it successfully identified pedestrian-related drift, the system’s accuracy varied when overlapping with larger social events. The results indicate the model’s potential to foresee relevant shifts before they impact autonomous systems, although further refinement is needed to handle high-impact concurrent events. Research limitations This study focused on detecting concept drift in the pedestrian domain within autonomous vehicles, with results varying across domains. To assess generalizability, we tested the framework for airplane-related incidents and demonstrated adaptability. However, unpredictable events and data biases from social media and news may obscure domain-specific drifts. Further evaluation across diverse applications is needed to enhance robustness in evolving AI environments. Practical implications The proactive detection of concept drift has significant implications for AI-driven domains, especially in safety-critical applications like autonomous driving. By identifying early signs of drift, this framework provides actionable insights for AI system updates, potentially reducing misclassification risks and enhancing public safety. Moreover, it enables timely interventions, reducing costly and laborintensive retraining requirements by focusing only on the relevant aspects of evolving concepts. This method offers a streamlined approach for maintaining AI system performance in environments where domain knowledge rapidly changes. Originality/value This study contributes a novel domain-agnostic framework that combines natural language processing with image analysis to predict concept drift early. This unique approach, which is focused on real-time data sources, offers an effective and scalable solution for addressing the evolving nature of domain-specific concepts in AI applications.