Bringing Machine Learning Models Beyond the Experimental Stage with Explainable AI

Machine learning (ML) is about to revolutionize information systems, with AI -enabled technologies emerging across different sectors. Yet, companies often struggle to bring ML models beyond the experimental stage. Many challenges remain in developing ML components for large-scale IT-systems and integrating them into production systems. This article presents a case study and its findings at a large Danish bank, where a previously unused ML model was successfully integrated into a mission critical IT system. A key factor to this success was the use of explainable AI, which increased the confidence of human experts in ML models and enabled them to validate and retrain ML models in production, even though they were not software developers. Our findings are backed by a structured evaluation and interviews with expert users.