Risk inventory and mitigation actions for AI in medical imaging—a qualitative study of implementing standalone AI for screening mammography

Recent prospective studies have shown that AI may be integrated in double-reader settings to increase cancer detection. The ScreenTrustCAD study was conducted at the breast radiology department at the Capio S:t Göran Hospital where AI is now implemented in clinical practice. This study reports on how the hospital prepared by exploring risks from an enterprise risk management perspective, i.e., applying a holistic and proactive perspective, and developed risk mitigation actions. The study was conducted as an integral part of the preparations before implementing AI in a breast imaging department. Collaborative ideation sessions were conducted with personnel at the hospital, either directly or indirectly involved with AI, to identify risks. Two external experts with competencies in cybersecurity, machine learning, and the ethical aspects of AI, were interviewed as a complement. The risks identified were analyzed according to an Enterprise Risk Management framework, adopted for healthcare, that assumes risks to be emerging from eight different domains. Finally, appropriate risk mitigation actions were identified and discussed. Twenty-three risks were identified covering seven of eight risk domains, in turn generating 51 suggested risk mitigation actions. Not only does the study indicate the emergence of patient safety risks, but it also shows that there are operational, strategic, financial, human capital, legal, and technological risks. The risks with most suggested mitigation actions were ‘Radiographers unable to answer difficult questions from patients’, ‘Increased risk that patient-reported symptoms are missed by the single radiologist’, ‘Increased pressure on the single reader knowing they are the only radiologist to catch a mistake by AI’, and ‘The performance of the AI algorithm might deteriorate’. Before a clinical integration of AI, hospitals should expand, identify, and address risks beyond immediate patient safety by applying comprehensive and proactive risk management.