Development of an artificial intelligence (AI) governance and ethics policy: first application for the introduction of AI systems at a German University Hospital

With the rising number of innovative artificial intelligence (AI) developments claiming to be "ready for clinical use", we see an urgent need for an AI governance and ethics policy that supports the introduction of AI systems within university hospitals. To gain a comprehensive overview, we interviewed professionals from various disciplines within the hospital environment who may be involved in AI applications, including physicians, data protection officers, legal department personnel, and IT specialists. Based on this, we developed a governance and ethics policy for the introduction of AI systems in university hospitals. We conducted six group discussions (focus groups) at four German university hospitals. The results were analyzed and subsequently converted into an AI governance and ethics policy. A preliminary version of a generic AI governance and ethics policy for the introduction of AI systems within university hospitals was developed. Based on this, an AI introduction guideline and checklist were established at the University Hospital Erlangen (UKER) as a first use case. The developed AI governance and ethics policy was actively piloted at the UKER aiming to carry out the introduction of AI systems in German university hospitals in a structured manner and to establish a systematic process to mitigate potential risks. The policy is an initial step towards the development of a holistic AI governance and ethics framework and enables hospital administrators to systematically evaluate AI systems that are requested to be implemented by using standardized criteria aligning with the key requirements of the European Union (EU) AI Act. So far, 44 AI-system inquiries have been processed and evaluated based on the developed AI governance and ethics policy. The roll-out of the policy across additional German university hospitals is ongoing as part of the multicenter project Open Medical Inference (OMI) within the Medical Informatic Initiative Germany (MII).