External Validation of AI Models in Swedish Healthcare : Challenges and Opportunities for Safe and Effective Adoption

Using AI systems in healthcare has shown advantages in detecting diseases, this indicates that adopting AI systems into clinical practice can be beneficial and promote public health. Although there are challenges regarding how to evaluate and quality assure AI systems to ensure they are robust, reliable and patient safety. AI development is at the forefront within breast radiology in Sweden and therefore this study was conducted to examine the evaluation methods within that field. This study investigates the processes and challenges associated with external validation of AI models in Swedish healthcare during adoption to ensure patient safety. A specific focus is on the validation factors: robustness, reliability and bias, as well as the impact of regulatory frameworks on the evaluation process. The study was carried out using a qualitative approach where three semi-structured interviews were conducted. All interviewees are engaged in the field of AI in healthcare. A thematic analysis was used to analyze the data and identify common themes. The result indicates that it is a need to establish standardized evaluation methods for external validation to guarantee AI´s performance and ensure patient safety. The lack of established methods results in significant variations in how the evaluation process is carried out. The result further indicates that innovative projects face difficulties in accessing patient data. This is because the purpose of innovation is viewed as profit marking and falls outside the scope of health and medical care, which in turn hinders AI initiatives to be developed. The regulations have a strong foundation in protecting privacy which limits the access to patient data and hinders AI models from being trained and validated on comprehensive amounts of patient data. Furthermore, the study shows the need for standardized frameworks for external validation of AI systems in healthcare. To facilitate AI development within innovation projects the purpose of innovation needs to be defined within healthcare. The regulatory frameworks need to be adapted to promote AI development and the potential it entails for the patients.