AMLOP: A Framework for the Operational Safety Assurance of Machine Learning Leveraging Model Monitoring

In recent years, machine learning (ML), including deep learning, has been adopted in various safety-critical domains, such as healthcare, manufacturing, communications and transportation. Despite safety practices implemented during model design and development, unique ML challenges, such as data corruption, data drift, and data anomalies or novelties, can emerge after model deployment. These challenges underscore a pivotal concern: ensuring the operational safety of ML models. Currently standards such as ISO/IEC TR 5469 and IEC 61508 lack comprehensive guidelines for managing these post-deployment challenges across the full ML lifecycle. Moreover, the inherent complexity and lack of transparency in various ML models further contribute to this issue. To bridge this gap, we propose AMLOP, a comprehensive, sevenstage framework specifically designed to assure ML operational safety through systematic model monitoring. This framework is based on four ML monitoring architectures, offering systematic guidance for implementing ML runtime monitoring effectively. The framework's efficacy is demonstrated through a case study on 5G network slicing, presenting its potential to enhance ML operational safety. This framework aligns with global standardization efforts, providing clear guidance toward achieving safe, trustworthy, and robust ML deployments.