Subgroup-Aware Evaluation and Interpretability in Myopia Prediction using Deep ROC and XAI

Machine Learning (ML) models are familiar for their great success in classification challenges within different domains. For medical diagnostics, those models have numerous applications as well. Basically, these ML models generate good predictions. However, sometimes, for clinicians, they are unable to depend on and use those models in a timely manner while making clinical decisions. Moreover, this factor arises because of the "black box" nature of these models. In medicine, it is a matter of concern. Because it is crucial to understand the base for predictions of patients’ safety and treatment assurance. Besides, recording key performance differences between risk groups is significant for informed clinical decision-making. In that case, traditional measures like the Receiver Operating Characteristic (ROC) curve and the Area Under the Curve (AUC) show some limitations. Hence, to defeat such matters, this study proposes a unified framework for experimenting with Myopia prediction (known as nearsightedness) and evaluating the results using some advanced techniques. This approach integrates Deep ROC analysis with Explainable AI (XAI) methods, where for XAI, Local Interpretable Model-agnostic Explanations (LIME) and Shapley Additive Explanations (SHAP) are applied. Here, the Deep ROC analysis is more refined and has clinical relevance, as the model performance is generated within subgroups defined by False Positive Rate (FPR). On the other hand, it allows for an in-depth understanding of model behavior in high-risk, medium-risk, and low-risk patient populations. Meanwhile, LIME and SHAP offer XAI value by explaining how individual attributes contribute to model output. While doing myopia prediction, this proposed evaluation structure is applied to the machine learning models, named Support Vector Machine (SVM), Random Forest (RF), Multi-Layer Perceptron (MLP), and Decision Tree (DT). Firstly, these models are trained and then evaluated to ensure robustness and reduce bias. After the evaluation, Random Forest (RF) outperforms the other three models across all subgroups, with an AUC of 98.92% in high-risk data 99.49% in low-risk data. The Brier score further confirmed the reliability of the model and improved its predictive power. The union of Deep ROC and XAI techniques significantly enhances the explainability and clinical utility of RF, with proof of how it forecasts for different risk levels. The approach is a leap forward towards the production of explainable, accurate, and clinically valuable AI models in medicine.