Quantitative Assessment of Segmented Masks: A Deep Learning Regression and Classification Study

Motivation: Ability to track real-world performance of AI based spine segmentation models without access to ground-truth data. Goal(s): Develop AI models which allow prediction of spine vertebrae segmentation model performance in real time. Approach: Developed a regression and classification deep learning (DL) models that determines quality of segmentation results in terms of Dice overlap metric from a parent segmentation DL model. Results: For regression model, dice prediction error of 4.3% was obtained, while for categorical classification model, sensitivity between 63-87% observed across evaluation categories. Combination of regression and classification models improves model performance evaluation with sensitivity between 71 to 91%. Impact: : We developed DL models to automatically evaluate accuracy of spine-vertebrae segmentation models during their deployment in clinical practice without access to ground-truth in both quantitatively (Dice) and qualitatively (Perfect, good, medium, poor). This ensures automatic-logging model effectiveness in real-world data.