P007 Comparative real-world performance of an artificial intelligence as a medical device and consultant teledermatologists in diagnosing benign skin lesion subtypes

Abstract Providing a consistent diagnosis is helpful when reassuring and discharging patients with benign skin lesions. A UK Conformity Assessed (UKCA) class IIa artificial intelligence as a medical device (AIaMD) classifies lesions with specific benign labels, in addition to premalignant and malignant labels. This study investigated the agreement between the benign diagnostic labels provided by the AIaMD and consultant teledermatologists for skin lesions prospectively assessed in the real world. Skin lesions were included that were assessed as part of standard care across 13 National Health Service sites in the UK (from April 2022 to April 2024). The lesions were assessed as benign by the AIaMD, routed for further review by a second read teledermatologist (SRTd) and discharged by a hospital teledermatologist. Lesions labelled benign were further classified into one of five specific subtypes – benign melanocytic naevus (BMN), dermatofibroma, seborrhoeic keratosis, solar lentigo (lentigo) or vascular lesions – or given an overall benign categorization. Positive predictive values (PPV) for the AIaMD and SRTd were calculated based on a ground truth diagnosis provided by the hospital teledermatologist. In total, 2133 lesions were included. The patients ages ranged from 18 to 97 years, and all Fitzpatrick skin types were represented. The PPVs (with 95% confidence intervals) of the AIaMD for BMN, dermatofibroma, seborrhoeic keratosis, solar lentigo and vascular lesions were respectively 92% (84.3–96.0), 93.3% (70.2–98.8), 94.6% (91.3–96.7), 60% (31.3–83.2) and 96.2% (89.4–98.7). The PPVs of the SRTd assessments for BMN, dermatofibroma, seborrhoeic keratosis, solar lentigo and vascular lesions were 87.9% (83.7–91.1), 88.5% (78.2–94.3), 97.1% (95.1–98.3), 59.6% (26.5–71.8) and 95.4% (88.8–98.2). There were no significant differences (P > 0.05) between lesion-specific PPVs provided by the AIaMD or SRTd. In conclusion, when the AIaMD outputs a specific benign diagnostic label, its performance is comparable with that of a consultant teledermatologist. The AIaMD referenced is marked UKCA Class IIa and CE Class III.