BT04 A dermatology image quality assessment tool for skin cancer teledermatology

Abstract In November 2024, the British Association of Dermatologists released a new Service Guidance and Standards for Teledermatology to provide quality assurance for services nationally. One standard is to ensure personnel are appropriately trained to capture high-quality images. Thus, it is important to have a reliable and measurable way of assessing image quality and providing explanations to guide improvement. This is due to the expanding workforce undertaking skin lesion photography with variable levels of training, dermatology experience and devices. At present, there are few publicly available validated tools for skin lesion image quality assessment. None have been tested and validated on nondermatologists, who make up most of the expanding workforce capturing those images. Through literature review, consultations and focus group testing we developed a scale to assess image quality based on four elements: ‘lighting’, ‘angle’, ‘magnification’ and ‘focus’. Images were visually assessed, scored 0–5, and grouped into three quality categories: low (0–2), medium (3–4) and high (5). Following a preliminary expert-led group survey, a wider validation survey of experts and nonexperts was conducted, including representation from primary and secondary care. The survey consisted of 30 anonymized community-acquired and medical photography images. To reduce user variability, participants received 10 minutes of presurvey training. Inter-rater reliability (IRR) or user concordance was calculated using Fleiss’ kappa calculations. Fifty-one experts and nonexperts were recruited, including consultants, specialty trainees, nurses, healthcare assistants, technicians and medical students. Participants were able to differentiate images of low vs. medium vs. high quality with moderate concordance (IRR 0.45) and demonstrated good concordance (IRR 0.65) differentiating low vs. medium + high. This suggested users with variable clinical experience used this scale reliably to isolate poor-quality images. On in-group analysis, the expert group (consultants and general practitioners) had a high concordance (IRR 0.88) when differentiating low vs. medium + high. The nonexpert groups had moderate concordances (IRR 0.57–0.71). Analysis of individual elements showed fair-to-moderate performances (IRR 0.33–0.54). On average, participants rated the scale easy to use and averaged 35 s per image for assessment. Participants ranked ‘focus’ as the most important element (IRR 0.47–0.67), followed by ‘lighting’ (IRR 0.27–0.42), ‘magnification’ (IRR 0.42–0.71) and ‘angle’ (IRR 0.38–0.50). In conclusion, we have developed and validated an image quality scale that has guided explanations to quantify image quality. The scale has application in training and teledermatology quality assurance. Moreover, the scale’s high concordance when used among experts implies future application in training artificial intelligence models on expert annotations as groundwork for future governance and quality improvement processes.