Multimodal deep learning approach for wound classification and severity estimation using depth analysis

Classification and evaluation of wounds pose significant challenges to the provision of health care particularly in rural areas where the rate of chronic wounds is greater compared to urban locations. Manual evaluation methods are subjective, lengthy and often provide different interpretations across clinicians. Modern studies are actively using deep learning models to automate wound evaluation, experimenting with various convolutional neural network (CNN) designs, hybrids and attention methods to improve analytical accuracy and precision. The study presents a multi-modal deep learning approach that combines U-Net for segmentation, EfficientNet B5 for detection, and ResNet for classification, using both RGB and depth data. The dataset is a combination of wound images of hospital patients and established public repositories, such as the Medetec Wound Dataset, AZH Dataset and DFUC2021 Challenge Dataset, to train model on various range of wound typologies and patient demographics. A user interface allows healthcare experts to obtain RGB and depth images in real time, providing instant wound classification and analysis outcomes in the point of care. The average accuracy of the system is 96%. Dice coefficients and intersection-over-union (IoU) over 0.80 indicates high concordance between ground-truth annotations and predicted segmentations. Severity assessment using the depth mapping and the automated severity assessment enhances the accuracy of wound measurement and classification in comparison with the traditional single source methodologies and thus provides a feasible solution in both clinical and remote healthcare settings.