Multi-Modal Data Fusion for Robust CNN-Based Diabetic Foot Ulcer Classification

The (DFU) Diabetic Foot Ulcers are considered one of the severe complications for diabetic patients, which require very precise diagnosis and treatment to prevent more critical medical complications. This study presents a novel multi-modal deep learning approach that integrates image data along with non-image clinical data to automatically classify diabetic foot ulcers. The system uses a Convolutional Neural Network (CNN) to segment ulcer images into three tissue types: granulation, necrotic, and slough. In addition to this, an MLP was used to process structured non-image data like patient demographics, lab results, and sensor readings. The model combines features from both modalities at an intermediate stage, thereby being able to utilise insights from both modalities for robust classification. Outputs of our method include segmented foot ulcer images with colour-coded tissue classification and analysis of the ulcer from all sides. This leads to a better overall performance of the model, found to be a resourceful tool in the study of healthcare professions for the diagnosis, management of diabetic foot ulcers and may lead to improved patient outcomes through more informed clinical decisions. With experimental results showing the success of the proposed method in attaining high classification accuracy.