Predicting the diabetic foot in the population of type 2 diabetes mellitus from tongue images and clinical information using multi-modal deep learning

We established a DF prediction model based on clinical features and objectified tongue color, which showed the unique advantages and important role of objectified tongue demonstration in the DF risk prediction, thus further proving the scientific nature of TCM tongue diagnosis. Based on the qualitative and quantitative fusion data, we combined tongue images with DF indicators to establish a multi-mode DF prediction model, in which tongue demonstration and objectified foot data can correct the subjectivity of prior knowledge. The successful establishment of the feature fusion diagnosis model can demonstrate the clinical practical value of objectified tongue demonstration. According to the results, the model had better performance to distinguish between T2DM and DF, and by comparing the performance of the model with and without tongue images, it was found that the model with tongue images performed better.