Application of multimodal deep learning in the auxiliary diagnosis and treatment of dermatological diseases

Skin diseases are important factors affecting health and quality of life, especially in grass-roots areas where medical resources are limited. Early and accurate diagnosis can reduce unnecessary health and economic losses. However, traditional visual diagnosis poses high demand on both doctors' experience and the examination equipment, and there is a risk of missed diagnosis and misdiagnosis. In recent years, with the development of artificial intelligence (AI) technology, especially deep learning, unimodal computer-aided diagnosis and treatment technologies based on skin images have been increasingly employed in dermatology. However, due to the small amount of information contained in unimodality, this technology cannot fully demonstrate the advantages of multimodal data in the real medical environment. Multimodal data fusion can fully integrate various types of data to help doctors make more accurate clinical decisions. This review aimed to provide a comprehensive overview of multimodal data and deep-learning methods that can help dermatologists diagnose and treat skin diseases.