Multimodal Learning for Enhanced Detection in Oral Cancer Screening

Multimodal deep learning integrates diverse data types to enhance model performance by capturing complex interactions between different modalities. Traditional models Fare limited to single data types, whereas multimodal approaches use fusion strategies-early, late, and hybrid-to combine information from different sources effectively. In this study, we introduce a multimodal learning framework for oral cancer detection. This framework integrates images from various imaging systems, specifically white light, and fluoroscopy, along with a tabular dataset containing patient information. Leveraging the CLIP model and deep neural network classification models, such as EfficientNet, DenseNet, and ResNet, we conducted a comprehensive analysis and comparison using various performance metrics. Our approach demonstrates the potential for improved diagnostic accuracy through enhanced feature extraction from multimodal sources.