A dual-stream deep learning framework for skin cancer classification using histopathological-inherited and vision-based feature extraction

Skin cancer, particularly melanoma, remains one of the most life-threatening forms of cancer worldwide, with early detection being critical for improving patient outcomes. Traditional diagnostic methods, such as dermoscopy and histopathology, are often limited by subjectivity, interobserver variability, and resource constraints. To address these challenges, this study proposes a dual-stream deep learning framework that combines histopathological-inherited and vision-based feature extraction for accurate and efficient skin lesion diagnosis. The framework uses the U-Net architecture for precise lesion segmentation, followed by a dual-stream approach: the first stream employs Virchow2, a pretrained model, to extract high-level histopathological embeddings, whereas the second stream uses Nomic, a vision-based model, to capture spatial and contextual information. The extracted features are fused and integrated to create a comprehensive representation of the lesion, which is then classified via a multilayer perceptron (MLP). The proposed approach is evaluated on the HAM10000 dataset, achieving a mean accuracy of 96.25% and a mean F1 score of 93.79% across 10 trials. Ablation studies demonstrate the importance of both feature streams, with the removal of either stream resulting in significant performance degradation. Comparative analysis with existing studies highlights the superiority of the proposed framework, which outperforms traditional single-modality approaches. The results underscore the potential of the dual-stream framework to enhance skin cancer diagnosis, offering a robust, interpretable, and scalable solution for clinical applications.