Explainable GI lesion classification via class prototype guided hybrid feature fusion on Kvasir datasets

Accurate and automated classification of gastrointestinal (GI) diseases from endoscopic images is crucial for supporting clinical diagnosis and reducing inter-observer variability. This study presents ClinFusion-Net, a hybrid deep learning framework that integrates convolutional neural networks and vision transformers for multi-class GI disease classification. ConvNeXt-XL extracts fine-grained local features, whereas Swin-Large captures broader contextual information. Their representations are fused through a class-prototype-guided MedicalAttention mechanism that assigns greater weight to diagnostically relevant features. Training employs a composite MedicalLoss objective combining weighted cross-entropy, focal loss, and attention-entropy regularization to improve class sensitivity and predictive calibration. ClinFusion-Net was trained and evaluated separately on Kvasir v1 and its expanded release, Kvasir v2, using release-specific data partitions, achieving an accuracy of 95.1% and a macro-AUC of 0.998 on Kvasir v2, and an accuracy of 96.3% and macro-AUC of 0.998 on Kvasir v1. On Kvasir v2, its accuracy was comparable to that of the strongest single-backbone baseline (95.1% versus 95.0%), while maintaining strong class-balanced discrimination through the integrated local–global representation. Because Kvasir v1 and Kvasir v2 are related releases from the same dataset family, these results are presented as separate within-release evaluations rather than as evidence of cross-dataset generalization. Controlled ablations identify feature fusion as the principal contributor to performance, with class-prototype attention and the composite objective providing complementary gains in calibration and difficult-class sensitivity. Furthermore, model interpretability is enhanced through Grad-CAM, LIME, and t-SNE embedding visualizations, which collectively provide spatial, perturbation-based, and feature-level evidence that predictions are associated with diagnostically relevant mucosal regions. Overall, the proposed framework demonstrates a high-performing, and interpretable solution for automated GI disease classification, with strong potential to support clinical decision-making and facilitate earlier diagnosis in real-world settings.