Bridging global semantic representation and inference efficiency via hybrid knowledge distillation for clinically deployable cardiac MRI classification

In cardiac magnetic resonance imaging (MRI), achieving a balance between high-level semantic representation and real-time inference remains a fundamental challenge. While Vision Transformers (ViTs) effectively capture global dependencies, their computational overhead limits clinical deployment. Conversely, Convolutional Neural Networks (CNNs) offer efficiency but often fail to model global cardiac structures. This study proposes an optimized hybrid knowledge distillation framework that transfers rich semantic representations from a computationally intensive teacher model to a lightweight student network, enabling accurate and efficient cardiac MRI classification suitable for clinical environments. A dual-branch teacher architecture was designed by integrating an EfficientNetB0 backbone with a customized Multi-Head Self-Attention module to jointly capture local tissue characteristics and global structural context. Knowledge distillation was performed using a hybrid loss function enhanced with label smoothing to mitigate overconfidence and cosine decay restarts to improve optimization dynamics in the student model. Using 5-fold cross-validation, the distilled student model achieved a mean classification accuracy of 86.84% ± 1.12% and a sensitivity of 86.46% ± 0.95%. The low variance across folds indicates robustness to data heterogeneity. Compared with a baseline ViT model, the proposed framework delivered comparable diagnostic performance while achieving a 3.01-fold improvement in computational efficiency, enabling real-time inference on standard medical hardware. The proposed framework effectively bridges the gap between semantic richness and inference efficiency in cardiac MRI analysis, offering a practical and deployable solution for intelligence-based clinical decision support systems. Optimized Hybrid Distillation Framework for Cardiac MRI • Addresses the trade-off between semantic representation and inference efficiency in cardiac MRI • Introduces a hybrid CNN–attention teacher for effective knowledge distillation • Enhances student model generalization using label smoothing and cosine decay restarts • Achieves stable performance with low variance across cross-validation folds • Enables real-time cardiac MRI classification on standard clinical hardware