YOLO11-guided swin transformer for molar occlusion classification

Molar occlusion identification is a fundamental component of orthodontic diagnosis and treatment planning. Conventional assessment using intraoral photographs relies on manual visual inspection, making it time-consuming and susceptible to inter-observer variability. Moreover, existing deep learning (DL)-based approaches either analyze the entire image without explicit anatomical localization or depend on manually defined regions of interest, thereby limiting clinical applicability. Accordingly, this study proposes a DL framework for molar occlusion classification. The proposed framework blends YOLO11-based anatomical localization with a Swin Transformer (Swin-T) for occlusion recognition. YOLO11 performs automatic instance segmentation to identify the molar region and eliminate non-diagnostic background content. The resulting prediction-guided molar crops are then processed by Swin-T, which learns discriminative representations of posterior occlusal relationships through hierarchical shifted-window attention and multiscale feature modeling. A dataset of 1101 intraoral images spanning five occlusion classes was used. The localization stage achieved a Dice coefficient of 96.82%. The developed framework attained classification accuracy of 98.17%, outperforming expert orthodontist assessment. Furthermore, the proposed framework outperformed well-established models, including ResNet50V2, DenseNet201, MobileNetV3-Large, EfficientNetV2-S, and vision transformer (ViT-B/16). Experimental results demonstrated perfect recall for Half Class II, Class III, and Half Class III, while maintaining strong predictive performance for Class I and Class II. Localization and explainability analyses confirmed that the framework consistently attended to clinically relevant posterior occlusal structures, affirming the anatomical plausibility of its predictions. These findings provide a promising foundation for automated molar occlusion assessment and suggest potential for future integration of DL-driven decision support into digital orthodontic workflows.