Multimodal deep learning for automated assessment of periodontal bone resorption severity using panoramic radiographs and structured metadata

Abstract Purpose Assessment of periodontal bone resorption severity from panoramic radiographs can be time-consuming and subject to inter-observer variability. This study evaluates a multimodal deep learning framework for automated Bone Resorption Age Ratio (BRAR)-based severity classification by integrating panoramic radiographic features with structured metadata. Methods Using 988 annotated panoramic radiographs from a publicly available dataset, image-only, metadata-only, and multimodal models were evaluated across 20 repeated stratified random splits. The multimodal model integrates DenseNet-121 radiographic features with a multilayer perceptron encoding structured metadata through feature-level concatenation and progressive fine-tuning. Performance was assessed using accuracy, macro F1-score, macro area under the receiver operating characteristic curve (macro AUC-ROC), average precision, and class-specific metrics. Paired comparisons of three primary outcomes (macro F1, macro AUC-ROC, and Level 3 recall) used the Wilcoxon signed-rank test with Holm–Bonferroni correction. A four-way metadata ablation assessing the contributions of age and the remaining structured variables, an exploratory probability-level ensemble, qualitative Gradient-weighted Class Activation Mapping (GradCAM), and a targeted expert error analysis on 88 purposively selected cases were also performed. Results The multimodal model achieved an accuracy of $$0.67 \pm 0.03$$ , macro F1-score of $$0.65 \pm 0.03$$ , macro AUC-ROC of $$0.81 \pm 0.03$$ , and Level 3 recall of $$0.60 \pm 0.07$$ . It outperformed the image-only model in macro F1 and macro AUC-ROC (both Holm-adjusted $$p < 0.001$$ ) and Level 3 recall ( $$p=0.036$$ ), with all three primary outcomes also significantly higher than the metadata-only model ( $$p < 0.001$$ ), which identified only $$0.14 \pm 0.07$$ of severe cases. In an ablation of the metadata inputs, neither age alone nor the five non-age variables alone significantly improved macro F1 or Level 3 recall over the image-only model, whereas the full model significantly outperformed both. In the targeted expert subset, dentist–original agreement was higher among model-correct cases ( $$\kappa=0.585$$ ; weighted $$\kappa=0.650$$ ) than among model-misclassified cases ( $$\kappa=0.208$$ ; weighted $$\kappa=0.248$$ ). Conclusions Integrating panoramic radiographs with structured metadata improved prediction of BRAR-derived periodontal severity labels compared with either modality alone. The metadata ablation showed that age alone did not account for the multimodal improvement, with the findings supporting complementary contributions from age and the remaining structured metadata. The targeted expert analysis showed greater dentist–annotation disagreement among selected model-misclassified cases but, because it involved one dentist and a purposively selected subset, does not establish annotation uncertainty as the cause of model errors. External validation, automated localization, and evaluation of direct radiographic bone-resorption estimation followed by age normalization are important directions for future work.