Cross-Domain Generalization of Deep Learning Architectures for Cephalometric Landmark Detection: A Dual-Dataset and Multi-Device Benchmark

Background/Objectives: Deep learning models for cephalometric landmark detection report near-ceiling accuracy on single benchmarks, yet most are trained and tested on the same dataset. Whether the best in-domain architecture remains best out-of-domain has not been systematically quantified. Methods: Four architecture families (heatmap CNN, two-stage cascade, coordinate-regression Vision Transformer, pretrained ResNet-50) were each trained on two independently sourced datasets—ISBI 2015 (400 images, one device) and Aariz (1000 images, seven devices)—and evaluated on both, over their 19 shared landmarks in millimeters. All axes used three seeds (mean ± SD): the cross-dataset matrix, leave-one-device-out shift, balanced joint training, a pretrained-versus-scratch ablation, and a landmark-level breakdown. Results: In-domain mean radial error (MRE) was 2.47–3.04 mm (Aariz) and 4.12–5.55 mm (ISBI); cross-dataset error rose steeply (Generalization Drop—the relative increase in error out-of-domain—155–426%). The most accurate model in-domain (a from-scratch heatmap CNN, 2.47 mm) showed the largest drop, and every pretrained estimate fell below every from-scratch estimate (family means 248% vs. 380%): in-domain ranking did not predict cross-domain ranking. Leave-one-device-out revealed reproducible device-specific shift (held-out MRE 1.89–7.94 mm). Inter-observer variability was 0.53 mm, so cross-domain errors were 15–44× the human band. Balanced joint training reduced the cross-domain gap for all four architectures (both domains ≈ 2.1–3.5 mm) without harming in-domain accuracy. Pretraining more than halved cross-domain error from Aariz to ISBI (7.73 vs. 16.36 mm) but not in reverse, supporting the mechanism in one direction rather than uniformly. A-point, B-point, Nasion, and Menton all exceeded the 2 mm clinical threshold out-of-domain, including points localized to sub-millimeter accuracy in-domain. Conclusions: Single-dataset accuracy substantially overstates clinical generalizability, and the best in-domain architecture is not the most transferable, so in-domain leaderboards are an unreliable basis for selecting a model for clinical deployment; balanced multi-source training recovers most of the loss across the architectures tested.