HCT-Mammo: A Compact Hybrid CNN-Transformer for Benign–Malignant Mammogram Classification with Uncertainty and Cross-Dataset Evaluation

The difficulty in comparing public-dataset results for benign–malignant mammogram classification is due to patient overlap, lesion crops, and a lack of independent testing. This study uses 896×448 breast-cropped mammograms, and the evaluated model is a hybrid convolutional neural network (CNN)–Transformer (HCT-Mammo) with a truncated ResNet-18 stem, a 4-layer Transformer encoder, and 4.91 M parameters. Patient-level separation, training, and model selection were done on the Curated Breast Imaging Subset of DDSM (CBIS-DDSM), and the models were frozen for zero-shot evaluation on the Chinese Mammography Database (CMMD). Sigmoid Confidence, Temperature Scaling and Monte Carlo (MC) Dropout were evaluated with calibration and risk–coverage analyses, and Grad-CAM and Attention Rollout were compared to lesion masks and chance baselines. The mean area under the receiver operating characteristic curve (AUROC) of HCT-Mammo was 0.780 ± 0.005 across three seeds, with a median-seed AUROC of 0.781 (95% confidence interval 0.742–0.821). HCT-Mammo did not differ significantly from EfficientNet-B0 and ResNet-50 (DeLong p = 0.59 and p = 0.33), whereas ViT-S/16 was poorer (p = 0.002). On CMMD, AUROC decreased to 0.664, and the expected calibration error after temperature scaling increased from 0.059 to 0.121. MC-Dropout offered no obvious improvement in calibration or error ranking, internally or externally. The mass localisation hit rate for Grad-CAM was 35.5%, compared with 24.4% for Attention Rollout and 4.7% for a breast-uniform baseline. The compact hybrid was competitive in-domain, but its discrimination and calibration were worse across datasets. Both datasets contain only abnormal examinations; therefore, the results reflect lesion characterization rather than screening.