Class weights miscalibrate chest X-ray classifiers by a predictable amount: calibration, prevalence shift and subgroup noise in a reproducible ChestX-ray14 baseline

Class-weighted binary cross-entropy is a common default for multi-label chest X-ray classification, and models are usually judged by mean AUROC alone. We train DenseNet121 (three seeds) and EfficientNet-B0 on NIH ChestX-ray14 with the official patient-wise split and study what AUROC does not show. Seed-to-seed variation is small (test mean AUROC 0.801 to 0.803, standard deviation 0.001), and averaging the three DenseNet121 runs gives 0.814. The predicted probabilities, however, are 2.3 to 16 times the true prevalence (mean expected calibration error, ECE, 0.26 to 0.29). We show that this inflation is almost exactly what the class weights predict: subtracting the logarithm of each label's positive weight from the logits, a correction that needs no data, lowers mean ECE to 0.016 to 0.018, close to per-class Platt scaling fitted on the validation set (0.011), and the fitted Platt intercepts track the same offsets. Neither correction fully survives a shift in label prevalence: the official test split has more findings than the validation split (62% against 43% of images), and pneumothorax remains under-predicted after both. Finally, we show how a subgroup table can mislead: a single hernia case among 8,290 test images of patients under 40 made that group look up to 0.04 worse and unstable across seeds; with labels restricted to at least 10 positives, the gap disappears. Code, predictions and a browser demo whose outputs match the Python pipeline within 0.2 percentage points are public.