Improving Robustness of Post-hoc Calibration Against Common Corruptions By Learnable Augmentation

Various research has addressed the overconfidence problem, and we focus on improving the robustness of post-hoc calibration (e.g., temperature scaling, TS) when the test set shifts from the training set by image corruption. TS is greatly affected by the validation set, which previous work has proposed to perturb by Gaussian noise to improve calibration under domain drift. Inspired by this, we discovered that the same or similar augmentation on the validation set substantially improved TS under corrupted shift. We proposed a learnable and dynamic augmentation-based TS method, AugTS, which minimizes the maximum mean discrepancy (MMD) between the augmented validation and corrupted test set. Experiments on corrupted versions of CIFAR-10, CIFAR-100, and TinyImageNet show that AugTS can significantly improve calibration under corrupted shifts compared with competitive baselines.