Shift-ACT: Test-time adaptation via shift-aware class-specific dynamic thresholding for domain generalization

Test-time adaptation (TTA) addresses the challenge of distribution shifts between training and test datasets by fine-tuning a model using only unlabeled test data in an online manner. This approach is well-suited for Domain Generalization (DG) settings, as it leverages real-time test data availability during the inference phase. Most existing TTA methods rely on pseudo-labels derived from the model’s predictions on test data. However, under test-time domain shifts, these pseudo-labels often suffer from noise, leading to performance degradation as the adaptation progresses. To overcome this limitation, we propose Shift-ACT, a novel TTA framework that employs a shift-aware, class-specific dynamic thresholding approach. This approach utilizes prior knowledge from source data, particularly class-specific confidence scores, and integrates them into a dynamic thresholding strategy during test-time adaptation. Our analysis shows that the extent of distribution shifts varies across classes, with larger confidence discrepancies between source and target classes indicating greater shifts. Shift-ACT adjusts class-specific thresholds dynamically, ensuring that classes with significant shifts, which likely include noisy samples, have reduced influence during adaptation. This method categorizes test samples into two groups: 1) Reliable samples, which are refined using an entropy minimization loss to align with the source distribution. 2) Unreliable samples, which are processed through a prototypical contrastive learning approach leveraging second-order feature statistics to align them with reliable target features. Extensive experiments on DG benchmark tasks, including VLCS, PACS, OfficeHome, and TerraIncognita, verify the effectiveness of our method. The code is available at https://github.com/saeedkarimi90/Shift-act .