Fast and Accurate Continual Test Time Domain Adaptation

Continual test time adaptation (CTTA) aims to adapt a model pre-trained on the source domain to continually changing target domains without using any data from the source domain. Current CTTA methods are either ineffective or inefficient. Some approaches suffer from insufficient adaptation, resulting in low accuracy. Conversely, other techniques excessively adapt the model, leading to high computation cost, error accumulation, and catastrophic forgetting. Addressing the latter two issues typically requires additional measures, such as data augmentation, which further compromises efficiency. To resolve these challenges, we introduce Fast and Accurate Continual Test Time Adaptation (FACTTA) which features dynamic adaptation and dynamic resolution. Specifically, the dynamic adaptation technique, guided by the proposed prediction confidence gain, initiates adaptation in inadequately adapted domains and terminates it in sufficiently adapted ones. This method prevents both excessive and insufficient adaptation, enhancing accuracy and efficiency. The dynamic resolution technique optimizes input resolution based on prediction confidences across various resolutions, further improving efficiency and accuracy. Extensive experiments conducted on image segmentation and image classification datasets demonstrate that FACTTA achieves superior accuracy while maintaining high efficiency compared to state-of-the-art approaches.