Human activity recognition under low-light conditions using frozen CLIP with lightweight adaptation

Abstract Human activity recognition (HAR) in low-light conditions remains challenging due to photometric domain shift, reduced contrast, altered colour statistics, and noise patterns absent from standard RGB pre-training data, causing models to degrade even after fine-tuning. To address this, we propose a lightweight adaptation framework leveraging frozen CLIP representations that ensures computational efficiency while delivering competitive performance across the three evaluated benchmarks. Our approach extracts embeddings from uniformly sampled frames without backbone updates, avoiding the computational overhead of full fine-tuning. By employing lightweight parameter-efficient adapters, we enable effective adaptation with minimal computational overhead. This design yields measurable accuracy improvements under low-light photometric shift. For instance, temporal adapters with only 88.8K parameters improve ARID v1.5 accuracy from 68.37 ± 0.49% to 74.38 ± 0.13%, an absolute gain of 6.01 percentage points. We further demonstrate the benefits of this approach in other settings: (1) linear probing achieves 87.26 ± 0.50% accuracy on UCF101 at 17.44 GFLOPs, within 6% of fine-tuned SlowOnly (93.16% at 27.3 GFLOPs) with 36% fewer computations, highlighting frozen CLIP’s efficiency as a baseline for standard datasets; and (2) zero-shot classification shows a performance gap of approximately 15–46 percentage points across datasets, underscoring the necessity of task-specific adaptation. All lightweight adapters operate at essentially the same cost as the frozen backbone since temporal adapters, LoRA, attention pooling, and prompt tuning each add under 0.2% additional FLOPs. These results establish frozen vision-language models with lightweight adaptation as a scalable and efficient framework for HAR under photometric domain shift, demonstrated on low-light infrared and standard RGB benchmarks, achieving meaningful gains with no backbone retraining required.