Ph.D. Forum: Responsive On-Device DNN Adaptation for Non-Stationary Mobile Environments

On-device adaptation to unpredictable domain shifts is crucial for mobile applications like autonomous driving and augmented reality. Test-time adaptation (TTA) offers a solution by tuning model parameters with unlabeled live data before prediction, but its forward-backward-reforward pipeline increases latency, compromising responsiveness. This paper presents AdaShadow, a responsive TTA framework for non-stationary mobile data distribution and resource dynamics via selective updates of critical layers. It utilizes a backpropagation-free assessor for quick identification of critical layers, a unit-based runtime predictor for latency estimation, and an online scheduler for effective updates. Evaluations show AdaShadow reduces adaptation latency by up to 3.5× and improves accuracy by up to 25.4%, with low memory and energy costs.