Learning a Video Neural Memory with Bias Mitigation for Video Class-Incremental Learning

Video class-incremental learning retains old knowledge while learning new actions under limited replay memory. Temporal compression stores more videos but may lose temporal details and introduce discrepancies between compressed and original videos. We propose Video Neural Memory with Bias Mitigation, combining a Spatio-Temporal Neural Memorizer and Contrastive Bias Correction. The Spatio-Temporal Neural Memorizer preserves temporal and task-related information in compact neural video representations. Contrastive Bias Correction uses class-wise adaptation prompts and contrastive learning to reduce compression-induced feature bias. Experiments on three action recognition benchmarks show higher average incremental accuracy than the compared methods under the same replay-memory budgets, including a gain of 9.18% over the strongest baseline on Something-Something V2 with 18 steps.