Towards Unbiased Continual Learning: Avoiding Forgetting in the Presence of Spurious Correlations

Continual Learning (CL) has emerged as a paramount area in Artificial Intelligence (AI) because of its ability to learn multiple tasks sequentially without significant performance degradation. Despite the growing interest in CL frameworks, a critical aspect must be addressed: the inherent biases within training data. In this work, we show that, if overlooked, these biases can significantly impair the efficacy of continual learning models by inducing reliance on suboptimal shortcuts during data stream and memory retention, exacerbating catastrophic forgetting. In response, we present Learning without Shortcuts (LwS), which sets forth two primary objectives: (i) to identify and mitigate the exploitation of spurious correlations within the data stream and (ii) to develop a novel mechanism that constructs a fair memory buffer used in replay-based CL strategies. Our buffer construction strategy exploits the model confidence in a given example to balance the portion of samples per class, hence their contribution when replay activates. Unlike existing methods, LwS is agnostic to protected attributes, and results highlight that the proposed solution is indeed resilient to spurious correlations in CL settings. Code is available at https://github.com/aimagelab/mammoth.