GAIN: Gradual Adaptation for Continual Learning in Low-resource Environments

Continual Learning (CL) faces challenges such as catastrophic forgetting and data scarcity in low-resource tasks, which limit the model's ability to adapt to gradually changing tasks.We propose a general CL method for low-resource tasks to address these challenges called GAIN.This method addresses cross-domain continual learning through two approaches: (i) the Gradual Adaptation Module, which incrementally stacks lightweight adapters to effectively retain knowledge from previous tasks and adapt to new ones, thereby mitigating catastrophic forgetting, and (ii) a bottleneck layer size tuning strategy to improve learning efficiency in low-resource scenarios.Finally, we have extensive experiments on four datasets to validate the effectiveness and robustness of GAIN, showing that it alleviates catastrophic forgetting and enhances learning efficiency across various task types.The code is available at https://github.com/swguo/GAIN.