Beyond catastrophic forgetting : advancing continual learning for robust and fair medical image analysis

Traditional Deep Learning (DL) models struggle to adapt to evolving environments, making them ineffective for clinical scenarios where data continuously changes. Continual Learning (CL) offers a promising solution by enabling models to learn sequentially while addressing catastrophic forgetting. However, challenges remain in resource efficiency, generalization, adaptation, fairness and the inefficient integration of pretrained models, especially in medical image analysis. To tackle these challenges, this dissertation proposes several novel methods. For resource efficiency, we introduce Culprit-Prune-Net (CPN), a fixed-size network that sequentially learns tasks by dynamically allocating subnetworks using a culpability-based pruning strategy. To improve generalization, we develop the Generalizable Continual Classification Network (GC²), which enhances CPN with a mechanism for learning representations adaptable to unseen domains, and BoosterNet, which leverages a core network’s errors to improve single-domain generalization without altering its architecture or training process. For adaptation, we propose Continual-GEN, an online learning method that overcomes the need for explicit task boundary information by using an ensemble of clusters to measure image batch similarity, dynamically assigning or updating subnetworks. To address fairness, we introduce BiasPruner, which mitigates bias transfer by selectively forgetting biased decision-making units, enhancing minority group performance over time. Finally, we present Continual-Zoo, a framework that integrates pretrained networks into CL, reducing computational overhead and improving generalization for medical downstream tasks. Evaluations on multiple medical imaging datasets show our methods significantly outperform existing baselines and State-Of-The-Art (SOTA) CL techniques.