Overcoming catastrophic forgetting in heterogeneous federated learning

Federated learning (FL) has gained prominence in the medical domain for enabling collaborative machine learning across decentralized datasets while preserving patient privacy. However, key challenges such as catastrophic forgetting and local overfitting hinder the effectiveness of FL models in heterogeneous healthcare environments. This thesis addresses these challenges through two complementary studies. The first study introduces Federated Impressions, a synthetic data generation method that extracts the knowledge of the global model without compromising privacy. By training the local model on these impressions, the method mitigates catastrophic forgetting, allowing the model to retain critical knowledge while generalizing effectively across diverse datasets. This approach ensures robustness and adaptability to evolving medical data distributions. The second study enhances FL for medical imaging by leveraging Vision Transformers (ViTs) in a DualPrompt architecture. Prompts are divided into shared and group prompts: shared prompts capture generalized medical features and are trained exclusively on Federated Impressions alongside local data to prevent overfitting, while group prompts adapt to specific client tasks using local data. This method effectively mitigates catastrophic forgetting, preserving personalization through group prompts while maintaining global generalization via shared prompts. It strikes a balance between retaining global medical knowledge and adapting to localized data distributions. Experimental evaluations on benchmark medical datasets demonstrate that the proposed approaches improve global generalization by avoiding local overfitting. These advancements make FL more robust and effective in heterogeneous medical environments, addressing critical challenges in privacy, scalability, and catastrophic forgetting. This thesis lays the groundwork for scalable and adaptive FL frameworks tailored to medical applications, fostering innovation in privacy-preserving healthcare AI.