MOST: MR reconstruction Optimization for multiple downStream Tasks via continual learning

Motivation: This research aims to address the problem of performance degradation when a reconstruction network and a downstream network are cascaded. The proposed solution, MOST, optimizes a MR reconstruction network for multiple downstream tasks. Goal(s): Our objective is to sequentially finetune a reconstruction network using losses from multiple downstream tasks while preventing catastrophic forgetting such that the same reconstruction network can be used for the multiple tasks. Approach: We introduce replay-based continual learning into finetuning for multiple downstream tasks. Results: Our method successfully circumvents catastrophic forgetting, exhibiting stable performance across all downstream tasks, enabling a single reconstruction network to be used for multiple tasks. Impact: When k-space reconstruction and downstream tasks are performed using two separate networks (individually optimized), the cascade may introduce suboptimal results. Here, we propose a solution when multiple downsteam tasks exist, addressing challenges in realistic user environment.