Suppressing catastrophic forgetting in deep-learning-based imaging through scattering media

Scattering media such as fog or frosted glass diffusely spread transmitted light, making it difficult to observe objects located behind them. Deep-learning-based scattering imaging addresses this challenge by training neural networks on pairs of images before and after scattering, allowing reconstruction of the original object from a speckle image. Once trained, these models can reconstruct objects from unseen speckled images. However, when employing continual learning, deep learning models suffer from catastrophic forgetting, where learning a new task causes a degradation in performance on previously learned tasks. In this study, we investigate methods to suppress catastrophic forgetting in deep-learning-based scattering imaging. We define Task 1 as the classification of Japanese cursive characters from the KMNIST dataset, and Task 2 as the classification of handwritten digits from the MNIST dataset. After sequential training—first on KMNIST and then on MNIST—the Task 1 accuracy dropped from 85.2% to 65.2%, indicating catastrophic forgetting. To address this, we apply two strategies: the rehearsal method, which reuses past data, and the elastic weight consolidation (EWC) method, which preserves critical parameters without using prior data. The rehearsal method maintained Task 1 accuracy at 85.7%, effectively mitigating forgetting. The EWC method achieved Task 1 accuracy between 80.4% and 86.07%, demonstrating its effectiveness without relying on past data. These findings confirm that both rehearsal and EWC methods are effective in overcoming catastrophic forgetting in deep-learning-based scattering imaging.