Exploration of the generalization and hallucination in deep learning: a study based on imaging through scattering medium

With fast developments of computational power and algorithms, deep learning has made breakthroughs and been applied in many fields. However, generalization remains to be a major challenge, and the limited generalization capability severely constrains applications of deep learning in practice. The hallucinations issue is another unresolved conundrum faced by deep learning and large models. By leveraging a physical model of imaging through scattering media, we studied the lack of generalization to datasets and system response functions in deep learning respectively, identified their causes, and proposed universal solutions. The research also provides an explanation for hallucinations. In general, it enhances the interpretability of deep learning from a physics-based perspective. It will pave a way for direct interaction between deep learning and the physical world, facilitating the transition of deep learning from a demo model to a practical tool.