Deep Neural Networks Inspired by Differential Equations

Deep learning has emerged as an important technology in fields such as computer vision, scientific computing, and dynamical systems, thereby driving major advancements across these domains. However, neural networks persistently face challenges related to theoretical understanding, interpretability, and generalization. Thus, researchers are increasingly adopting a differential equations perspective to propose a unified theoretical framework and systematic design methodologies. We establish a comprehensive taxonomy of the literature from two distinct perspectives. First, we classify the models based on their mathematical formulations, covering ordinary, partial, and stochastic differential equations (ODEs, PDEs, and SDEs). Second, we categorize them by their methodological roles, distinguishing between equation-guided networks and networks designed for solving equations. Moreover, we conduct a qualitative meta-analysis supported by consolidated evidence summaries from existing literature to illustrate their characteristics and performance. Finally, we explore promising research directions in integrating differential equations with deep learning to offer new insights for developing computational methods with improved interpretability and generalization.