Efficient Deep Learning Models for Physics Simulation

Many natural and engineered systems are governed by partial differential equations (PDEs), spanning atomic interactions in molecular systems to large scale cosmological dynamics. Solving these PDEs is essential for deepening our understanding of complex physical phenomena, enabling precise predictions, and guiding informed decision-making across diverse scientific and engineering fields. Numerical solvers are widely employed for simulating and predicting PDEs, particularly as many PDEs are challenging and infeasible to be solved analytically. These solvers typically discretize the continuous domain into a grid, converting differential equations into algebraic equations via methods such as finite difference, finite element, finite volume, or spectral approaches. Recent advances in machine learning, combined with the success of deep learning across many fields, have unlocked new possibilities for modeling com plex sub-scale physical processes and developing efficient neural-network-based PDE solvers. In these approaches, neural networks parameterize the solution function of the target equation or approximate the solution operator itself, providing a flexible alternative to traditional solvers. Compared to numerical solvers, neural PDE solvers tend to be more tolerant of coarser discretiza tions and can eliminate the need for fine meshing, making them adaptable to various domains with reduced computational overhead. Additionally, since these models can exploit the patterns directly from data, knowledge of the precise underlying equations is not strictly necessary, offering a streamlined and efficient approach to complex physics simulations. In this thesis, we discuss the efforts in developing and build neural-network based models for accurate and efficient prediction of a variety of physical systems. We first introduce Fluid Graph Networks (FGN) and Graph neural networks-Accelerated Molecular Dynamics (GAMD), two data-driven models parameterized with message passing neural networks, for efficient particle based system simulation. We further present a series of Transformer-based models for modeling various physics phenomena including turbulent flow and global weather dynamics. The first is Operator Transformer (OFormer), which features a Transformer encoder-decoder framework that can be flexibly applied to different discretizations. To improve the scalability of the Transformer on higher dimensional problems, we next propose a axial factorized attention that greatly reduces the computational cost associated with high-dimensional grid. We further extend the proposed factorized attention mechanism to the sphere for accurate and efficient global weather forecasting. Lastly, we present a generative neural PDE solver drawing inspiration from recent progress of dif fusion probabilistic models to improve the simulation robustness on turbulent time-dependent systems.