Evolution of physics-informed neural networks: Recent architectural variants and optimization strategies

Physics-Informed Neural Networks (PINNs) are a machine learning technique that directly incorporates the governing physics of problems, such as partial differential equations (PDEs) and ordinary differential equations (ODEs), into the neural network architecture. The primary goal of PINNs is to approximate solutions while satisfying given constraints and minimizing the residuals of the differential equations. PINNs have been employed to solve various problems, including integro-differential equations, fractional differential equations, and stochastic PDEs. Over the past two years, significant advancements have addressed the challenges associated with PINNs, resulting in notable improvements in accuracy and performance. This article provides a comprehensive summary of the latest methodologies contributing to these advancements, focusing on innovations in hyperparameter optimization and novel PINN variants inspired by other neural networks. Examples include MultiInNet-PINN, Transformer-based PINNs such as Tr-PINN and PINNsFormer, as well as PINNs incorporating attention mechanisms and recurrent neural network (RNN) architectures (PIANN). Additionally, recent research on domain decomposition techniques in PINN architectures are highlighted. By consolidating recent architectural and algorithmic advances, this research identifies critical research opportunities for enhancing the reliability, efficiency, and broader applicability of PINNs in scientific computing.