Unveiling the optimization process of physics informed neural networks: How accurate and competitive can PINNs be?

This study investigates the potential accuracy boundaries of physics-informed neural networks, contrasting their approach with previous similar works and traditional numerical methods. We find that selecting improved optimization algorithms significantly enhances the accuracy of the results. Simple modifications to the loss function may also improve precision, offering an additional avenue for enhancement. Despite optimization algorithms having a greater impact on convergence than adjustments to the loss function, practical considerations often favor tweaking the latter due to ease of implementation. On a global scale, the integration of an enhanced optimizer and a marginally adjusted loss function enables a reduction in the loss function by several orders of magnitude across diverse physical problems. Consequently, our results obtained using compact networks (typically comprising 2 or 3 layers of 20-30 neurons) achieve accuracies comparable to finite difference schemes employing thousands of grid points. This study encourages the continued advancement of PINNs and associated optimization techniques for broader applications across various fields. • We explore how the optimization process influences convergence and accuracy of Physics-Informed Neural Networks. • We suggest adjustments to the BFGS optimization algorithm and the mean squared error loss function, which could significantly improve precision by several orders of magnitude. • We explain our findings by analyzing the conditioning of the Hessian matrix and the spectrum of its eigenvalues. • We demonstrate that our scheme is applicable to a wide range of physical problems, yielding more accurate results while reducing the computational cost.