Use of ELMs in Physics-Informed Machine Learning: Estimates on the Generalization Error in Learning ODE Solutions

Abstract Physics-Informed Neural Networks have been introduced to learn the solutions to Differential Problems. Extreme Learning Machines (ELMs), which are shallow feedforward architectures based on random projection, have been utilized along with other types of Neural Networks. Recently, ELM-based Physics-Informed Neural Networks have been combined with a functional interpolation technique called the Theory of Functional Connections, leading to an efficient method named the Extreme Theory of Functional Connections. The Extreme Theory of Functional Connections allows us to analytically satisfy the constraints, e.g., initial and/or boundary conditions that complete the differential problem removing one of the limitations of the traditional Physics-Informed Neural Networks frameworks: the need to learn the constraints alongside learning the solution within the domain. In this work, we provide upper bounds on the generalization error, in terms of the training error and number and choice of training points, of The Extreme Theory of Functional Connections in learning the solutions for the class of univariate Differential Problems. We fully cover three kinds of problems: first-order initial value problems, higher-order IVPs, and second-order two-point boundary value problems. The theoretical results for these three cases are validated with numerical experiments on some benchmark linear and non-linear problems. Furthermore, the resolution using the proposed method is tested by comparing the results obtained with the ones computed by MATLAB’s routines, and the proposed method outperforms the other solvers in terms of accuracy and computational efficiency.