A novel Sigmexd activation function to enhance Physics-Informed Neural Networks (PINNs) for differential equations with a perspective on the role of activation functions in PINNs

Physics-Informed Neural Networks (PINNs) is a rapidly evolving machine learning technique that leverages deep learning strengths to solve physical equations. In PINNs, Activation Functions (AFs) are key to capturing nonlinearities of complex real-world problems. In addition, the inherent randomness of neural-network learning makes ‘seeding’ an important consideration for ensuring reproducibility and consistency. In this background, this investigation focuses on enhancing PINN activation to achieve reliable solutions across diverse differential equations. The key objectives include: a) Evaluating the role of different AFs and seeding arrangements in PINNs; b) Proposing a novel AF specifically suitable for PINN-based computational modeling; and c) Providing preliminary recommendations on PINN activation and seeding selection. In achieving these objectives, nine different differential-equation setups have been considered with three Ordinary Differential Equations (ODEs) and six Partial Differential Equations (PDEs). Six different AFs, including the newly proposed Sigmexd AF and a range of widely-utilized AFs, have been comprehensively compared with respect to PINNs. The Sigmexd AF demonstrates superior performance, achieving up to five-fold improvements compared to the next-best AF. The practical impact of Sigmexd has been clearly demonstrated, as a versatile AF that can consistently deliver high levels of accuracy across a wide range of linear and nonlinear differential equations of varying orders. The analyses further reveal an interesting interaction between seeding arrangements and AFs. A preliminary activation selection setup is recommended for PINN-based modeling, offering foundational-level guidance for optimal activation and seeding. To the best of the authors’ knowledge, this represents the first systematic approach of its kind. • Assessing the role of activation functions and seeding in PINNs. • A novel Sigmexd activation function to improve PINN learning and accuracy. • Preliminary recommendations on optimal activation and seeding selection for PINNs.