Introducing the Second-Order Features Adjoint Sensitivity Analysis Methodology for Neural Ordinary Differential Equations—I: Mathematical Framework

This work introduces the mathematical framework of the novel “First-Order Features Adjoint Sensitivity Analysis Methodology for Neural Ordinary Differential Equations” (1st-FASAM-NODE). The 1st-FASAM-NODE methodology produces and computes most efficiently the exact expressions of all of the first-order sensitivities of NODE-decoder responses with respect to the parameters underlying the NODE’s decoder, hidden layers, and encoder, after having optimized the NODE-net to represent the physical system under consideration. Building on the 1st-FASAM-NODE, this work subsequently introduces the mathematical framework of the novel “Second-Order Features Adjoint Sensitivity Analysis Methodology for Neural Ordinary Differential Equations (2nd-FASAM-NODE)”. The 2nd-FASAM-NODE methodology efficiently computes the exact expressions of the second-order sensitivities of NODE decoder responses with respect to the NODE parameters. Since the physical system modeled by the NODE-net necessarily comprises imprecisely known parameters that stem from measurements and/or computations subject to uncertainties, the availability of the first- and second-order sensitivities of decoder responses to the parameters underlying the NODE-net is essential for performing sensitivity analysis and quantifying the uncertainties induced in the NODE-decoder responses by uncertainties in the underlying uncertain NODE-parameters.