The First- and Second-Order Features Adjoint Sensitivity Analysis Methodologies for Fredholm-Type Neural Integro-Differential Equations: An Illustrative Application to a Heat Transfer Model—Part II

This work illustrates the application of the “First-Order Features Adjoint Sensitivity Analysis Methodology for Neural Integro-Differential Equations of Fredholm-Type” (1st-FASAM-NIDE-F) and the “Second-Order Features Adjoint Sensitivity Analysis Methodology for Neural Integro-Differential Equations of Fredholm-Type” (2nd-FASAM-NIDE-F) to a paradigm heat transfer model. This physically based heat transfer model has been deliberately constructed so that it can be represented either by a neural integro-differential equation of a Fredholm type (NIDE-F) or by a conventional second-order “neural ordinary differential equation (NODE)” while admitting exact closed-form solutions/expressions for all quantities of interest, including state functions and first-order and second-order sensitivities. This heat transfer model enables a detailed comparison of the 1st- and 2nd-FASAM-NIDE-F versus the recently developed 1st- and 2nd-FASAM-NODE methodologies, highlighting the considerations underlying the optimal choice for cases where the neural net of interest is amenable to using either of these methodologies for its sensitivity analysis. It is shown that the 1st-FASAM-NIDE-F methodology enables the most efficient computation of exactly determined first-order sensitivities of the decoder response with respect to the optimized NIDE-F parameters, requiring a single “large-scale” computation for solving the 1st-Level Adjoint Sensitivity System (1st-LASS), regardless of the number of weights/parameters underlying the NIDE-F decoder, hidden layers, and encoder. The 2nd-FASAM-NIDE-F methodology enables the computation, with unparalleled efficiency, of the second-order sensitivities of decoder responses with respect to the optimized/trained weights.