XNODE: A XAI Suite to Understand Neural Ordinary Differential Equations

Neural Ordinary Differential Equations (Neural ODEs) have emerged as a promising approach for learning the continuous-time behaviour of dynamical systems from data. However, Neural ODEs are black-box models, posing challenges in interpreting and understanding their decision-making processes. This raises concerns about their application in critical domains such as healthcare and autonomous systems. To address this challenge and provide insight into the decision-making process of Neural ODEs, we introduce the eXplainable Neural ODE (XNODE) framework, a suite of eXplainable Artificial Intelligence (XAI) techniques specifically designed for Neural ODEs. Drawing inspiration from classical visualisation methods for differential equations, including time series, state space, and vector field plots, XNODE aims to offer intuitive insights into model behaviour. Although relatively simple, these techniques are intended to furnish researchers with a deeper understanding of the underlying mathematical tools, thereby serving as a practical guide for interpreting results obtained with Neural ODEs. The effectiveness of XNODE is verified through case studies involving a Resistor–Capacitor (RC) circuit, the Lotka–Volterra predator-prey dynamics, and a chemical reaction. The proposed XNODE suite offers a more nuanced perspective for cases where low Mean Squared Error values are obtained, which initially suggests successful learning of the data dynamics. This reveals that a low training error does not necessarily equate to comprehensive understanding or accurate modelling of the underlying data dynamics.