A general framework for neural delay differential equations with various delay types

This work proposes a general framework for Neural Delay Differential Equations (NDDEs), a class of continuous-depth neural networks, namely, the Generalized NDDEs (GNDDEs), incorporating various types of delays beyond the previously considered constant delays, including time-dependent, state-dependent, and time-state-dependent delays. Furthermore, we employ a simulation-free training strategy for the vector field, allowing the system reconstruction directly from the irregularly sampled time series without the prior model knowledge. Specifically, we perform the regression between the preprocessed target and parameterized vector fields, bypassing the need to numerically solve the differential equations as required in conventional time-series regression. Additionally, GNDDEs enable adaptive, model-free identification of the delay functions, along with the model-based identification of the system parameters. The experimental results demonstrate that the proposed framework exhibits the notable effectiveness and computational efficiency across a variety of delay differential equation problems, further broadening the applicability of continuous-depth neural networks in the delay system modeling.