Operator-Splitting Architectures for Neural Ordinary Differential Equations

Neural Ordinary Differential Equations (Neural ODEs) replace the discrete layer stack of a residual network by a continuous-time vector field parameterised by a neural network. This reformulation opens a dialogue with classical numerical analysis: every time integrator can be read as a particular network architecture. We study operator splitting as an architectural principle and apply it. The vector field is decomposed into two sub-networks, typically a linear part advanced exactly by the matrix exponential and a nonlinear part advanced by a Runge–Kutta solver, which are composed by Lie–Trotter or Strang splitting (LT-NODE and S-NODE). We prove that the local splitting error is O(τ3)for Lie–Trotter and O(τ3) for Strang splitting, with the corresponding global convergence; that splitting preserves the contraction rate μA + μB of one-sided Lipschitz sub-fields for every step size; and we derive and verify a split adjoint for gradient computation. In our experimental studies with five independent seeds, validation-based model selection and baselines including GRU, ODE-RNN, unsplit additive Neural ODEs and Neural CDEs (six UEA datasets, four missing-data rates, a synthetic system, a spiral benchmark and a stiff reaction–diffusion system), we show differentiated pictures. At least splitting does not improve accuracy by itself relative to an unsplit model with the same additive structure, and it is not universally cheaper. It pays off when one part of the dynamics is stiff and can be integrated exactly, for example for a reaction–diffusion system with known diffusion, S-NODE, we reach the accuracy of the unsplit hybrid model with 19 times fewer function evaluations and 23 times shorter training. We also included a report of an encoder defect that caused the chance-level spiral result.