A simultaneous framework for training neural ODEs using full discretization and large-scale nonlinear programming

Neural Ordinary Differential Equations (Neural ODEs) represent continuous-time dynamics with neural networks, offering a flexible, data-driven framework for system identification and control-oriented modeling. However, training Neural ODEs requires solving differential equations at every epoch, leading to high computational costs. This work investigates simultaneous optimization as an efficient alternative to standard sequential training. In particular, we present a collocation-based, fully discretized formulation and use IPOPT — a solver for large-scale nonlinear optimization — to jointly optimize collocation coefficients and neural network parameters. We demonstrate the proposed framework across synthetic and real-world case studies, highlighting its promise as a computationally efficient alternative to traditional training methods. Furthermore, we introduce a decomposition framework utilizing Alternating Direction Method of Multipliers (ADMM) to effectively coordinate sub-models among data batches. Together, these results underscore the potential of collocation-based simultaneous training pipelines for Neural ODEs.