Layerwise goal-oriented adaptivity for neural ODEs: an optimal control perspective
Abstract In this work, we propose a novel layerwise adaptive construction method for neural network architectures. Our approach is based on a goal–oriented dual-weighted residual technique for the optimal control of neural differential equations. This leads to an ordinary differential equation constrained optimization problem with controls acting as coefficients and a specific loss function. We implement our approach on the basis of a DG(0) Galerkin discretization of the neural ODE, leading to an explicit Euler time marching scheme. The resulting optimization problem is solved using the Adam algorithm and a BFGS method adapted to the H 1 topology induced by the regularization term. Finally, we apply our method to the construction of neural networks for the classification of data sets, where we present results for a selection of well known examples from the literature.
Authors
- Michael Hintermüller (ORCID: https://orcid.org/0000-0001-9471-2479)
- Michael Hinze (ORCID: https://orcid.org/0000-0001-9688-0150)
- Denis Korolev (ORCID: https://orcid.org/0000-0001-5686-7699)
Institutions
- Humboldt-Universität zu Berlin (DE)
- Weierstrass Institute for Applied Analysis and Stochastics (DE)
- Universität Koblenz (DE)
Publication Details
- Journal
- Journal of Numerical Mathematics
- Published
- 2026-11-17
- DOI
- https://doi.org/10.1515/jnma-2026-0008
- Citations
- 1
- Primary Topic
- Model Reduction and Neural Networks
- Type
- article
- Field-Weighted Citation Impact
- 1.18
Funders
- Deutsche Forschungsgemeinschaft
- Bundesministerium für Bildung und Forschung
- Berlin Mathematics Research Center MATH+