Layerwise goal-oriented adaptivity for neural ODEs: an optimal control perspective

Abstract In this work, we propose a novel layerwise adaptive construction method for neural network architectures. Our approach is based on a goal–oriented dual-weighted residual technique for the optimal control of neural differential equations. This leads to an ordinary differential equation constrained optimization problem with controls acting as coefficients and a specific loss function. We implement our approach on the basis of a DG(0) Galerkin discretization of the neural ODE, leading to an explicit Euler time marching scheme. The resulting optimization problem is solved using the Adam algorithm and a BFGS method adapted to the H 1 topology induced by the regularization term. Finally, we apply our method to the construction of neural networks for the classification of data sets, where we present results for a selection of well known examples from the literature.

Authors

Institutions

Publication Details

Journal
Journal of Numerical Mathematics
Published
2026-11-17
DOI
https://doi.org/10.1515/jnma-2026-0008
Citations
1
Primary Topic
Model Reduction and Neural Networks
Type
article
Field-Weighted Citation Impact
1.18

Funders

Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
article

Layerwise goal-oriented adaptivity for neural ODEs: an optimal control perspective

Michael Hintermüller, Michael Hinze, Denis Korolev
1 citations
Journal of Numerical Mathematics
Model Reduction and Neural Networks
1.18
article

Layerwise goal-oriented adaptivity for neural ODEs: an optimal control perspective

Michael Hintermüller, Michael Hinze, Denis Korolev
article en
1 citations

Abstract

Abstract In this work, we propose a novel layerwise adaptive construction method for neural network architectures. Our approach is based on a goal–oriented dual-weighted residual technique for the optimal control of neural differential equations. This leads to an ordinary differential equation constrained optimization problem with controls acting as coefficients and a specific loss function. We implement our approach on the basis of a DG(0) Galerkin discretization of the neural ODE, leading to an explicit Euler time marching scheme. The resulting optimization problem is solved using the Adam algorithm and a BFGS method adapted to the H 1 topology induced by the regularization term. Finally, we apply our method to the construction of neural networks for the classification of data sets, where we present results for a selection of well known examples from the literature.

Journal of Numerical Mathematics
Humboldt-Universität zu Berlin (DE), Weierstrass Institute for Applied Analysis and Stochastics (DE), Universität Koblenz (DE)
Deutsche Forschungsgemeinschaft, Bundesministerium für Bildung und Forschung, Berlin Mathematics Research Center MATH+
Openalex Percentile: Top 25%
Model Reduction and Neural Networks
1.18
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon