Backpropagation as Dirac-bracket dynamics of dissipative training

Gradient-based training of a parametric model is the motion of a damped particle in the potential set by the loss. Using the Bateman–Galley doubling of the degrees of freedom, which gives dissipative systems an action principle, we write gradient flow, the heavy-ball method and Nesterov's accelerated gradient method as Hamiltonian systems. We show that, within this Hamiltonian description, backpropagation is Dirac evolution. The activations of a feed-forward network are treated as coordinates of phase space, and the layer equations as constraints. These constraints are second class for every architecture, and Hamilton's equations with the resulting Dirac bracket reproduce training with the exact gradient of the loss, although the Hamiltonian contains only the cost of the network output. The Dirac bracket of an activation with the momenta of the parameters is its Jacobian, the inverse of the constraint matrix contains both the forward and the reverse modes of automatic differentiation, and the errors propagated by backpropagation are the Lagrange multipliers of the layer constraints: the constraint forces of a kinematic chain under the load of the cost. These results hold in the full doubled phase space, not only in the physical limit. The network's part of the constraint algebra is the same for every optimizer, Nesterov's look-ahead adds one layer of momentum-dependent constraints that couple the two copies of the doubled system, and gradient flow is obtained by constructing the Dirac bracket in stages. We confirm these statements numerically to machine precision and train a network by integrating the Dirac flow of its constraints, with no backward pass in the computation.

Authors

Institutions

Publication Details

Journal
Zenodo (CERN European Organization for Nuclear Research)
Published
2026-11-18
DOI
https://doi.org/10.5281/zenodo.24352737
Primary Topic
Stochastic Gradient Optimization Techniques
Type
preprint
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
preprint

Backpropagation as Dirac-bracket dynamics of dissipative training

N. L. Holanda, Ivan Eugênio da Cunha
Zenodo (CERN European Organization for Nuclear Research)
Stochastic Gradient Optimization Techniques
preprint

Backpropagation as Dirac-bracket dynamics of dissipative training

N. L. Holanda, Ivan Eugênio da Cunha
preprint en

Abstract

Gradient-based training of a parametric model is the motion of a damped particle in the potential set by the loss. Using the Bateman–Galley doubling of the degrees of freedom, which gives dissipative systems an action principle, we write gradient flow, the heavy-ball method and Nesterov's accelerated gradient method as Hamiltonian systems. We show that, within this Hamiltonian description, backpropagation is Dirac evolution. The activations of a feed-forward network are treated as coordinates of phase space, and the layer equations as constraints. These constraints are second class for every architecture, and Hamilton's equations with the resulting Dirac bracket reproduce training with the exact gradient of the loss, although the Hamiltonian contains only the cost of the network output. The Dirac bracket of an activation with the momenta of the parameters is its Jacobian, the inverse of the constraint matrix contains both the forward and the reverse modes of automatic differentiation, and the errors propagated by backpropagation are the Lagrange multipliers of the layer constraints: the constraint forces of a kinematic chain under the load of the cost. These results hold in the full doubled phase space, not only in the physical limit. The network's part of the constraint algebra is the same for every optimizer, Nesterov's look-ahead adds one layer of momentum-dependent constraints that couple the two copies of the doubled system, and gradient flow is obtained by constructing the Dirac bracket in stages. We confirm these statements numerically to machine precision and train a network by integrating the Dirac flow of its constraints, with no backward pass in the computation.

Zenodo (CERN European Organization for Nuclear Research)
Universidade do Estado de Santa Catarina (BR)
Stochastic Gradient Optimization Techniques
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon