M20g Operational Learning: What the Charts Do - Classical Equivalences, a Symmetry Theorem and Three Closures
M20g Operational Learning What the Charts Do (v0.2) studies optimization in declared operational charts. It separates four choices often conflated in gradient methods: the chart in which parameters move, the arithmetic of the finite update, the model of gradient history and the observer reporting progress. Its central result classifies the symmetries of two chart-based updates. The note also identifies coordinatewise links with mirror descent, relates the room-2 chart to a standard stable-pole parameterisation of recurrent models, gives finite-step margins, limits what can be inferred from loss traces and proves that generic momentum histories are not crate histories. Most mechanisms are classical; no general optimizer improvement is claimed. Charts and updates For an increasing chart phi:I->V, loss L(w) and gradient g=L'(w), the note compares two exact updates. Chart-coordinate gradient descent is w+ = phi^(-1)(phi(w)-eta*g/phi'(w)). It is ordinary gradient descent on the transformed loss L o phi^(-1) and has first-order preconditioner 1/phi'(w)^2. The link update is w+ = phi^(-1)(phi(w)-eta*g), with first-order preconditioner 1/phi'(w). For the room charts phi=ell_k=ln^[k], define w_k = 1/ell_k' = product_(j=0)^(k-1) ell_j. The two preconditioners are then w_k^2 and w_k. In room 1 the link is unnormalised exponentiated gradient. In room 2 the updates use powers whose exponents depend exponentially on the step. Every coordinatewise link is unconstrained mirror descent. If Psi(w) = sum_i integral phi_i(w_i) dw_i, then grad(Psi)=phi. The room potentials include Psi_1(w) = w*ln(w)-w, Psi_2(w) = w*ln(ln(w))-li(w). The Burg chart phi(w)=-1/w generates the Itakura–Saito divergence and the link w+ = w/(1+eta*w*g). It is the chart of the parallel sum, not a room chart. Symmetry classification The note classifies all transformations preserving the form of the two updates. For chart-coordinate gradient descent, every symmetry is affine in chart coordinates: phi(T(w)) = a*phi(w)+b. The exact symmetries have a=+1 or a=-1, giving chart translations and reflections. For room k, these are precisely the room-k translations and reflections. Room 1 therefore carries ordinary scalings w->q*w and inversions w->q/w. Room 2 carries power transformations and their reflections: w -> w^(ln(q)), w -> exp(ln(q)/ln(w)). Allowing a fixed rescaling of the step admits affine dilations in chart space. Under ordinary parameter scaling, this selects the Box–Cox family, including the logarithm. The link update is more restrictive because it mixes a chart coordinate with a gradient covector in the original coordinate. Its symmetries must be affine in the original parameter. The Burg chart is the unique link chart with exact scale invariance. Exponentiated gradient is only scale-covariant after rescaling the step, and room-k links have no nontrivial symmetry for k>=2. A room chart is characterised by the transformations it converts into translations. Room-k chart descent respects those transformations exactly, while the corresponding link generally does not. This gives a selection rule: when a loss has room-k translation symmetry, room-k chart descent preserves that symmetry with an unchanged step. When step rescaling is allowed, symmetry alone no longer selects a chart; conditioning and finite-step behaviour become decisive. Natural gradient and the pole chart In one dimension, the Fisher-flat coordinate satisfies phi_F'(theta)=sqrt(I(theta)). Gradient descent in this coordinate is natural-gradient descent with the inverse chart as its retraction. If u=ell_(k-1)(theta) is a scale parameter with Fisher information C/u^2, then phi_F(theta)=sqrt(C)*ell_k(theta)+constant. Room-k chart descent is therefore natural-gradient descent when the preceding logarithmic coordinate is statistically a scale parameter. A concrete instance appears in linear recurrent units, which use |lambda| = exp(-exp(nu)). Writing the decay rate as rho=-ln(|lambda|) gives nu = ln(rho) = ln(ln(1/|lambda|)). This is room 2 on the reciprocal pole modulus and room 1 on the decay rate. For fitting one exponential decay to another, the infinite-horizon loss has the exact form L(rho,rho*) = 1/2*coth(rho)+1/2*coth(rho*)-coth((rho+rho*)/2). Its leading approximation is homogeneous of degree -1. With x=rho/rho*, ordinary gradient descent has a cubic plateau at large x, whereas logarithmic descent reduces the distance approximately linearly. For x_0>=2, ordinary gradient descent under the stated step condition needs at least (x_0^3-8)/18 steps to reach x<=2. Logarithmic descent reaches that region within ceil((x_0-2)/0.49) steps. At x_0=100, the direct computations reported in the note take 57,833 and 39 steps respectively. This establishes a conditioning advantage for the exact pole problem, not a new general-purpose optimizer. The surrounding parameterisation is already used in recurrent and state-space models. Three closures Loss-trace identifiability. A loss trace depends only on the loss expressed in chart coordinates and the initial chart point. Simultaneously reparameterising the model and chart leaves the trace unchanged. Chart scale and learning rate also form a gauge pair. In one dimension, with a known loss that is monotone along the trajectory, the chart derivative can be recovered on the explored interval: phi'(theta(t))^2 = L'(theta(t))^2/(-dL/dt). In two or more dimensions, a scalar trace cannot identify a general chart or metric. Infinite-dimensional families of loss-preserving transformations can change the parameter trajectory without changing the trace. Finite-step margins. Exact chart and link updates remain inside the room domain, but their first-order approximations may leave it. M20g derives trust ratios controlling the variation of the room velocity along a step and gives sufficient finite-step descent conditions. Floating-point range sharply restricts higher rooms. In double precision, the largest finite chart coordinates are approximately 709.78, 6.565 and 1.882 for rooms 1, 2 and 3. Room 3 therefore compresses the usable numerical domain into a very short interval, making still higher rooms impractical as ordinary optimization charts. Crate histories. Generic momentum dynamics do not satisfy the spectral restrictions of a crate. Two distinct underdamped Hessian eigenvalues in heavy-ball descent generate four distinct modes with the same decay and different frequencies. No crate can represent such a history, while an unrestricted Prony or matrix-pencil model represents it exactly. For plain gradient descent with two distinct excited eigenvalues, a crate representation is possible only on a measure-zero parameter set. At equal modal count, unrestricted exponential fitting can never have a larger best residual than crate fitting. This closes the crate-controller proposal negatively for exact quadratic systems. A noisy regularisation study remains logically possible but would require strong state-space and matrix-pencil baselines. Verdict Equal-tuning experiments support the symmetry theorem: room 1 succeeds on logarithmic or scale-symmetric losses; room 2 succeeds on a log-log quadratic and its power orbit; the Burg link succeeds on a scale-free relative-error loss; logarithmic descent on decay rate succeeds on the pole problem. Room-2 charts are not best on the tested rotated and diagonal quadratics or on Rosenbrock, where the matching symmetry is absent. Finite-step arithmetic also matters independently of continuous-time equivalence: room-1 chart descent and the Burg link share the same first-order flow and scale symmetry but behave differently because one uses an exponential step and the other a rational step. M20g establishes what operational charts contribute and where their reach ends. Coordinatewise links are mirror descent; chart descent is selected by the transformations made translational in its chart; the Burg chart is the unique exactly scale-invariant link; room-2 geometry is useful for decay parameters already expressed through log-rate coordinates; scalar traces identify charts only under restricted conditions; floating-point margins sharply limit higher rooms; and crates do not represent generic momentum histories. The rooms provide a systematic language for matching parameter geometry to symmetry and conditioning, not a universal optimization advantage. Status M20g Operational Learning What the Charts Do (v0.2) is a research note. Its principal positive results are the symmetry classification, mirror potentials, pole-loss analysis, trace-identifiability result and finite-step margins. Its principal negative result is the crate-history no-go. Adaptive geometry, optimizer-state transport, observer diagnostics and stochastic stability remain open. The note claims neither a training benchmark nor a general optimizer improvement.
Authors
- Paweł Łukasz Garycki
Publication Details
- Journal
- Zenodo (CERN European Organization for Nuclear Research)
- Published
- 2026-10-19
- DOI
- https://doi.org/10.5281/zenodo.23469916
- Primary Topic
- Stochastic Gradient Optimization Techniques
- Type
- preprint