Singular-Subspace Alignment in Norm-Constrained Asymmetric Low-Rank Updates
We study the singular-subspace geometry of an asymmetric low-rank update X=BA under a Frobenius-norm constraint. For the affine linearization of a smooth matrix loss, factorized first-order stationarity pairs singular subspaces of the gradient, while the factorized second-order condition selects the leading r subspaces exactly. For a nonlinear loss, let G and G̃ denote the gradients at the pretrained and adapted points. Under full-rank factors, an active constraint, a positive multiplier, and suitable spectral separation, we prove a Wedin-type bound: the alignment error is at most βc/δr(G,G̃), where β is the gradient-Lipschitz constant and c is the update budget. If βc<γG/4, this is bounded by 4βc/(3γG) relative to the pretrained gradient. Globally smooth constructions, including a convex one, attain the same asymptotic order. For r=1, exact alignment can fail outside this perturbative regime at a genuine factorized second-order stationary point. The results concern stationary-point geometry, not algorithmic convergence.
Authors
- Arnav Gupta (ORCID: https://orcid.org/0009-0003-0592-2530)
Publication Details
- Journal
- Zenodo (CERN European Organization for Nuclear Research)
- Published
- 2026-09-19
- DOI
- https://doi.org/10.5281/zenodo.22845045
- Primary Topic
- Tensor decomposition and applications
- Type
- preprint