Singular-Subspace Alignment in Norm-Constrained Asymmetric Low-Rank Updates

We study the singular-subspace geometry of an asymmetric low-rank update X=BA under a Frobenius-norm constraint. For the affine linearization of a smooth matrix loss, factorized first-order stationarity pairs singular subspaces of the gradient, while the factorized second-order condition selects the leading r subspaces exactly. For a nonlinear loss, let G and G̃ denote the gradients at the pretrained and adapted points. Under full-rank factors, an active constraint, a positive multiplier, and suitable spectral separation, we prove a Wedin-type bound: the alignment error is at most βc/δr(G,G̃), where β is the gradient-Lipschitz constant and c is the update budget. If βc<γG/4, this is bounded by 4βc/(3γG) relative to the pretrained gradient. Globally smooth constructions, including a convex one, attain the same asymptotic order. For r=1, exact alignment can fail outside this perturbative regime at a genuine factorized second-order stationary point. The results concern stationary-point geometry, not algorithmic convergence.

Authors

Publication Details

Journal
Zenodo (CERN European Organization for Nuclear Research)
Published
2026-09-19
DOI
https://doi.org/10.5281/zenodo.22845045
Primary Topic
Tensor decomposition and applications
Type
preprint
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
preprint

Singular-Subspace Alignment in Norm-Constrained Asymmetric Low-Rank Updates

Arnav Gupta
Zenodo (CERN European Organization for Nuclear Research)
Tensor decomposition and applications
preprint

Singular-Subspace Alignment in Norm-Constrained Asymmetric Low-Rank Updates

Arnav Gupta
preprint en

Abstract

We study the singular-subspace geometry of an asymmetric low-rank update X=BA under a Frobenius-norm constraint. For the affine linearization of a smooth matrix loss, factorized first-order stationarity pairs singular subspaces of the gradient, while the factorized second-order condition selects the leading r subspaces exactly. For a nonlinear loss, let G and G̃ denote the gradients at the pretrained and adapted points. Under full-rank factors, an active constraint, a positive multiplier, and suitable spectral separation, we prove a Wedin-type bound: the alignment error is at most βc/δr(G,G̃), where β is the gradient-Lipschitz constant and c is the update budget. If βc<γG/4, this is bounded by 4βc/(3γG) relative to the pretrained gradient. Globally smooth constructions, including a convex one, attain the same asymptotic order. For r=1, exact alignment can fail outside this perturbative regime at a genuine factorized second-order stationary point. The results concern stationary-point geometry, not algorithmic convergence.

Zenodo (CERN European Organization for Nuclear Research)
Tensor decomposition and applications
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Singular-Subspace Alignment in Norm-Constrained Asymmetric Low-Rank Updates — Arnav Gupta · Zenodo (CERN European Organization for Nuclear Research) (2026) | TGRS Research Map | TGRS