Linear Algebra Problems Solved with Damped Dynamical Systems on the Stiefel Manifold

Abstract This article presents a unified framework for constrained optimization solved by using damped dynamical systems on the Stiefel manifold, combining variational principles, projected-gradient methodologies, and asymptotic stability theory. For smooth objective functions defined on the Stiefel manifold, first-order optimality conditions are derived using both intrinsic tangent-space projections and classical Lagrange multiplier formulations, which naturally lead to second-order damped dynamical systems whose equilibrium points coincide with the Karush–Kuhn–Tucker solutions of the constrained optimization problem. Two complementary formulations are studied in detail: a Lagrange-based approach in which constraint satisfaction is enforced through dynamically evolving multipliers, and a projected-gradient formulation in which the dynamics evolve intrinsically on the tangent bundle of the Stiefel manifold. It is shown that both formulations admit identical stationary solutions, and explicit analytical relationships between the Lagrangian and projected dynamics are established. The proposed framework is applied to two canonical problems, namely the linear eigenvalue problem for computing invariant subspaces associated with the smallest eigenvalues of a symmetric positive definite matrix and the orthogonal Procrustes problem formulated in the Frobenius norm, for which explicit expressions for the gradients, multiplier dynamics, and reduced systems are derived. A rigorous asymptotic stability analysis is carried out by linearizing the resulting first-order systems and characterizing the spectra of the associated reduced Jacobian operators acting on the tangent space of the constraint manifold, leading to sufficient conditions for asymptotic stability that clarify the role of damping parameters in guaranteeing convergence. Numerical implementations are developed by discretizing the proposed second-order dynamical systems using a symplectic Euler scheme that preserves the qualitative stability properties of the continuous-time models, resulting in algorithms that rely on standard linear algebra operations, including Sylvester equation solvers and thin singular value decompositions, and that enable a direct comparison between intrinsic and extrinsic approaches to optimization on the Stiefel manifold.