An Efficient Decoupled Optimization Algorithm for a Class of Regression Models

This paper proposes an efficient Decoupled Optimization Algorithm (DOA) to address a class of regression problems commonly encountered in signal processing, system identification, and machine learning. Unlike existing methods such as proximal gradient algorithms and Alternating Direction Method of Multipliers (ADMM), the DOA algorithm incorporates the core principles of variable projection to resolve the coupling relationships between different parameters in the optimization process and express some parameters as functions of others, allowing for efficient optimization in a reduced-dimensional space. Furthermore, we investigate strategies for addressing these coupling relationships in both first-order and second-order settings, leading to the development of two variants of DOA: DOA-1 and DOA-2, each tailored to different application requirements. Finally, numerical experiments demonstrate that the DOA algorithm significantly accelerates convergence, confirming its effectiveness.