Robust stochastic gradient descent for linearly constrained problems via adaptive barrier amplification

Methods based on Stochastic Gradient Descent using relaxed barrier functions provide a powerful framework for linearly constrained optimization but often suffer from sensitivity to hyperparameter settings and slow recovery from infeasible regions. Our investigation reveals that the optimal performance of standard relaxed barrier methods is confined to a narrow and unstable, ‘special-case’ optimization path, limiting their practical robustness. This paper introduces a novel algorithm designed to overcome these limitations through a Dynamic Barrier Amplification mechanism. This approach adaptively intensifies the corrective force of the barrier gradient in proportion to the magnitude of constraint violations, ensuring a strong push towards the feasible set specifically when iterates become highly infeasible. We provide a rigorous theoretical analysis, preserving the almost sure convergence guarantees of the baseline algorithm and deriving an explicit linear convergence rate to a neighborhood of the solution. Numerical results on challenging linearly constrained quadratic programming problems demonstrate that our algorithm exhibits superior robustness: it significantly reduces constraint violations in ill-conditioned scenarios compared to the standard method, acting as a reliability safety net, while maintaining competitive efficiency in nominal conditions.