DMSO: A Dynamic Momentum-Smoothing Optimizer for Learned Image Compression

Learned Image Compression (LIC) has rapidly evolved and recently surpassed traditional methods in Rate-Distortion (R-D) performance. However, most LIC approaches improve network architectures while increasing computational overhead and overlooking using optimizers tailored specifically for LIC. This paper proposes a Dynamic Momentum-Smoothing Optimizer (DMSO) tailored for LIC to achieve faster convergence and better R-D performance. Specifically, DMSO leverages historical gradient information to smooth the optimization process dynamically, thereby reducing in-stability from gradient oscillations. Furthermore, DMSO introduces a novel Enhanced Second-order Momentum mechanism to mitigate cumulative noise and align momentum updates more closely with the true gradient. Experimental results demonstrate that DMSO operates as a universal optimization plugin for LIC methods, achieving faster and more stable convergence while improving R-D performance to varying degrees without additional parameter count or computational cost.