The logarithmic factor in the tuned constant-step rate of random reshuffling
For strongly convex finite sums, the known last-iterate upper bounds for random reshuffling with a constant step size chosen from the number of epochs carry a logarithmic factor, while the known lower bounds carry none. We show that for constant steps the factor cannot be removed. On a one-dimensional sum of $n_1$ quadratic and $n-n_1$ linear components, with average curvature $\mu$, gradient bound $G$ at the minimiser and initial distance $D$, every constant step $\gamma\le 1/L$ leaves $\mathbb{E} F(x_K)-F^*\ge G^2(\Lambda-\ln\Lambda)^2/(512\,\mu T^2)$ after $K$ epochs, where $T=nK$ and $\Lambda=\ln(\mu TD/G)$, while the step $\gamma=\Lambda/(\mu T)$ gives at most $G^2\Lambda^2(1+o(1))/(8\mu T^2)$ once $K$ is large compared with $n_1\Lambda$. The proof computes one epoch exactly: the stationary mean of the epoch-end iterate is $\gamma G\varphi/2$, where $\varphi$ is a weighted mean lying between $(1-\gamma L)^{n_1-1}$ and $1$, and the epoch variance reduces to the variance of a Mann–Whitney statistic. The bound concerns the worst starting point: from the other side of the minimiser a tuned step can cancel the transient against the bias, while a start of random sign keeps the bound for every step.
Authors
- Yitzchak Shmalo (ORCID: https://orcid.org/0000-0002-2886-3049)
Institutions
- Hebrew University of Jerusalem (IL)
Publication Details
- Journal
- Zenodo (CERN European Organization for Nuclear Research)
- Published
- 2026-11-08
- DOI
- https://doi.org/10.5281/zenodo.24069892
- Primary Topic
- Stochastic Gradient Optimization Techniques
- Type
- preprint