Silver Rate Is (Almost) Optimal for Gradient Descent

arXiv:2609.09152 · math.OC, cs.LG · Submitted 2026-09-08 · Read on arXiv

math.OC, cs.LG

Submitted: 2026-09-08

Updated: 2026-09-10

Comments: 35 pages, 4 figure

License: http://creativecommons.org/licenses/by/4.0/

The gist: We study how far gradient descent (GD) can be accelerated by predetermined stepsizes in smooth convex optimization.

Abstract

We study how far gradient descent (GD) can be accelerated by predetermined stepsizes in smooth convex optimization. Writing p sil= 2(1+ sqrt 2), we prove an Ω (n-p sil-O(sqrt n/ n)) non-anytime lower bound. In the anytime setting, every infinite schedule has infinitely many horizons with error Ω (n-2p sil over 1+p sil-O(sqrt n/ n)). Together with the silver-schedule upper bound [Altschuler and Parrilo, 2025] and the anytime upper bound [Zhang et al., 2025], our results determine the optimal polynomial convergence exponents in both settings.

Related papers