Silver Rate Is (Almost) Optimal for Gradient Descent
math.OC, cs.LG
Submitted: 2026-09-08
Updated: 2026-09-10
Comments: 35 pages, 4 figure
License: http://creativecommons.org/licenses/by/4.0/
The gist: We study how far gradient descent (GD) can be accelerated by predetermined stepsizes in smooth convex optimization.
Abstract
We study how far gradient descent (GD) can be accelerated by predetermined stepsizes in smooth convex optimization. Writing p sil= 2(1+ sqrt 2), we prove an Ω (n-p sil-O(sqrt n/ n)) non-anytime lower bound. In the anytime setting, every infinite schedule has infinitely many horizons with error Ω (n-2p sil over 1+p sil-O(sqrt n/ n)). Together with the silver-schedule upper bound [Altschuler and Parrilo, 2025] and the anytime upper bound [Zhang et al., 2025], our results determine the optimal polynomial convergence exponents in both settings.
Related papers
- Lions and Muons: Optimization via Stochastic Frank-Wolfe under Heavy-Tailed Noise
- Adam-HNAG: A Convergent Reformulation of Adam with Accelerated Rate
- Incremental Learning in Mirror Flows
- Online Control via Counterfactual Tracking
- Asynchronous Replanning in Two Population Linear Quadratic Mean Field Games: Information Requirements and Stability
- Petrov-Galerkin operator inference with application to stability-encouraging identification