A regret minimization approach to fixed-point iterations

arXiv:2509.21653 · math.OC, cs.LG, cs.NA, math.NA · Submitted 2025-09-25 · Read on arXiv

math.OC, cs.LG, cs.NA, math.NA

Submitted: 2025-09-25

Updated: 2026-09-18

License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/

Terminology

• Regret — 79× in this paper · explained in 3 episodes
• Adam — 30× in this paper · explained in 2 episodes
• Adaptivity — 16× in this paper · explained in 1 episode
• Convergence Guarantees — 6× in this paper · explained in 3 episodes
• Objective Function — 4× in this paper · explained in 2 episodes
• Regularization — 4× in this paper · explained in 3 episodes
• Hilbert Space — 3× in this paper · explained in 2 episodes
• Adaptation — 2× in this paper · explained in 1 episode
• Core — 2× in this paper · explained in 2 episodes
• Robustness — 2× in this paper · explained in 5 episodes
• Theoretical Guarantees — 2× in this paper · explained in 2 episodes
• Trace — 2× in this paper · explained in 2 episodes
• Adaptive Gradient Methods — 1× in this paper · explained in 1 episode
• Calibration — 1× in this paper · explained in 7 episodes
• Generalization — 1× in this paper · explained in 13 episodes

Sources

• Provable Complexity Improvement of AdaGrad over SGD: Upper and Lower Bounds in Stochastic Non-Convex Optimization
• AdaGrad under Anisotropic Smoothness
• Online Learning: A Modern Introduction Using Convex Optimization

Related papers

• Lions and Muons: Optimization via Stochastic Frank-Wolfe under Heavy-Tailed Noise
• Adam-HNAG: A Convergent Reformulation of Adam with Accelerated Rate
• Incremental Learning in Mirror Flows
• Online Control via Counterfactual Tracking
• Asynchronous Replanning in Two Population Linear Quadratic Mean Field Games: Information Requirements and Stability
• Petrov-Galerkin operator inference with application to stability-encouraging identification