Derivative-Free Structured Updates for Muon

arXiv:2609.17759 · math.OC, cs.LG · Submitted 2026-09-15 · Read on arXiv

math.OC, cs.LG

Submitted: 2026-09-15

Updated: 2026-09-15

Comments: 23 pages

License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/

Terminology

• Ablation — 3× in this paper · explained in 1 episode
• Accuracy — 3× in this paper · explained in 1 episode
• Query Complexity — 2× in this paper · explained in 2 episodes
• Sparsity — 2× in this paper · explained in 4 episodes
• Activation Function — 1× in this paper · explained in 1 episode
• Convergence Guarantees — 1× in this paper · explained in 3 episodes
• Matrix Completion — 1× in this paper · explained in 4 episodes

Sources

• Modular Duality in Deep Learning
• Muon Optimizes Under Spectral Norm Constraints
• The Newton-Muon Optimizer
• A Note on the Convergence of Muon
• Muon is Scalable for LLM Training
• On the Convergence Analysis of Muon
• Isotropic Curvature Model for Understanding Deep Learning Optimization: Is Gradient Orthogonalization Optimal?

Related papers

• Lions and Muons: Optimization via Stochastic Frank-Wolfe under Heavy-Tailed Noise
• Adam-HNAG: A Convergent Reformulation of Adam with Accelerated Rate
• Incremental Learning in Mirror Flows
• Online Control via Counterfactual Tracking
• Asynchronous Replanning in Two Population Linear Quadratic Mean Field Games: Information Requirements and Stability
• Petrov-Galerkin operator inference with application to stability-encouraging identification