Value Mirror Descent for Reinforcement Learning

arXiv:2604.06039 · math.OC, cs.LG, math.PR · Submitted 2026-04-07 · Read on arXiv

math.OC, cs.LG, math.PR

Submitted: 2026-04-07

Updated: 2026-09-23

Terminology

• Sample Complexity — 25× in this paper · explained in 15 episodes
• Value Mirror Descent — 25× in this paper
• Reinforcement Learning — 15× in this paper · explained in 4 episodes
• Generative Model — 10× in this paper · explained in 2 episodes
• Stochastic Value Mirror Descent — 10× in this paper
• Action Space — 6× in this paper · explained in 1 episode
• Convergence Guarantees — 4× in this paper · explained in 3 episodes
• Markov Decision Process — 3× in this paper
• Reinforcement Learning (RL) — 3× in this paper · explained in 25 episodes
• Accuracy — 2× in this paper · explained in 1 episode
• Adaptation — 2× in this paper · explained in 1 episode
• Fine-Tuning — 2× in this paper · explained in 5 episodes
• Online Adaptation — 2× in this paper · explained in 3 episodes
• Q-learning — 2× in this paper · explained in 3 episodes
• Continual Learning — 1× in this paper · explained in 2 episodes
• Generalization — 1× in this paper · explained in 13 episodes
• Generative Models — 1× in this paper · explained in 2 episodes
• Hilbert Space — 1× in this paper · explained in 2 episodes
• Kullback-Leibler (KL) Divergence — 1× in this paper · explained in 3 episodes
• Markov Decision Process (MDP) — 1× in this paper · explained in 11 episodes
• Online Reinforcement Learning — 1× in this paper · explained in 2 episodes

Sources

• Stochastic Primal-Dual Methods and Sample Complexity of Reinforcement Learning
• Policy Optimization over General State and Action Spaces
• Strongly-polynomial time and validation analysis of policy gradient methods
• Auto-exploration for online reinforcement learning
• First-order Policy Optimization for Robust Markov Decision Process
• Proximal Policy Optimization Algorithms
• Primal-Dual $\pi$ Learning: Sample Complexity and Sublinear Run Time for Ergodic Markov Decision Problems

Related papers

• Lions and Muons: Optimization via Stochastic Frank-Wolfe under Heavy-Tailed Noise
• Adam-HNAG: A Convergent Reformulation of Adam with Accelerated Rate
• Incremental Learning in Mirror Flows
• Online Control via Counterfactual Tracking
• Asynchronous Replanning in Two Population Linear Quadratic Mean Field Games: Information Requirements and Stability
• Petrov-Galerkin operator inference with application to stability-encouraging identification