Optimal Rates of Convergence for Entropy Regularization in Discounted Markov Decision Processes

arXiv:2406.04163 · math.OC, cs.LG, cs.SY, eess.SY · Submitted 2024-06-06 · Read on arXiv

math.OC, cs.LG, cs.SY, eess.SY

Submitted: 2024-06-06

Updated: 2026-09-10

Comments: 32 pages, 1 figure

Journal ref: Information and Inference: A Journal of the IMA, Volume 15, Issue 1, March 2026, iaaf034

DOI: 10.1093/imaiai/iaaf034

License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/

Terminology

Regularization — 108× in this paper · explained in 3 episodes
Gradient Flow — 83× in this paper · explained in 2 episodes
Reinforcement Learning — 17× in this paper · explained in 4 episodes
Algorithmic Bias — 5× in this paper · explained in 1 episode
Action Space — 4× in this paper · explained in 1 episode
Actor — 3× in this paper · explained in 3 episodes
Generalization — 2× in this paper · explained in 13 episodes
Reward Function — 2× in this paper · explained in 2 episodes
Sample Complexity — 2× in this paper · explained in 15 episodes
Accuracy — 1× in this paper · explained in 1 episode
Convergence Guarantees — 1× in this paper · explained in 3 episodes
Core — 1× in this paper · explained in 2 episodes
Deep Reinforcement Learning — 1× in this paper · explained in 2 episodes
Distribution Shift — 1× in this paper · explained in 5 episodes
Error Decomposition — 1× in this paper · explained in 3 episodes
KL Regularization — 1× in this paper · explained in 2 episodes
Matrix Completion — 1× in this paper · explained in 3 episodes
Sparsity — 1× in this paper · explained in 4 episodes

Sources

Related papers