Last-Iterate Guarantees for Online Reinforcement Learning in Structured Constrained MDPs

arXiv:2609.32933 · cs.LG · Submitted 2026-09-26 · Read on arXiv

cs.LG

Submitted: 2026-09-26

Updated: 2026-09-26

Terminology

• Actor — 92× in this paper · explained in 3 episodes
• Accuracy — 24× in this paper · explained in 1 episode
• Reinforcement Learning — 19× in this paper · explained in 4 episodes
• Regret — 13× in this paper · explained in 3 episodes
• Consistency — 10× in this paper · explained in 3 episodes
• Sample Complexity — 7× in this paper · explained in 15 episodes
• Constrained Reinforcement Learning — 5× in this paper
• Action Space — 2× in this paper · explained in 1 episode
• Online Reinforcement Learning — 2× in this paper · explained in 2 episodes
• Actor-Critic Model — 1× in this paper · explained in 1 episode
• Adam — 1× in this paper · explained in 2 episodes
• Benchmark — 1× in this paper · explained in 2 episodes
• Calibration — 1× in this paper · explained in 7 episodes
• Degeneracy — 1× in this paper · explained in 3 episodes
• Generative AI — 1× in this paper · explained in 2 episodes
• Generative Model — 1× in this paper · explained in 2 episodes
• Regularization — 1× in this paper · explained in 3 episodes
• Regularsied Policy-Gradient Primal Dual — 1× in this paper
• Reinforcement Learning (RL) — 1× in this paper · explained in 25 episodes
• Reward Function — 1× in this paper · explained in 2 episodes
• Sample Efficiency — 1× in this paper · explained in 4 episodes
• Theoretical Guarantees — 1× in this paper · explained in 2 episodes
• Trace — 1× in this paper · explained in 2 episodes

Sources

• A Policy Gradient Primal-Dual Algorithm for Constrained MDPs with Uniform PAC Guarantees
• Policy Optimization for Constrained MDPs with Provable Fast Global Convergence
• Near-Constant Strong Violation and Last-Iterate Convergence for Online CMDPs via Decaying Safety Margins

Related papers

• Polynomial-Augmented Neural Networks (PANNs) with Weak Orthogonality Constraints for Enhanced Function and PDE Approximation
• AIRL-S: Unifying Reinforcement Learning and Search-Based Test-Time Scaling via Adversarial Inverse Reinforcement Learning
• Transformers as Bayesian In-Context Experimenters: Smoothness-Adaptive Efficient ATE Estimation
• Convergence issues in Relational Concept Analysis based on AOC-posets
• Beliefs Beyond Posteriors: Local-Consistency Optimisation for Bayesian Neural Networks
• Understanding Diffusion Models via Ratio-Based Function Approximation with SignReLU Networks