On Policy Stochasticity in Mutual Information Optimal Control of Linear Systems

arXiv:2507.21543 · math.OC, cs.LG, cs.SY, eess.SY · Submitted 2025-07-29 · Read on arXiv

math.OC, cs.LG, cs.SY, eess.SY

Submitted: 2025-07-29

Updated: 2026-09-19

Comments: 20 pages. Added some new theoretical results and revised potentially misleading phrasing from v2. The main arguments and discussions remain unchanged

License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/

Terminology

• Regularization — 24× in this paper · explained in 3 episodes
• Alternating Optimization Algorithm — 8× in this paper · explained in 1 episode
• Actor — 5× in this paper · explained in 3 episodes
• Reinforcement Learning — 4× in this paper · explained in 4 episodes
• Robustness — 2× in this paper · explained in 5 episodes
• Deep Reinforcement Learning — 1× in this paper · explained in 2 episodes
• Generalization — 1× in this paper · explained in 13 episodes
• Objective Function — 1× in this paper · explained in 2 episodes
• Optimal Transport — 1× in this paper · explained in 5 episodes
• Q-learning — 1× in this paper · explained in 3 episodes
• Reinforcement Learning (RL) — 1× in this paper · explained in 25 episodes
• Trace — 1× in this paper · explained in 2 episodes

Sources

• Soft Actor-Critic Algorithms and Applications
• Reinforcement Learning and Control as Probabilistic Inference: Tutorial and Review
• Deep RL With Information Constrained Policies: Generalization in Continuous Control
• Meta-SAC: Auto-tune the Entropy Temperature of Soft Actor-Critic via Metagradient
• Maximum Entropy RL (Provably) Solves Some Robust RL Problems

Related papers

• Lions and Muons: Optimization via Stochastic Frank-Wolfe under Heavy-Tailed Noise
• Adam-HNAG: A Convergent Reformulation of Adam with Accelerated Rate
• Incremental Learning in Mirror Flows
• Online Control via Counterfactual Tracking
• Asynchronous Replanning in Two Population Linear Quadratic Mean Field Games: Information Requirements and Stability
• Petrov-Galerkin operator inference with application to stability-encouraging identification