Tsallis Entropy Regularization for Linear Quadratic Regulator and Kullback-Leibler Control
math.OC, cs.LG, cs.SY, eess.SY
Submitted: 2024-03-04
Updated: 2026-09-20
Comments: 7 figures
License: http://creativecommons.org/licenses/by/4.0/
The gist: Shannon entropy regularization is widely adopted in optimal control due to its ability to promote exploration and enhance robustness, e.g., maximum entropy reinforcement learning known as Soft
Terminology
Abstract
Shannon entropy regularization is widely adopted in optimal control due to its ability to promote exploration and enhance robustness, e.g., maximum entropy reinforcement learning known as Soft Actor-Critic. The aim of this paper is to show that formulations based on Tsallis entropy, which is a one-parameter extension of Shannon entropy, retain many of the structural and computational advantages of Shannon-entropy-based approaches while offering additional benefits. In particular, we derive a closed-form solution for the linear quadratic regulator and an efficient computational method for the Kullback-Leibler control problem. We also demonstrate its usefulness in balancing between exploration and sparsity of the obtained control law.
Sources
- Imitation-regularized Optimal Transport on Networks: Provable Robustness and Application to Logistics Planning
- Central limit theorem, deformed exponentials and superstatistics
- A unified view of entropy-regularized Markov decision processes
- Sparse Sequence-to-Sequence Models
- Maximum Entropy Density Control of Discrete-Time Linear Systems with Quadratic Cost
Related papers
- Lions and Muons: Optimization via Stochastic Frank-Wolfe under Heavy-Tailed Noise
- Adam-HNAG: A Convergent Reformulation of Adam with Accelerated Rate
- Incremental Learning in Mirror Flows
- Online Control via Counterfactual Tracking
- Asynchronous Replanning in Two Population Linear Quadratic Mean Field Games: Information Requirements and Stability
- Petrov-Galerkin operator inference with application to stability-encouraging identification