Fitted Q-Evaluation without Bellman Completeness via Occupancy Weighting

arXiv:2512.23805 · stat.ML, cs.LG · Submitted 2025-12-29 · Read on arXiv

stat.ML, cs.LG

Submitted: 2025-12-29

Updated: 2026-08-30

Terminology

• Reinforcement Learning — 26× in this paper · explained in 4 episodes
• Regularization — 13× in this paper · explained in 3 episodes
• Benchmark — 12× in this paper · explained in 2 episodes
• Q-learning — 7× in this paper · explained in 3 episodes
• Fitted Q-evaluation — 6× in this paper
• Adam — 4× in this paper · explained in 2 episodes
• Causal Inference — 4× in this paper · explained in 4 episodes
• Regret — 4× in this paper · explained in 3 episodes
• Deep Reinforcement Learning — 3× in this paper · explained in 2 episodes
• Distribution Shift — 3× in this paper · explained in 5 episodes
• Accuracy — 2× in this paper · explained in 1 episode
• Action Space — 2× in this paper · explained in 1 episode
• Online Reinforcement Learning — 2× in this paper · explained in 2 episodes
• Aggregation — 1× in this paper · explained in 1 episode
• Approximation Bound — 1× in this paper · explained in 1 episode
• Convergence Guarantees — 1× in this paper · explained in 3 episodes
• Minimax Optimal Rates — 1× in this paper · explained in 2 episodes
• Robustness — 1× in this paper · explained in 5 episodes
• Theoretical Guarantees — 1× in this paper · explained in 2 episodes

Sources

• Harnessing Density Ratios for Online Reinforcement Learning
• A Unifying View of Coverage in Linear Off-Policy Evaluation
• Sequential causal inference in a single world of connected units
• Pessimistic Nonlinear Least-Squares Value Iteration for Offline Reinforcement Learning
• Offline Reinforcement Learning: Fundamental Barriers for Value Function Approximation
• Learning density ratios in causal inference using Bregman-Riesz regression
• COptiDICE: Offline Constrained Reinforcement Learning via Stationary Distribution Correction Estimation
• Understanding the theoretical properties of projected Bellman equation, linear Q-learning, and approximate value iteration
• Emphatic Temporal-Difference Learning
• Off-Policy Evaluation in Markov Decision Processes under Weak Distributional Overlap
• AlgaeDICE: Policy Gradient from Arbitrary Experience
• Source-Condition Analysis of Kernel Adversarial Estimators
• Finite Sample Analysis of Minimax Offline Reinforcement Learning: Completeness, Fast Rates and First-Order Efficiency
• A Researcher's Guide to Empirical Risk Minimization
• Semiparametric Double Reinforcement Learning with Applications to Long-Term Causal Inference
• Nonparametric Instrumental Variable Inference with Many Weak Instruments
• Empirical Study of Off-Policy Policy Evaluation for Reinforcement Learning
• The Role of Coverage in Online Reinforcement Learning
• GenDICE: Generalized Offline Estimation of Stationary Values

Related papers

• Behavior of prediction performance metrics with rare events
• Optimal Estimation of Generic Dynamics by Path-Dependent Neural Jump ODEs
• A Posterior-Dynamics Framework for Imaging Inverse Problems with Pretrained Diffusion Priors
• One Permutation Is All You Need: Fast, Deterministic Feature Importance and Model Stress-Testing
• Online Conformal Prediction for Non-Exchangeable Panel Data
• Deep Time-Series Forecasting in 10 Years: A Survey