Bellman Calibration for Marginalized Importance Weighting in Offline Reinforcement Learning
cs.LG, stat.ML
Submitted: 2026-08-25
Updated: 2026-08-25
Comments: 43 pages, 1 figure, 4 tables
Code: https://github.com/google-research/google-research
License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
Terminology
Sources
- Improving the Finite Sample Estimation of Average Treatment Effects using Double/Debiased Machine Learning with Propensity Score Calibration
- Calibrated and Conformal Propensity Scores for Causal Effect Estimation
- D4RL: Datasets for Deep Data-Driven Reinforcement Learning
- Propensity score models are better when post-calibrated
- Reinforcement Learning in Low-Rank MDPs with Density Features
- SCOPE-RL: A Python Library for Offline Reinforcement Learning and Off-Policy Evaluation
- Calibration Strategies for Robust Causal Estimation: Theoretical and Empirical Insights on Propensity Score-Based Estimators
- Off-Policy Evaluation in Markov Decision Processes under Weak Distributional Overlap
- The Scope of Multicalibration: Characterizing Multicalibration via Property Elicitation
- Finite Sample Analysis of Minimax Offline Reinforcement Learning: Completeness, Fast Rates and First-Order Efficiency
- A Review of Off-Policy Evaluation in Reinforcement Learning
- Generalized Venn and Venn-Abers Calibration with Applications in Conformal Prediction
- Bellman Calibration for $V$-Learning in Offline Reinforcement Learning
- Soft Fitted Q-Iteration without Bellman Completeness: Occupancy Reweighting and Temperature Annealing
- Fitted Q-Evaluation without Bellman Completeness via Occupancy Weighting
- Fitted Occupancy-Ratio Evaluation without Bellman Completeness
- Doubly robust inference via calibration
- Efficient Inference for Inverse Reinforcement Learning and Dynamic Discrete Choice Models
- Semiparametric Double Reinforcement Learning with Applications to Long-Term Causal Inference
- Orthogonal Causal Calibration
Related papers
- Polynomial-Augmented Neural Networks (PANNs) with Weak Orthogonality Constraints for Enhanced Function and PDE Approximation
- AIRL-S: Unifying Reinforcement Learning and Search-Based Test-Time Scaling via Adversarial Inverse Reinforcement Learning
- Transformers as Bayesian In-Context Experimenters: Smoothness-Adaptive Efficient ATE Estimation
- Convergence issues in Relational Concept Analysis based on AOC-posets
- Beliefs Beyond Posteriors: Local-Consistency Optimisation for Bayesian Neural Networks
- Understanding Diffusion Models via Ratio-Based Function Approximation with SignReLU Networks