When Does On-Policy Interaction Help? Representational Tradeoffs in Value-Based Imitation Learning
Luca Viano, Antoine Moulin, Audrey Huang, Volkan Cevher, Philip Amortila, Dylan J. Foster
cs.LG, cs.AI, stat.ML
Submitted: 2026-07-31
Code: https://github.com/lviano/ovi
License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
Terminology
Sources
- On-Policy Distillation of Language Models: Learning from Self-Generated Mistakes
- Mitigating Covariate Shift in Misspecified Regression with Applications to Reinforcement Learning
- Scalable Online Exploration via Coverability
- Regret Analysis of Stochastic and Nonstochastic Multi-armed Bandit Problems
- Understanding Behavior Cloning with Action Quantization
- Fast Policy Learning through Imitation and Reinforcement
- Training Verifiers to Solve Math Word Problems
- Is a Good Representation Sufficient for Sample Efficient Reinforcement Learning?
- Offline Reinforcement Learning: Fundamental Barriers for Value Function Approximation
- IQ-Learn: Inverse soft-Q Learning for Imitation
- Offline Reinforcement Learning in Large State Spaces: Algorithms and Guarantees
- A Theory of Learning with Autoregressive Chain of Thought
- MobILE: Model-Based Imitation Learning From Observation Alone
- Revisiting DAgger in the Era of LLM-Agents
- Competition-Level Code Generation with AlphaCode
- Auto-Regressive Next-Token Predictors are Universal Learners
- Online Learning: A Modern Introduction Using Convex Optimization
- An Algorithmic Perspective on Imitation Learning
- Provably Breaking the Quadratic Error Compounding Barrier in Imitation Learning, Optimally
- SQIL: Imitation Learning via Reinforcement Learning with Sparse Rewards
Related papers
- Polynomial-Augmented Neural Networks (PANNs) with Weak Orthogonality Constraints for Enhanced Function and PDE Approximation
- AIRL-S: Unifying Reinforcement Learning and Search-Based Test-Time Scaling via Adversarial Inverse Reinforcement Learning
- Transformers as Bayesian In-Context Experimenters: Smoothness-Adaptive Efficient ATE Estimation
- Convergence issues in Relational Concept Analysis based on AOC-posets
- Beliefs Beyond Posteriors: Local-Consistency Optimisation for Bayesian Neural Networks
- Understanding Diffusion Models via Ratio-Based Function Approximation with SignReLU Networks