A Recipe for Efficient Sim-to-Real Transfer in Manipulation with Online Imitation-Pretrained World Models
cs.RO
Submitted: 2025-10-02
Updated: 2026-09-18
License: http://creativecommons.org/licenses/by/4.0/
The gist: We are interested in solving the problem of imitation learning with a limited amount of real-world expert data.
Terminology
Abstract
We are interested in solving the problem of imitation learning with a limited amount of real-world expert data. Existing offline imitation methods often struggle with poor data coverage and severe performance degradation. We propose a solution that leverages robot simulators to achieve online imitation learning. Our sim-to-real framework is based on world models and combines online imitation pretraining with offline finetuning. By leveraging online interactions, our approach alleviates the data coverage limitations of offline methods, leading to improved robustness and reduced performance degradation during finetuning. It also enhances generalization during domain transfer. Our empirical results demonstrate its effectiveness, improving success rates by at least 31.7% in sim-to-sim transfer and 23.3% in sim-to-real transfer over existing offline imitation learning baselines.
Sources
- Finetuning Offline World Models in the Real World
- Watch Less, Feel More: Sim-to-Real RL for Generalizable Articulated Object Manipulation via Motion Adaptation and Impedance Control
- Sim-to-Real Reinforcement Learning for Vision-Based Dexterous Manipulation on Humanoids
- NavDP: Learning Sim-to-Real Navigation Diffusion Policy with Privileged Information Guidance
- Sim-and-Real Co-Training: A Simple Recipe for Vision-Based Robotic Manipulation
- Proximal Policy Optimization Algorithms
- Coupled Distributional Random Expert Distillation for World Model Online Imitation Learning
- Reconciling Reality through Simulation: A Real-to-Sim-to-Real Approach for Robust Manipulation
- A Real-to-Sim-to-Real Approach to Robotic Manipulation with VLM-Generated Iterative Keypoint Rewards
- LodeStar: Long-horizon Dexterity via Synthetic Data Augmentation from Human Demonstrations
- Dream to Control: Learning Behaviors by Latent Imagination
- Temporal Difference Learning for Model Predictive Control
- Mastering Diverse Domains through World Models
- Exploration by Random Network Distillation
- SAM 2: Segment Anything in Images and Videos
- Efficient Online Reinforcement Learning Fine-Tuning Need Not Retain Offline Data
- Online Pre-Training for Offline-to-Online Reinforcement Learning
Related papers
- FMT x: An Efficient and Asymptotically Optimal Extension of the Fast Marching Tree for Dynamic Replanning
- MPCFormer: A physics-informed data-driven approach for explainable socially-aware autonomous driving
- RoboLab: A High-Fidelity Simulation Benchmark for Analysis of Task Generalist Policies
- HRDexDB: A 4D Dexterous Grasping Dataset Across Human and Multiple Robot Embodiments
- APT: Action Expert Pretraining Improves Instruction Generalization of Vision-Language-Action Policies
- Fine-tuning is Not Enough: A Parallel Framework for Collaborative Imitation and Reinforcement Learning in End-to-end Autonomous Driving