DreamStream: Towards Policy-Oriented Generative Simulation for End-to-End Driving

arXiv:2609.26792 · cs.RO, cs.CV · Submitted 2026-09-22 · Read on arXiv

cs.RO, cs.CV

Submitted: 2026-09-22

Updated: 2026-09-22

Code: https://github.com/VAIL-UCLA/DreamStream

Terminology

• Benchmark — 28× in this paper · explained in 2 episodes
• Distillation — 25× in this paper · explained in 2 episodes
• Alignment — 20× in this paper · explained in 1 episode
• World Model — 14× in this paper · explained in 7 episodes
• Driving Score — 9× in this paper
• Fidelity — 7× in this paper · explained in 2 episodes
• Benchmarking — 6× in this paper · explained in 3 episodes
• Failure Modes — 6× in this paper · explained in 3 episodes
• Robustness — 6× in this paper · explained in 5 episodes
• Ablation — 5× in this paper · explained in 1 episode
• Diffusion Model — 5× in this paper · explained in 8 episodes
• Reinforcement Learning — 5× in this paper · explained in 4 episodes
• KV cache — 4× in this paper · explained in 3 episodes
• Sim-to-Real Gap — 3× in this paper · explained in 6 episodes
• Accuracy — 2× in this paper · explained in 1 episode
• Consistency — 2× in this paper · explained in 3 episodes
• Generative Model — 2× in this paper · explained in 2 episodes
• Generative Models — 2× in this paper · explained in 2 episodes
• Grounding — 2× in this paper · explained in 3 episodes
• In Conference on Robot Learning — 2× in this paper
• Lane Keeping — 2× in this paper
• Multi-Agent System — 2× in this paper · explained in 6 episodes
• Transformer — 2× in this paper · explained in 2 episodes
• Adaptive Reasoning — 1× in this paper · explained in 2 episodes
• Adversarial scenarios — 1× in this paper · explained in 1 episode
• Alignment Dimensions — 1× in this paper · explained in 1 episode
• Calibration — 1× in this paper · explained in 7 episodes
• Decoupling — 1× in this paper · explained in 2 episodes
• Drivable Area Compliance — 1× in this paper
• Driving Direction Compliance — 1× in this paper
• Ego Progress — 1× in this paper
• Extended Comfort — 1× in this paper
• Extended Predictive Driver Model Score — 1× in this paper
• Fine-Tuning — 1× in this paper · explained in 5 episodes
• Foundation Model — 1× in this paper · explained in 3 episodes
• Generalization — 1× in this paper · explained in 13 episodes
• History Comfort — 1× in this paper
• Hybrid Approach — 1× in this paper · explained in 4 episodes
• Multimodal Dataset — 1× in this paper · explained in 2 episodes
• Nash Equilibrium — 1× in this paper · explained in 2 episodes
• No At-Fault Collisions — 1× in this paper
• Temporal consistency — 1× in this paper · explained in 2 episodes
• Traffic Light Compliance — 1× in this paper

Sources

• Do Open-Loop Metrics Predict Closed-Loop Driving? A Cross-Benchmark Correlation Study of NAVSIM and Bench2Drive
• BridgeSim: Unveiling the OL-CL Gap in End-to-End Autonomous Driving
• GPUDrive: Data-driven, multi-agent driving simulation at 1 million FPS
• Fail2Drive: Benchmarking Closed-Loop Driving Generalization
• DreamForge: Motion-Aware Autoregressive Video Generation for Multi-View Driving Scenes
• WorldLens: Full-Spectrum Evaluations of Driving World Models in Real World
• Dreamland: Controllable World Creation with Simulator and Generative Models
• Representation Fr'echet Loss for Visual Generation
• GAIA-1: A Generative World Model for Autonomous Driving
• MagicDrive-V2: High-Resolution Long Video Generation for Autonomous Driving with Adaptive Control
• InfiniCube: Unbounded and Controllable Dynamic 3D Driving Scene Generation with World-Guided Video Models
• Wan: Open and Advanced Large-Scale Video Generative Models
• NuPlan: A closed-loop ML-based planning benchmark for autonomous vehicles
• QuantV2X: A Fully Quantized Multi-Agent System for Cooperative Perception
• MDrive: Benchmarking Closed-Loop Cooperative Driving for End-to-End Multi-agent Systems
• RAP: 3D Rasterization Augmented End-to-End Planning
• Adv-BMT: Bidirectional Motion Transformer for Safety-Critical Traffic Scenario Generation
• MotionStream: Real-Time Video Generation with Interactive Motion Controls
• DreamDojo: A Generalist Robot World Model from Large-Scale Human Videos
• Classifier-Free Diffusion Guidance

Related papers

• FMT x: An Efficient and Asymptotically Optimal Extension of the Fast Marching Tree for Dynamic Replanning
• MPCFormer: A physics-informed data-driven approach for explainable socially-aware autonomous driving
• RoboLab: A High-Fidelity Simulation Benchmark for Analysis of Task Generalist Policies
• HRDexDB: A 4D Dexterous Grasping Dataset Across Human and Multiple Robot Embodiments
• APT: Action Expert Pretraining Improves Instruction Generalization of Vision-Language-Action Policies
• Fine-tuning is Not Enough: A Parallel Framework for Collaborative Imitation and Reinforcement Learning in End-to-end Autonomous Driving