A Survey on End-to-End Autonomous Driving Training from the Perspectives of Data, Strategy, and Platform

arXiv:2610.00926 · cs.RO, cs.CV, cs.LG · Submitted 2026-10-01 · Read on arXiv

cs.RO, cs.CV, cs.LG

Submitted: 2026-10-01

Updated: 2026-10-01

Code: https://github.com/OpenDriveLab/OpenScene

Project page: https://www.xpeng.com/intelligent

Terminology

• World Model — 49× in this paper · explained in 7 episodes
• Reinforcement Learning — 38× in this paper · explained in 4 episodes
• Adaptation — 22× in this paper · explained in 1 episode
• Imitation Learning — 21× in this paper
• Fidelity — 15× in this paper · explained in 2 episodes
• Alignment — 13× in this paper · explained in 1 episode
• Robustness — 13× in this paper · explained in 5 episodes
• Generalization — 12× in this paper · explained in 13 episodes
• Interpretability — 12× in this paper · explained in 15 episodes
• Foundation Model — 11× in this paper · explained in 3 episodes
• Consistency — 10× in this paper · explained in 3 episodes
• Foundation Models — 10× in this paper · explained in 7 episodes
• Diffusion Model — 9× in this paper · explained in 8 episodes
• Benchmark — 8× in this paper · explained in 2 episodes
• Core — 6× in this paper · explained in 2 episodes
• Diffusion Models — 6× in this paper · explained in 6 episodes
• Fine-Tuning — 6× in this paper · explained in 5 episodes
• Transformer — 6× in this paper · explained in 2 episodes
• Adaptability — 5× in this paper · explained in 1 episode
• Distillation — 5× in this paper · explained in 2 episodes
• Generative Model — 5× in this paper · explained in 2 episodes
• Grounding — 5× in this paper · explained in 3 episodes
• Latent Space — 5× in this paper · explained in 7 episodes
• Verification — 5× in this paper · explained in 2 episodes
• Accuracy — 4× in this paper · explained in 1 episode
• Aggregation — 4× in this paper · explained in 1 episode
• Benchmarking — 4× in this paper · explained in 3 episodes
• Catastrophic Forgetting — 4× in this paper · explained in 5 episodes
• Covariate Shift — 4× in this paper · explained in 2 episodes
• Deep Reinforcement Learning — 4× in this paper · explained in 2 episodes
• Generative Models — 4× in this paper · explained in 2 episodes
• Hallucination — 4× in this paper · explained in 3 episodes
• Pipeline Parallelism — 4× in this paper
• Abstraction — 3× in this paper · explained in 1 episode
• Adam — 3× in this paper · explained in 2 episodes
• Calibration — 3× in this paper · explained in 7 episodes
• Decode — 3× in this paper · explained in 2 episodes
• Representation Learning — 3× in this paper · explained in 2 episodes
• Vision-language models — 3× in this paper · explained in 2 episodes
• Agent Interactions — 2× in this paper · explained in 1 episode
• Continual Learning — 2× in this paper · explained in 2 episodes
• Data Parallelism — 2× in this paper
• Digital Twin — 2× in this paper · explained in 5 episodes
• Distribution Shift — 2× in this paper · explained in 6 episodes
• Domain Adaptation — 2× in this paper · explained in 5 episodes
• Expert Parallelism — 2× in this paper
• Federated Learning — 2× in this paper · explained in 15 episodes
• Generative AI — 2× in this paper · explained in 2 episodes
• Multimodal Dataset — 2× in this paper · explained in 2 episodes
• Redundancy — 2× in this paper · explained in 2 episodes
• Sample Efficiency — 2× in this paper · explained in 4 episodes
• Sequence Parallelism — 2× in this paper
• Sim-to-Real Gap — 2× in this paper · explained in 6 episodes
• Temporal consistency — 2× in this paper · explained in 2 episodes
• Tensor Parallelism — 2× in this paper
• Unified Framework — 2× in this paper · explained in 5 episodes
• 3D Gaussian Splatting — 1× in this paper · explained in 2 episodes
• Adaptive Reasoning — 1× in this paper · explained in 2 episodes
• Closed-Loop Optimization — 1× in this paper · explained in 2 episodes
• Conformal Prediction — 1× in this paper · explained in 5 episodes
• Contrastive Learning — 1× in this paper · explained in 5 episodes
• Decoupling — 1× in this paper · explained in 2 episodes
• Dynamic Adaptation — 1× in this paper · explained in 2 episodes
• Edge Devices — 1× in this paper · explained in 2 episodes
• Embodied AI — 1× in this paper · explained in 2 episodes
• Epistemic Uncertainty — 1× in this paper · explained in 10 episodes
• Gaussian Splatting — 1× in this paper · explained in 4 episodes
• Generalizability — 1× in this paper · explained in 6 episodes
• Generative Modeling — 1× in this paper · explained in 3 episodes
• IEEE Intelligent Vehicles Symposium — 1× in this paper
• IEEE International Conference on Robotics and Automation — 1× in this paper
• Imitation Learning (IL) — 1× in this paper · explained in 2 episodes
• Instruction Tuning — 1× in this paper · explained in 2 episodes
• Knowledge Distillation — 1× in this paper · explained in 11 episodes
• Latent Diffusion Model — 1× in this paper · explained in 2 episodes
• Latent Dynamics — 1× in this paper · explained in 2 episodes
• Latent Representation — 1× in this paper · explained in 2 episodes
• Latent Representations — 1× in this paper · explained in 3 episodes
• Lyapunov function — 1× in this paper · explained in 2 episodes
• Mixture of Experts — 1× in this paper · explained in 2 episodes
• Mode Collapse — 1× in this paper · explained in 2 episodes
• Modular Architecture — 1× in this paper · explained in 4 episodes
• Modularity — 1× in this paper · explained in 9 episodes
• Multimodal Inputs — 1× in this paper · explained in 2 episodes
• Predictive Modeling — 1× in this paper · explained in 2 episodes
• Pseudo-labeling — 1× in this paper · explained in 3 episodes
• Quantization — 1× in this paper · explained in 12 episodes
• Regret — 1× in this paper · explained in 3 episodes
• Regularization — 1× in this paper · explained in 3 episodes
• Reinforcement Learning (RL) — 1× in this paper · explained in 25 episodes
• Self-Supervised Learning — 1× in this paper · explained in 8 episodes
• Steering — 1× in this paper · explained in 2 episodes
• Trace — 1× in this paper · explained in 2 episodes
• Transfer Learning — 1× in this paper · explained in 10 episodes
• Transferability — 1× in this paper · explained in 4 episodes
• Transformer Models — 1× in this paper · explained in 2 episodes

Sources

• Generative AI for Autonomous Driving: Frontiers and Opportunities
• End to End Learning for Self-Driving Cars
• A Survey for Foundation Models in Autonomous Driving
• OpenDriveVLA: Towards End-to-end Autonomous Driving with Large Vision Language Action Model
• GAIA-1: A Generative World Model for Autonomous Driving
• GAIA-2: A Controllable Multi-View Generative World Model for Autonomous Driving
• Tokenize the World into Object-level Knowledge to Address Long-tail Events in Autonomous Driving
• Towards Human-Centric Autonomous Driving: A Fast-Slow Architecture Integrating Large Language Model Guidance with Reinforcement Learning
• NuPlan: A closed-loop ML-based planning benchmark for autonomous vehicles
• Argoverse 2: Next Generation Datasets for Self-Driving Perception and Forecasting
• One Million Scenes for Autonomous Driving: ONCE Dataset
• TrafficGamer: Reliable and Flexible Traffic Simulation for Safety-Critical Scenarios with Game-Theoretic Oracles
• Interactive Adversarial Testing of Autonomous Vehicles with Adjustable Confrontation Intensity
• Learning-based 3D Reconstruction in Autonomous Driving: A Comprehensive Survey
• Cosmos-Drive-Dreams: Scalable Synthetic Driving Data Generation with World Foundation Models
• DDD17: End-To-End DAVIS Driving Dataset
• A2D2: Audi Autonomous Driving Dataset
• DriveMLM: Aligning Multi-Modal Large Language Models with Behavioral Planning States for Autonomous Driving
• DriveCoT: Integrating Chain-of-Thought Reasoning with End-to-End Driving
• StyleDrive: Towards Driving-Style Aware Benchmarking of End-To-End Autonomous Driving

Related papers

• FMT x: An Efficient and Asymptotically Optimal Extension of the Fast Marching Tree for Dynamic Replanning
• MPCFormer: A physics-informed data-driven approach for explainable socially-aware autonomous driving
• RoboLab: A High-Fidelity Simulation Benchmark for Analysis of Task Generalist Policies
• HRDexDB: A 4D Dexterous Grasping Dataset Across Human and Multiple Robot Embodiments
• APT: Action Expert Pretraining Improves Instruction Generalization of Vision-Language-Action Policies
• Fine-tuning is Not Enough: A Parallel Framework for Collaborative Imitation and Reinforcement Learning in End-to-end Autonomous Driving