WALT: Learning World-Model-Aligned Latent Trajectories for Autonomous Driving

arXiv:2609.30436 · cs.RO, cs.CV · Submitted 2026-09-24 · Read on arXiv

cs.RO, cs.CV

Submitted: 2026-09-24

Updated: 2026-09-24

Terminology

• World Model — 43× in this paper · explained in 7 episodes
• Alignment — 32× in this paper · explained in 1 episode
• Representation Learning — 10× in this paper · explained in 2 episodes
• Latent Space — 9× in this paper · explained in 7 episodes
• Benchmark — 5× in this paper · explained in 2 episodes
• Conference on Computer Vision and Pattern Recognition — 4× in this paper
• Diffusion Model — 4× in this paper · explained in 8 episodes
• Transformer — 4× in this paper · explained in 2 episodes
• Cosine Similarity — 3× in this paper · explained in 3 episodes
• Embeddings — 3× in this paper · explained in 2 episodes
• Latent Representation — 3× in this paper · explained in 2 episodes
• Distillation — 2× in this paper · explained in 2 episodes
• Fidelity — 2× in this paper · explained in 2 episodes
• Joint-Embedding Predictive Architectures — 2× in this paper
• Latent Diffusion Model — 2× in this paper · explained in 2 episodes
• Quantization — 2× in this paper · explained in 12 episodes
• Regularization — 2× in this paper · explained in 3 episodes
• Reinforcement Learning — 2× in this paper · explained in 4 episodes
• Aggregation — 1× in this paper · explained in 1 episode
• Benchmarking — 1× in this paper · explained in 3 episodes
• Diffusion Models — 1× in this paper · explained in 6 episodes
• Fine-Tuning — 1× in this paper · explained in 5 episodes
• Flow Matching — 1× in this paper · explained in 7 episodes
• Gaussian Splatting — 1× in this paper · explained in 4 episodes
• Generative Modeling — 1× in this paper · explained in 3 episodes
• Mixture of Experts — 1× in this paper · explained in 2 episodes
• Normalizing Flow — 1× in this paper · explained in 3 episodes
• Normalizing Flows — 1× in this paper · explained in 4 episodes
• Rectified Flow — 1× in this paper · explained in 3 episodes
• Self-Supervised Learning — 1× in this paper · explained in 7 episodes
• Semantic Alignment — 1× in this paper · explained in 3 episodes
• WorldModel Alignment for Latent Trajectories — 1× in this paper

Sources

• DriveVLA-W0: World Models Amplify Data Scaling Law in Autonomous Driving
• MGVQ: Could VQ-VAE Beat VAE? A Generalizable Tokenizer with Multi-group Quantization
• Think Locally, Refine Globally for Memory-Efficient 3D Reconstruction
• Bridging Scene Generation and Planning: Driving with World Model via Unifying Vision and Motion Representation
• LeWorldModel: Stable End-to-End Joint-Embedding Predictive Architecture from Pixels
• Representation Alignment for Generation: Training Diffusion Transformers Is Easier Than You Think
• The Role of World Models in Shaping Autonomous Driving: A Comprehensive Survey
• 3D and 4D World Modeling: A Survey
• Enhancing End-to-End Autonomous Driving with Latent World Model
• ImagiDrive: A Unified Imagination-and-Planning Framework for Autonomous Driving
• WorldRFT: Latent World Model Planning with Reinforcement Fine-Tuning for Autonomous Driving
• EponaV2: Driving World Model with Comprehensive Future Reasoning
• Revisiting Feature Prediction for Learning Visual Representations from Video
• V-JEPA 2: Self-Supervised Video Models Enable Understanding, Prediction and Planning
• AnyTalker: Scaling Multi-Person Talking Video Generation with Interactivity Refinement
• DINO-Tok: Adapting DINO for Visual Tokenizers
• Drive-JEPA: Video JEPA Meets Multimodal Trajectory Distillation for End-to-End Driving
• Auto-JEPA: A Latent World Model of Continuous Intent for End-to-End Autonomous Driving
• Autoregressive End-to-End Planning with Time-Invariant Spatial Alignment and Multi-Objective Policy Refinement
• AdaThinkDrive: Adaptive Thinking via Reinforcement Learning for Autonomous Driving

Related papers

• FMT x: An Efficient and Asymptotically Optimal Extension of the Fast Marching Tree for Dynamic Replanning
• MPCFormer: A physics-informed data-driven approach for explainable socially-aware autonomous driving
• RoboLab: A High-Fidelity Simulation Benchmark for Analysis of Task Generalist Policies
• HRDexDB: A 4D Dexterous Grasping Dataset Across Human and Multiple Robot Embodiments
• APT: Action Expert Pretraining Improves Instruction Generalization of Vision-Language-Action Policies
• Fine-tuning is Not Enough: A Parallel Framework for Collaborative Imitation and Reinforcement Learning in End-to-end Autonomous Driving