DriveHierarchy: A Benchmark for Diagnosing VLM Driving Capabilities from Open-Loop Understanding to Closed-Loop Execution

arXiv:2609.31814 · cs.AI · Submitted 2026-09-25 · Read on arXiv

cs.AI

Submitted: 2026-09-25

Updated: 2026-09-25

Code: https://github.com/PerfectXu88/DriveHierarchy

Terminology

• Benchmark — 79× in this paper · explained in 2 episodes
• Grounding — 18× in this paper · explained in 3 episodes
• Benchmarking — 9× in this paper · explained in 3 episodes
• Fine-Tuning — 7× in this paper · explained in 5 episodes
• Calibration — 5× in this paper · explained in 7 episodes
• Steering — 5× in this paper · explained in 2 episodes
• Foundation Model — 4× in this paper · explained in 3 episodes
• Foundation Models — 4× in this paper · explained in 7 episodes
• Aggregation — 3× in this paper · explained in 1 episode
• Failure Modes — 3× in this paper · explained in 3 episodes
• Interpretability — 3× in this paper · explained in 15 episodes
• Redundancy — 3× in this paper · explained in 2 episodes
• Adaptation — 2× in this paper · explained in 1 episode
• Alignment — 2× in this paper · explained in 1 episode
• Consistency — 2× in this paper · explained in 3 episodes
• Fidelity — 2× in this paper · explained in 2 episodes
• Robustness — 2× in this paper · explained in 5 episodes
• Temporal Reasoning — 2× in this paper · explained in 2 episodes
• Active Exploration — 1× in this paper · explained in 2 episodes
• Actor — 1× in this paper · explained in 3 episodes
• Adaptive Reasoning — 1× in this paper · explained in 2 episodes
• Agent Interactions — 1× in this paper · explained in 1 episode
• Contextual Understanding — 1× in this paper · explained in 2 episodes
• Deep Reinforcement Learning — 1× in this paper · explained in 2 episodes
• Diffusion Model — 1× in this paper · explained in 8 episodes
• Distillation — 1× in this paper · explained in 2 episodes
• International Conference on Robotics and Automation — 1× in this paper
• LoRA Adapters — 1× in this paper · explained in 3 episodes
• Multimodal Dataset — 1× in this paper · explained in 2 episodes
• Provenance — 1× in this paper · explained in 4 episodes
• Pruning — 1× in this paper · explained in 4 episodes
• Reinforcement Learning — 1× in this paper · explained in 4 episodes
• Supervised Fine-Tuning (SFT) — 1× in this paper · explained in 8 episodes
• Trace — 1× in this paper · explained in 2 episodes
• Transformer — 1× in this paper · explained in 2 episodes
• Unified Framework — 1× in this paper · explained in 5 episodes
• Visual Grounding — 1× in this paper · explained in 2 episodes

Sources

• TeLL-Drive: Enhancing Autonomous Driving with Teacher LLM-Guided Deep Reinforcement Learning
• AutoVLA: A Vision-Language-Action Model for End-to-End Autonomous Driving with Adaptive Reasoning and Reinforcement Fine-Tuning
• AutoMoT: A Unified Vision-Language-Action Model with Asynchronous Mixture-of-Transformers for End-to-End Autonomous Driving
• DriveMoE: Mixture-of-Experts for Vision-Language-Action Model in End-to-End Autonomous Driving
• A Knowledge-Driven Diffusion Policy for End-to-End Autonomous Driving Based on Expert Routing
• DSBench: A Comprehensive Benchmark for Evaluating External and In-Cabin Risks
• STSBench: A Spatio-temporal Scenario Benchmark for Multi-modal Large Language Models in Autonomous Driving
• DriveE2E: Closed-Loop Benchmark for End-to-End Autonomous Driving through Real-to-Simulation
• EmbodiedBench: Comprehensive Benchmarking Multi-modal Large Language Models for Vision-Driven Embodied Agents
• ENACT: Evaluating Embodied Cognition with World Modeling of Egocentric Interaction
• InternVL3.5: Advancing Open-Source Multimodal Models in Versatility, Reasoning, and Efficiency
• Zooming without Zooming: Region-to-Image Distillation for Fine-Grained Multimodal Perception
• Pixtral 12B
• Mini-InternVL: A Flexible-Transfer Pocket Multimodal Model with 5% Parameters and 90% Performance
• VP-AutoTest: A Virtual-Physical Fusion Autonomous Driving Testing Platform
• DRAMA: An Efficient End-to-end Motion Planner for Autonomous Driving with Mamba
• Bench2Drive-VL: Benchmarks for Closed-Loop Autonomous Driving with Vision-Language Models

Related papers

• MAVEN-T: Reinforced Heterogeneous Distillation for Real-Time Multi-Agent Trajectory Prediction
• Model Discovery Agent: LLM-assisted Bayesian experiment design for data-efficient discovery of mechanistic world models
• The Clinician's Veto: Navigating Trust, Liability, and Uncertainty in Autonomous AI Prescribing
• MindHelper: Closed-Loop Embodied Mental-State Reasoning for Precision Intervention
• Incumbent Advantage: Brand Bias and Cognitive Manipulation Dynamics in LLM Recommendation Systems
• VSAL: A Vision Solver with Adaptive Layouts for Graph Property Detection