Do Vision-Language-Action Models Understand Instructions? A Mechanistic Interpretability Study on Language Grounding

arXiv:2610.10178 · cs.RO · Submitted 2026-10-07 · Read on arXiv

cs.RO

Submitted: 2026-10-07

Updated: 2026-10-07

Terminology

• Activation Patching — 34× in this paper · explained in 2 episodes
• Interpretability — 16× in this paper · explained in 15 episodes
• Transformer — 15× in this paper · explained in 2 episodes
• Grounding — 13× in this paper · explained in 3 episodes
• Mechanistic interpretability — 13× in this paper · explained in 2 episodes
• Steering — 6× in this paper · explained in 2 episodes
• Benchmark — 5× in this paper · explained in 2 episodes
• Foundation Model — 5× in this paper · explained in 3 episodes
• Alignment — 4× in this paper · explained in 1 episode
• Embeddings — 4× in this paper · explained in 2 episodes
• Robustness — 4× in this paper · explained in 5 episodes
• Abstraction — 3× in this paper · explained in 1 episode
• Centered Kernel Alignment — 3× in this paper
• Core — 3× in this paper · explained in 2 episodes
• Foundation Models — 3× in this paper · explained in 7 episodes
• Verification — 3× in this paper · explained in 2 episodes
• Action Decoder — 2× in this paper · explained in 1 episode
• Diffusion Model — 2× in this paper · explained in 8 episodes
• Flow Matching — 2× in this paper · explained in 8 episodes
• Activation Steering — 1× in this paper · explained in 4 episodes
• Adam — 1× in this paper · explained in 2 episodes
• Attention Mechanism — 1× in this paper · explained in 3 episodes
• Benchmarking — 1× in this paper · explained in 3 episodes
• Centered Kernel Alignment (CKA) — 1× in this paper · explained in 2 episodes
• Cross-Attention Mechanism — 1× in this paper · explained in 2 episodes
• Diffusion Models — 1× in this paper · explained in 6 episodes
• Failure Modes — 1× in this paper · explained in 3 episodes
• Fine-Tuning — 1× in this paper · explained in 5 episodes
• Generalization — 1× in this paper · explained in 13 episodes
• Latent Diffusion Model — 1× in this paper · explained in 2 episodes
• Latent Diffusion Models — 1× in this paper · explained in 2 episodes
• Model-Agnostic — 1× in this paper · explained in 2 episodes
• Modular Architecture — 1× in this paper · explained in 4 episodes
• Transformer Architecture — 1× in this paper · explained in 2 episodes
• Vision-Language Model (VLM) — 1× in this paper · explained in 3 episodes
• Visual Grounding — 1× in this paper · explained in 2 episodes

Sources

• GPT-4 Technical Report
• Gemini Robotics: Bringing AI into the Physical World
• Automatic Discovery of Visual Circuits
• How does GPT-2 compute greater-than?: Interpreting mathematical abilities in a pre-trained language model
• AtP*: An efficient and scalable method for localizing LLM behaviour to components
• Gemma 3 Technical Report

Related papers

• FMT x: An Efficient and Asymptotically Optimal Extension of the Fast Marching Tree for Dynamic Replanning
• MPCFormer: A physics-informed data-driven approach for explainable socially-aware autonomous driving
• RoboLab: A High-Fidelity Simulation Benchmark for Analysis of Task Generalist Policies
• HRDexDB: A 4D Dexterous Grasping Dataset Across Human and Multiple Robot Embodiments
• APT: Action Expert Pretraining Improves Instruction Generalization of Vision-Language-Action Policies
• Fine-tuning is Not Enough: A Parallel Framework for Collaborative Imitation and Reinforcement Learning in End-to-end Autonomous Driving