Agent as Policy for Robotic Manipulation

arXiv:2609.12541 · cs.CL · Submitted 2026-09-11 · Read on arXiv

cs.CL

Submitted: 2026-09-11

Updated: 2026-09-28

Code: https://github.com/kevinzakka/mink

License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/

Terminology

• Alignment — 11× in this paper · explained in 1 episode
• Verification — 9× in this paper · explained in 2 episodes
• Calibration — 8× in this paper · explained in 7 episodes
• Adaptation — 3× in this paper · explained in 1 episode
• Trace — 3× in this paper · explained in 2 episodes
• Accuracy — 2× in this paper · explained in 1 episode
• Affordance — 2× in this paper · explained in 1 episode
• Grounding — 2× in this paper · explained in 3 episodes
• Task Planning — 2× in this paper · explained in 2 episodes
• APEX — 1× in this paper · explained in 1 episode
• Action Flow — 1× in this paper · explained in 1 episode
• Agentic Framework — 1× in this paper · explained in 6 episodes
• Agentic Workflows — 1× in this paper · explained in 1 episode
• Aggregation — 1× in this paper · explained in 1 episode
• Benchmark — 1× in this paper · explained in 2 episodes
• Benchmarking — 1× in this paper · explained in 4 episodes
• Consistency — 1× in this paper · explained in 3 episodes
• Core — 1× in this paper · explained in 2 episodes
• Diffusion Model — 1× in this paper · explained in 8 episodes
• Diffusion Models — 1× in this paper · explained in 6 episodes
• Generative AI — 1× in this paper · explained in 2 episodes
• Reinforcement Learning — 1× in this paper · explained in 4 episodes
• Robustness — 1× in this paper · explained in 5 episodes
• Steering — 1× in this paper · explained in 2 episodes

Sources

• Do As I Can, Not As I Say: Grounding Language in Robotic Affordances
• $\pi_0$: A Vision-Language-Action Flow Model for General Robot Control
• RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Control
• RoboScript: Code Generation for Free-Form Manipulation Tasks across Real and Simulation
• GaP: A Graph-as-Policy Multi-Agent Self-Learning Harness For Variational Automation Tasks
• Diffusion Policy: Visuomotor Policy Learning via Action Diffusion
• Addressing the Orchestration Gap in Generalist Robots via Physical Agency
• MetaGPT: Meta Programming for A Multi-Agent Collaborative Framework
• Inner Monologue: Embodied Reasoning through Planning with Language Models
• OpenVLA: An Open-Source Vision-Language-Action Model
• RoboClaw: An Agentic Framework for Scalable Long-Horizon Robotic Tasks
• Unified Video Action Model
• A Survey of State of the Art Large Vision Language Models: Alignment, Benchmark, Evaluations and Challenges
• Code as Policies: Language Model Programs for Embodied Control
• ASPIRE: Agentic /Skills Discovery for Robotics
• Toolformer: Language Models Can Teach Themselves to Use Tools
• Reflexion: Language Agents with Verbal Reinforcement Learning
• ProgPrompt: Generating Situated Robot Task Plans using Large Language Models
• AutoMate: Specialist and Generalist Assembly Policies over Diverse Geometries
• OpenHands: An Open Platform for AI Software Developers as Generalist Agents

Related papers

• Exploring Solution Divergence and Its Effect on Large Language Model Problem Solving
• Ishigaki-IDS-Bench: A Benchmark for Generating Information Delivery Specification from BIM Information Requirements
• Subliminal Steering: Stronger Encoding of Hidden Signals
• MedStruct-S: A Benchmark for Key Discovery, Key-Conditioned QA and Semi-Structured Extraction from OCR Clinical Reports
• The End of Transformers? On Challenging Attention and the Rise of Sub-Quadratic Architectures
• Untangling the Mechanisms of Misleading Context in Medical Question Answering