Make Code as Policy Great Again: Frontier Agents Write, Call, and Evolve Robot Tools

arXiv:2609.39018 · cs.RO, cs.AI · Submitted 2026-09-30 · Read on arXiv

cs.RO, cs.AI

Submitted: 2026-09-30

Updated: 2026-09-30

Code: https://github.com/lykycy123/RoboJEV

Terminology

• Benchmark — 6× in this paper · explained in 2 episodes
• Foundation Model — 6× in this paper · explained in 3 episodes
• Benchmarking — 3× in this paper · explained in 3 episodes
• Foundation Models — 3× in this paper · explained in 7 episodes
• Frozen model — 2× in this paper · explained in 3 episodes
• Grounding — 2× in this paper · explained in 3 episodes
• Steering — 2× in this paper · explained in 2 episodes
• Temporal Reasoning — 2× in this paper · explained in 2 episodes
• Abstraction — 1× in this paper · explained in 1 episode
• Adaptation — 1× in this paper · explained in 1 episode
• Affordance — 1× in this paper · explained in 1 episode
• Alignment — 1× in this paper · explained in 1 episode
• Generalization — 1× in this paper · explained in 13 episodes
• Trace — 1× in this paper · explained in 2 episodes
• VLA Models — 1× in this paper · explained in 3 episodes
• Verification — 1× in this paper · explained in 2 episodes
• Visual Grounding — 1× in this paper · explained in 2 episodes
• World Action Model (WAM) — 1× in this paper · explained in 3 episodes

Sources

• OSWorld-Human: Benchmarking the Efficiency of Computer-Use Agents
• Agent S: An Open Agentic Framework that Uses Computers Like a Human
• Large Language Models as Tool Makers
• RoboDojo: A Unified Sim-and-Real Benchmark for Comprehensive Evaluation of Generalist Robot Manipulation Policies
• Show-Harness: Just a VLM Agent Can Play Robots
• In-Context Robot Learning with VLM Agents
• Universal Manipulation Interface: In-The-Wild Robot Teaching Without In-The-Wild Robots
• RHO: Your Coding Agent is Secretly a Roboticist
• CaP-X: A Framework for Benchmarking and Improving Coding Agents for Robot Manipulation
• VGGT-DP: Generalizable Robot Control via Vision Foundation Models
• RT-Trajectory: Robotic Task Generalization via Hindsight Trajectory Sketches
• VIA: Visual Interface Agent for Robot Control
• Inner Monologue: Embodied Reasoning through Planning with Language Models
• RoboStream: Weaving Spatio-Temporal Reasoning with Memory in Vision-Language Models for Robotics
• Sketch Interface for Teleoperation of Mobile Manipulator to Enable Intuitive and Intended Operation: A Proof of Concept
• Agent as Policy for Robotic Manipulation
• HAMSTER: Hierarchical Action Models For Open-World Robot Manipulation
• MOKA: Open-World Robotic Manipulation through Mark-Based Visual Prompting
• Spatial Policy: Guiding Visuomotor Robotic Manipulation with Spatial-Aware Modeling and Reasoning
• ASPIRE: Agentic /Skills Discovery for Robotics

Related papers

• FMT x: An Efficient and Asymptotically Optimal Extension of the Fast Marching Tree for Dynamic Replanning
• MPCFormer: A physics-informed data-driven approach for explainable socially-aware autonomous driving
• RoboLab: A High-Fidelity Simulation Benchmark for Analysis of Task Generalist Policies
• HRDexDB: A 4D Dexterous Grasping Dataset Across Human and Multiple Robot Embodiments
• APT: Action Expert Pretraining Improves Instruction Generalization of Vision-Language-Action Policies
• Fine-tuning is Not Enough: A Parallel Framework for Collaborative Imitation and Reinforcement Learning in End-to-end Autonomous Driving