CodeActionBench: Evaluating Agentic Code-as-Policy for Embodied Manipulation

arXiv:2609.33807 · cs.RO, cs.AI · Submitted 2026-09-27 · Read on arXiv

cs.RO, cs.AI

Submitted: 2026-09-27

Updated: 2026-09-27

Code: https://github.com/openai/codex

Terminology

• Verifier — 38× in this paper · explained in 2 episodes
• Benchmark — 24× in this paper · explained in 2 episodes
• Alignment — 9× in this paper · explained in 1 episode
• Calibration — 9× in this paper · explained in 7 episodes
• Verification — 9× in this paper · explained in 2 episodes
• Fine-Tuning — 4× in this paper · explained in 5 episodes
• Generalization — 4× in this paper · explained in 13 episodes
• Adaptation — 3× in this paper · explained in 1 episode
• Benchmarking — 3× in this paper · explained in 3 episodes
• Generative AI — 3× in this paper · explained in 2 episodes
• Grounding — 3× in this paper · explained in 3 episodes
• Task Planning — 3× in this paper · explained in 2 episodes
• Affordance — 2× in this paper · explained in 1 episode
• Foundation Model — 2× in this paper · explained in 3 episodes
• Tool Center Point — 2× in this paper
• Trace — 2× in this paper · explained in 2 episodes
• Abstraction — 1× in this paper · explained in 1 episode
• Action Space — 1× in this paper · explained in 1 episode
• Adam — 1× in this paper · explained in 2 episodes
• Consistency — 1× in this paper · explained in 3 episodes
• Core — 1× in this paper · explained in 2 episodes
• Cross-Embodiment Transfer — 1× in this paper · explained in 2 episodes
• Embodied AI — 1× in this paper · explained in 2 episodes
• Model Context Protocol (MCP) — 1× in this paper · explained in 8 episodes
• Physical Constraints — 1× in this paper · explained in 7 episodes
• Robustness — 1× in this paper · explained in 5 episodes
• Temporal Reasoning — 1× in this paper · explained in 2 episodes
• VLA Models — 1× in this paper · explained in 3 episodes

Sources

• SAM 3: Segment Anything with Concepts
• RoboScript: Code Generation for Free-Form Manipulation Tasks across Real and Simulation
• RoboTwin 2.0: A Scalable Data Generator and Benchmark with Strong Domain Randomization for Robust Bimanual Robotic Manipulation
• ETA: A New Agentic Paradigm for Embodied Tasks
• CaP-X: A Framework for Benchmarking and Improving Coding Agents for Robot Manipulation
• Instruct2Act: Mapping Multi-modality Instructions to Robotic Actions with Large Language Model
• Kimi K3: Open Frontier Intelligence
• DyPES-VLA: Learning Shared Dynamics Priors and Embodiment-Specific Control for Cross-Embodiment Manipulation
• From Fixed to Free Cameras: Calibration-Free View-Robust Vision-Language-Action Model
• Evaluating Real-World Robot Manipulation Policies in Simulation
• Being-H0.5: Scaling Human-Centric Robot Learning for Cross-Embodiment Generalization
• Robotic Long-Horizon Manipulation with Progressive In-Context Code Generation and Episodic Feedback
• G$^3$VLA: Geometric inductive bias for Vision-Language-Action Models
• cuRobo: Parallelized Collision-Free Minimum-Jerk Robot Motion Generation
• In-Context VLA: Endowing Vision-Language-Action Models with Language via In-Context Post-Training and Agentic Tool Use
• GeoProp: Grounding Robot State in Vision for Generalist Manipulation

Related papers

• FMT x: An Efficient and Asymptotically Optimal Extension of the Fast Marching Tree for Dynamic Replanning
• MPCFormer: A physics-informed data-driven approach for explainable socially-aware autonomous driving
• RoboLab: A High-Fidelity Simulation Benchmark for Analysis of Task Generalist Policies
• HRDexDB: A 4D Dexterous Grasping Dataset Across Human and Multiple Robot Embodiments
• APT: Action Expert Pretraining Improves Instruction Generalization of Vision-Language-Action Policies
• Fine-tuning is Not Enough: A Parallel Framework for Collaborative Imitation and Reinforcement Learning in End-to-end Autonomous Driving