Embodied Turing Machines: Stateful Code for Robot Recursive Self-Improvement

summary

Video file (mp4)

The gist

The gist The embodied world is an Embodied Turing Machine, whose tape is the robot and environment state and rules are the policy.

In short

The embodied world is modeled as an Embodied Turing Machine where code acts as a policy and state is explicit. Code-Only-as-Policy (COAP) replaces large models with executable code, offering explicit state storage and controllable execution. This approach achieves a 70.24% success rate on RoboDojo tasks, suggesting that explicit, editable code can drive robot behavior.

Key concepts

Embodied Turing Machine Model
This model treats a robot and its surroundings like a computer tape. The tape stores two types of information: the environment's state (like where objects are) and the robot's state (like its arm position). Rules, or policy, dictate how the robot reads this tape and changes it.
Code-Only-as-Policy (COAP)
COAP is a method where code directly implements the Embodied Turing Machine. Instead of using a large model at runtime, the policy is written as explicit code storing state in variables and rules in logic. This makes every decision transparent and executable without needing a separate model during operation.
Explicit State
In COAP, the robot's situation is stored directly within the code itself. This state is inspectable, persistent, and accurate because it resides in standard code variables rather than being inferred by a black-box model. This explicit state allows for precise action planning and reliable recovery.
Recursive Self-Improvement (RSI)
COAP enables RSI where coding agents develop the robot's policy in a loop. The code acts as a universal interface, allowing agents to modify the library, run tests on development layouts, find improvements, and update the code iteratively. This creates a closed loop for continuous self-improvement.

Terminology used across episodes

This episode discusses

The paper

Embodied Turing Machines: Stateful Code for Robot Recursive Self-Improvement · Read on arXiv

Kairui Hu, Siyuan Hu, Fangzhou Hong, Zhaoxi Chen, Ziwei Liu

Nanyang Technological University

Transcript

Introduction to the show: ident: Robotics Radio. Generated commentary on the latest robotics and control papers.

Rosa: Today's paper: "Embodied Turing Machines".

Dev: The gist The embodied world is an Embodied Turing Machine, whose tape is the robot and environment state and rules are the policy.

Rosa: First, who's behind it and why it matters.

Title and authors: Rosa: To get into more detail on this paper, we’re looking at how they introduce this idea of COAP as a new way to handle these embodied problems.

Dev: They are proposing that instead of relying on a large model running at test time, the code itself acts as the policy, storing all the state in code variables and rules written directly in code logic >

Taro: That means no need for an external model call during execution, which is a big deal for real-time robotics because it’s much faster than having a vision language model constantly make decisions >

The paper's summary: Rosa: The main idea they summarize is that this approach gives the system explicit state, meaning the state isn't hidden inside a black box model; it’s stored in code that you can inspect and save >

Dev: They emphasize three main advantages for COAP: first, explicit state, which lets you store and check it easily; second, execution, because the code makes decision making controllable so you can recover from failures flexibly by backtracking and re-attempting; and third, extensibility where new tasks can reuse or extend a shared library >

Taro: So it’s not just about making decisions fast; it’s about having a structure where if something goes wrong, the code itself knows how to try again in a controlled way, which is important when the world doesn't behave as expected >

The paper's improvements: Rosa: When we look at what they suggest as improvements to this idea, the focus shifts toward recursive self-improvement.

Dev: They argue that these explicit state and controllable execution features make COAP suitable for recursive self-improvement, where coding agents develop the library in a closed loop, and each change is explicit and controllable >

Taro: That closed loop idea is interesting; it means the AI system can actually learn to improve its own tools by developing a new library all on its own through this code interaction >

Conclusion: Rosa: So, to wrap up this discussion on "Embodied Turing Machines: Stateful Code for Robot Recursive Self-Improvement," the paper suggests that COAP can be a powerful new direction for embodied tasks by making the state explicit and using code as the universal interface >

Dev: It shows that this approach, with a success rate of seventy point two four percent on RoboDojo's forty-two bimanual tasks, has real potential to benefit other systems like VLAs or Agent Harnesses because it produces successful trajectories quickly >

Taro: I just think the idea of coding agents developing the library in a closed loop through COAP is what really sets this apart for long-term progress in robotics >

Rosa: Well, it moves us toward a system where code isn't just an interface but the actual decision-maker, which could open up a lot of possibilities for how robots learn and adapt to novel situations.

Dev: I agree, because that explicit state management and fast execution means we can finally build systems that are truly controllable online without needing massive models constantly running in the background.

Taro: It seems like the real promise is in those coding agents being able to iterate on their own libraries, making progress happen through explicit, controllable changes rather than just hoping a big model finds the right path.

More episodes

← Home