Robotic Long-Horizon Manipulation with Progressive In-Context Code Generation and Episodic Feedback

arXiv:2503.21969 · cs.RO, cs.AI · Submitted 2025-03-27 · Read on arXiv

Listen

Radio episode about this paper

Transcript

Introduction to the show: ident: Robotics Radio. Generated commentary on the latest robotics and control papers.

Rosa: Today's paper: "Robotic Long-Horizon Manipulation with Progressive In-Context Code Generation and Episodic Feedback".

Dev: Embodied long-horizon manipulation requires robotic systems to translate multimodal inputs into executable actions,

Rosa: First, who's behind it and why it matters.

Paper summary: Rosa: So we're looking at this paper called "Robotic Long-Horizon Manipulation with Progressive In-Context Code Generation and Episodic Feedback," and the main idea is that robotic systems need to translate things like vision and language into actual actions, but current learning methods are too reliant on huge task-specific datasets.

Dev: That sounds like a big gap, Rosa; what exactly does this paper claim it can do to fix that? I'm interested in how they propose using LLMs instead of just relying on those old low-level controllers.

Rosa: Well, the authors are proposing a completely different structure, essentially discarding the reliance on pre-trained low-level policies and letting the LLM directly generate executable code plans within a closed loop framework. This means it's moving away from assuming perfect execution from those underlying policies <ref:2503.21969#pg1>.

Taro: I'm curious about the core mechanism there; what makes this approach better than just using an LLM for simple planning without any feedback? Does it handle the complexity of a long horizon task differently?

Dev: It handles it by setting up a dual-tunnel pipeline, Rosa, which involves a planner and a reporter, forming this closed loop <ref:2503.21969#pg2>. The paper claims this structure allows the LLM to generate coherent plans for complex long-horizon tasks by using in-context learning with progressive example difficulty and Chain-of-Thought prompting <ref:2503.21969#pg1>.

Rosa: That's what caught my eye; they are using CoT prompting combined with that structured way of feeding examples to guide the LLM's reasoning process rather than just pattern matching <ref:2503.21969#pg1>. They focus on how the model decomposes tasks and their logic step by step, which they say enables better generalization beyond what it has seen before <ref:2503.21969#pg1>.

Taro: If the LLM is doing the reasoning, what happens when things go wrong in the physical world? How does this framework account for unexpected events or when the world misbehaves during execution?

Dev: That's where that second part of their proposal comes in; they have a reporter tunnel powered by a Vision-Language Model that evaluates task outcomes at the end of each loop, rather than step-by-step <ref:2503.21969#pg2>. This reporter provides structured contextual feedback with things like a success signal, the object to operate on, its location, and where the target is located <ref:2503.21969#pg2>.

Rosa: That structured feedback is what allows the planner to revise and recover effectively, which they emphasize because it improves evaluation reliability and enables efficient recovery in long-horizon tasks <ref:2503.21969#pg1>. It’s not just checking if an action succeeded or failed; it’s giving the planner precise data on the overall task state at that point <ref:2503.21969#pg2>.

Taro: So, when things misbehave, the system gets this structured update instead of just a generic failure signal? Does this mean it can learn from its own mistakes in a way that simple reinforcement learning can't easily do?

Paper summary: Dev: Exactly; because the feedback is contextual and detailed—specifying exactly which object was where and what the intended action was—it gives the planner much richer information to correct its course <ref:2503.21969#pg2>. This level of detail helps mitigate the noise that comes from just relying on per-step inference <ref:2503.21969#pg1>.

Rosa: It sounds like a really solid way to handle the long horizon by breaking it down and then getting detailed course corrections when the system drifts off track <ref:2503.21969#pg2>. But I do have to ask, Rosa's question for us as listeners: does this framework actually work outside of a controlled lab environment? How long can we expect it to operate reliably in a real-world setting?

Taro: That's the big question for the field, Rosa; if it needs that kind of detailed feedback loop to function properly, its reliance on stable environments might be a limitation. We need to see how robust these code generation plans are when confronted with real-world noise and variability <ref:2503.21969#pg0>.

Dev: From an engineering standpoint, the closed-loop nature implies there's an inherent latency involved in getting the observation back and generating the next plan, so we have to be very careful about loop rates and how that latency affects stability <ref:2503.21969#pg1>. We need to ensure those execution loops are fast enough for the system to react appropriately given the feedback cycle <ref:2503.21969#pg1>.

Rosa: It seems like the paper suggests it can handle complex recovery actions in real-world scenarios, but we need more proof on sustained operation outside of a simulation environment, especially since it's generating code plans from scratch <ref:2503.21969#pg0>. We need to see if the generalization holds up when the visual input is messy and the physical dynamics are unpredictable <ref:2503.21969#pg0>.

Taro: The implications of this framework, if it proves robust outside the lab, is that we could move towards systems that don't need massive amounts of data for every new manipulation task; they could adapt incrementally just by seeing a few examples and following the CoT reasoning path <ref:2503.21969#pg1>.

Dev: If it works reliably in real-world settings, the impact is huge because we're talking about systems that can handle complex, multi-step physical tasks without needing an impossibly large pre-labeled dataset for every single scenario <ref:2503.21969#pg0>. That would significantly reduce the reliance on painstakingly curated datasets for deploying these kinds of robots in varied settings.

Rosa: It really is about taking the burden off the dataset creation side, moving it toward a system that learns how to reason and adapt incrementally, which seems like a major step forward for practical robotics <ref:2503.21969#pg0>. We're hoping this framework can handle tasks like recovering from occluded objects in real-world scenarios, which is something we see happening now <ref:2503.21969#pg0>.

Paper summary: Taro: And if it can handle those complex recovery actions effectively, the impact on autonomous systems could be substantial because it means these robots become much more resilient to the inevitable imperfections of physical reality <ref:2503.21969#pg0>. That kind of adaptability is what we need for true autonomy in complex environments.

Dev: I agree with Taro; resilience against noise is critical, and this closed-loop feedback mechanism seems specifically designed to build that resilience into the planning process itself <ref:2503.21969#pg2>. We're looking at how the loop rate interacts with this feedback structure to see if we can maintain stability during high-speed recovery maneuvers <ref:2503.21969#pg1>.

Rosa: So, to wrap up this part, the paper introduces this dual-tunnel approach using LLM code generation guided by CoT and progressive examples coupled with structured episodic feedback for robust long-horizon manipulation tasks <ref:2503.21969#pg0>. It’s a framework focused on making the planning and adaptation process more generalizable than previous methods <ref:2503.21969#pg1>.

Taro: And it really hinges on that structured feedback from the reporter tunnel to give the planner enough context to actually recover when things go sideways <ref:2503.21969#pg2>. That's a key differentiator we should be paying attention to.

Dev: The engineering challenge remains making sure the entire cycle, from observation through plan generation and execution feedback, runs fast and reliably under real-world conditions <ref:2503.21969#pg1>. That loop latency is something we have to measure carefully when we test this stuff <ref:2503.21969#pg1>.

Rosa: We'll keep an eye on the real-world validation results to see how long they can sustain this kind of performance outside of controlled lab setups, which is our main question right now <ref:2503.21969#pg0>.

Taro: It’s exciting because it shifts the focus from just training massive models on datasets to creating a system that can reason and adapt incrementally in a closed physical loop <ref:2503.21969#pg1>. That kind of reasoning capability is what makes us optimistic about future autonomous systems <ref:2503.21969#pg0>.

Dev: It’s definitely a shift in methodology, moving away from relying on perfect low-level execution policies toward a self-correcting code generation pipeline <ref:2503.21969#pg1>. We'll need to see concrete data on the success rates across diverse tasks before we can really gauge its practical utility in deployment <ref:2503.21969#pg0>.

Rosa: So, for our listeners, the big picture is that this paper lays out a new way to build long-horizon manipulation systems by using LLMs to generate code plans in a closed loop with incremental learning and structured feedback <ref:2503.21969#pg0>. We'll be looking closely at how it holds up when we take these systems out of the simulation and into messy, unpredictable environments <ref:2503.21969#pg0>.

Conclusion: Rosa: So, to wrap up this discussion on "Robotic Long-Horizon Manipulation with Progressive In-Context Code Generation and Episodic Feedback," we've seen how they tackle complex physical tasks by using an LLM to write code plans in a closed loop with structured feedback.

Dev: I think the title really sums up what they did, Rosa; it’s about making manipulation longer than just a single step by having the system constantly check its progress and correct course based on that episodic feedback.

Taro: I agree with Dev; the progression part in that title suggests they aren't just using one static planning strategy, but rather adapting their in-context learning as they go along.

Rosa: Exactly, and it really points to the authors’ goal of creating a system that can handle those long sequences without needing a massive dataset for every single new manipulation challenge.

Dev: That means if we can get this feedback loop running reliably, the required data for training these planners could drop significantly, which is a big deal for deployment.

Taro: And I think the real implication is that we're moving towards agents that are more resilient because they are explicitly taught how to recover from errors through that structured feedback mechanism.

Rosa: It really suggests a future where robots can tackle incredibly intricate assembly or manipulation tasks in environments far messier than what we see in current lab settings.

Dev: I’m still focused on the technical hurdles, though; if this closed loop has any latency issues, it could severely limit how fast those recovery actions can happen in real-time.

Taro: That's a valid concern, Dev; the speed of that feedback cycle is what determines if the system can actually keep up with dynamic physical interactions when things go wrong.

Rosa: So, while the theoretical framework looks very promising for generalizability and long-horizon tasks, we still need to see sustained success in those unpredictable real-world scenarios before we can really say this is ready for widespread deployment.

Dev: That's the sticking point, Rosa; we need to see concrete data showing it handles the noise of the physical world without crashing or looping infinitely during recovery attempts.

School of Computation, Information and Technology, Technical University of Munich, Germany

cs.RO, cs.AI

Submitted: 2025-03-27

Updated: 2026-10-03

Comments: Accepted by IROS 2026 (8 pages)

License: http://creativecommons.org/licenses/by/4.0/

Importance score: 83/100

The gist: Embodied long-horizon manipulation requires robotic systems to translate multimodal inputs into executable actions, and this work addresses the limitations of existing learning-based approaches by

Key concepts

Planner Tunnel
This component uses an LLM as the main planner that collaborates with other models. It interprets multimodal inputs and generates high-level task plans by calling a library of pre-defined skill primitives. It uses progressive in-context learning and Chain-of-Thought reasoning to create coherent plans for complex, long sequences of actions.
Reporter Tunnel
This part uses a Vision-Language Model (VLM) to evaluate the overall task state after every execution loop, rather than step-by-step. It parses RGB-D observations to provide structured feedback detailing success signals, object locations, target locations, and intended actions. This detailed feedback enables the system to revise and recover effectively.
Progressively Increasing Example Difficulty
Instead of using random examples for in-context learning, this method structures the training data by gradually increasing its complexity. This ensures that the LLM learns to generalize better beyond previously seen examples, making it more reliable when faced with entirely new and challenging tasks during execution.
Temporal Abstraction
The framework executes an entire sequence of skill primitives before receiving feedback from the reporter tunnel. This temporal abstraction reduces noise caused by per-step inference and limits the accumulation of errors, allowing the agent to execute longer, more robust sequences before correction.

Terminology

Summary

Embodied long-horizon manipulation requires robotic systems to translate multimodal inputs into executable actions, and this work addresses the limitations of existing learning-based approaches by proposing a closed-loop framework that leverages large language models for direct code generation and incremental few-shot adaptation. The gist is: "Our framework introduces a dual-tunnel code-generation framework that leverages in-context learning with progressive example difficulty and CoT prompting, enabling LLMs to generate coherent plans for complex long-horizon tasks."

The Proposed Framework (DAHLIA)

The proposed framework is named DAHLIA: Data-Agnostic Hierarchical Closed-loop Framework for Long-horizon Incremental In-context Adaptation. It discards reliance on pre-trained low-level controllers and instead uses LLM-based code generation for solving long-horizon manipulation tasks. The framework consists of two main components: a planner tunnel and a reporter tunnel, forming a closed loop.

Planner Tunnel Functionality

The planner tunnel is powered by an LLM acting as the primary planner, collaborating with multiple co-planners to interpret multimodal inputs and generate high-level executable task plans by invoking a library of skill primitives. To ensure robust planning for complex tasks, the framework employs few-shot in-context learning with progressively increasing example difficulty, combined with the CoT [Chain-of-Thought], where CoT prompts can guide the model to reason step by step. This focuses on the reasoning process itself rather than just pattern matching, enabling the planner to generalize beyond seen examples and reliably handle unfamiliar tasks through structured reasoning. Detailed APIs are provided in prior knowledge, including available APIs, a coordinate system definition (e.g., front: x+, left: y-, top: z+), and general rules to ensure stable outputs.

Reporter Tunnel Functionality

The reporter tunnel is powered by a Vision-Language Model (VLM) that evaluates task outcomes after each execution loop, forming a closed-loop with structured feedback. Unlike prior approaches that evaluate actions step-by-step, the reporter assesses the overall task state at the end of each loop. It takes paired RGB-D observations and uses parse completion to provide structured contextual feedback consisting of four key elements: (1) a binary success signal, (2) the [object] to operate, (3) its [location], (4) the [target] location, and (5) the intended [action]. This detailed information enables the planner to revise and recover effectively.

Key Mechanisms for Robustness

The framework incorporates several mechanisms designed to enhance generalization and reliability:

  1. In-context learning is structured using progressively increasing example difficulty rather than randomly selected examples, addressing the limitation of prior works where random examples often fail to improve generalization on unseen tasks.

  2. Chain-of-Thought (CoT) prompting guides the LLM to decompose tasks and their logic to reach a solution, focusing on reasoning patterns for better generalization.

  3. Temporal abstraction is used instead of per-step inference, allowing the agent to execute an entire sequence of primitives before receiving inter-loop feedback, which substantially reduces noise from per-step inference and limits error accumulation.

Key Contributions and Results

The key contributions include:

We introduce a dual-tunnel code-generation framework that leverages in-context learning with progressive example difficulty and CoT prompting, enabling LLMs to generate coherent plans for complex long-horizon tasks.

We employ a closed-loop feedback with contextual refinement in the code-generation-only method, improving evaluation reliability and enabling efficient recovery in long-horizon tasks.

The framework demonstrates superior generalization and adaptability across diverse, long-horizon tasks in both simulation and real-world scenarios, achieving SOTA performance on 30+ diverse seen and unseen long-horizon tasks. The results show that the closed-loop setup achieves the highest success rates across both seen and unseen tasks, with DAHLIA outperforming baselines like CaP by incorporating structured feedback. Real-world performance demonstrates its ability to handle complex scenarios, such as recovering from occluded bread items in a basket.

Scalability and Real-World Validation

The framework is evaluated across diverse benchmarks including LoHoRavens, CALVIN, and Franka Kitchen. The results confirm that our default dual-tunnel DAHLIA setup consistently outperforms its planner-only variant (DAHLIA GPT4o-miniP), demonstrating the effectiveness of its closed-loop feedback in improving execution reliability. The framework achieves 100% success in most tasks, including lift block, open drawer, and slider cabinet. Real-world demonstrations show the system's ability to perform complex recovery actions, such as re-picking occluded objects while avoiding obstacles through closed-loop feedback recovery. This validates the pipeline’s ability to generalize across diverse task scenarios and embodiments.

Improvements for AI systems

Here are specific improvements for AI systems based on the DAHLIA framework:

  1. The system will be able to perform complex, multi-step, long-horizon manipulation tasks in unstructured environments (e.g., real-world settings like Franka Kitchen or cluttered rooms) by interpreting natural language instructions and directly generating executable code plans, bypassing the need for fine-tuned low-level RL policies.

  2. The system will exhibit superior generalization to unseen tasks across diverse embodiments and environments because it utilizes:

  3. The system will demonstrate robust adaptation to novel scenarios through a structured few-shot learning paradigm, where examples are presented in an incrementally increasing order of difficulty, guided by Chain-of-Thought (CoT) reasoning prompts, ensuring the LLM learns underlying task logic rather than surface patterns.

  4. The system will achieve high reliability and recover from execution errors in long sequences by employing a closed-loop feedback mechanism where a Vision-Language Model (VLM) evaluates task state after each action loop using paired RGB-D observations and provides structured, context-aware feedback (identifying object location, target location, required action).

  5. The system will reduce computational overhead and error accumulation compared to per-step inference by adopting temporal abstraction—executing entire sequences of primitives before receiving inter-loop feedback.

  6. The system will be highly adaptable to partial observability (e.g., occluded objects or unexpected environmental perturbations) because the structured feedback loop enables efficient re-planning, allowing the agent to detect failures and revise its plan dynamically until task completion is confirmed.

Abstract

Robotic long-horizon manipulation requires robots to compose perception, reasoning, and action over extended task sequences, yet existing language-conditioned frameworks often rely on dense step-wise feedback, learned action policies, or unstructured prompting, which limits robustness and generalization. We propose DAHLIA, a data-agnostic code-generation framework that treats long-horizon manipulation as episodic task planning and evaluation. DAHLIA uses progressively organized in-context examples with chain-of-thought reasoning to guide an LLM in synthesizing executable code plans from language instructions. To improve robustness under partial observability, a VLM-based reporter evaluates task outcomes only after each execution episode and provides structured feedback for replanning, avoiding the overhead of per-step inference. Experiments on LoHoRavens, CALVIN, Franka Kitchen, and real-world manipulation tasks show that DAHLIA achieves strong performance on over 30 long-horizon tasks and improves generalization to unseen scenarios, including cluttered scenes and occluded objects.

Sources

Related papers