Recover, Discover, Plan: Learning Skills and Concepts from Robot Failures

summary

The gist

The gist: ReSYNC introduces Recovery-Driven Synthesis of Relational Concepts (ReSYNC), a method that progressively discovers and refines state abstractions from failure-recovery experience to support

In short

ReSYNC introduces Recovery-Driven Synthesis of Relational Concepts to help robots avoid future failures. It uses an incremental dual-learning process: first, reinforcement learning learns specific recovery skills from failures, and second, concept learning discovers new relational predicates to generalize these skills. This allows the robot to convert local recoveries into global failure avoidance.

Key concepts

Skill Learning: This phase uses reinforcement learning (RL) to teach the robot how to recover from specific failures encountered during training. The robot learns a recovery policy and a terminal function that are relational, meaning they operate based on features of relevant objects. This builds the initial set of recovery skills.
Concept Learning: This phase discovers new relational predicates and operators that explain and generalize the learned recovery behaviors. It uses Offline Concept Learning with IVNTR, which alternates between learning symbolic structures and optimizing neural classifiers. This process creates new concepts that allow for more abstract planning later.
Relational Predicates: These are the abstract symbols or rules discovered by ReSYNC that describe relationships between objects in the environment. Instead of learning a policy for every failure, the robot learns these predicates, which act as a symbolic description to help it plan and generalize recovery skills across different scenarios.
Incremental Dual-Learning: This is the core methodology where skill learning and concept learning happen together in an iterative process. The system jointly learns skills (via RL) and concepts (via IVNTR). This interaction allows the robot to continuously refine its abstraction library, enabling it to solve increasingly complex, unseen problems.

Terminology used across episodes

Reinforcement Learning (RL) — explained in 25 episodes
Federated Learning — explained in 16 episodes
Retrieval-Augmented Generation (RAG) — explained in 16 episodes
Interpretability — explained in 15 episodes
Sample Complexity — explained in 15 episodes
Generalization — explained in 13 episodes

This episode discusses

The paper

Recover, Discover, Plan: Learning Skills and Concepts from Robot Failures · Read on arXiv

CMU University of California, Berkeley (CMU) · Princeton University of Technology (Princeton) · AI2 (AI2) · MIT Artificial Intelligence Laboratory (MIT AI2) · Centaur AI · Bosch Center for AI

Intelligent robots should not only recover from failures, but also acquire the abstract knowledge needed to avoid them in the future. While reinforcement learning (RL) can learn reactive recovery behaviors, training a separate policy for every distinct failure mode is highly inefficient. We introduce Recovery-Driven Synthesis of Relational Concepts (ReSYNC), the first approach that progressively discovers and refines state abstractions (relational predicates) from failure-recovery experience to support abstract planning. Unlike purely reactive methods, ReSYNC jointly learns skills and concepts through an incremental dual-learning process. In the skill-learning phase, the robot uses RL to learn to recover from failures seen in training tasks. In the concept-learning phase, the robot discovers new relational predicates and refines its abstract planning model to explain and generalize the learned recovery behaviors. This interaction enables ReSYNC to convert local recoveries seen during training into global failure avoidance at test time. Across four simulated domains, we show that ReSYNC's ability to continually expand and refine its abstraction library allows it to solve long-horizon, previously unseen problems, outperforming strong baselines by over 50%. Additionally, we demonstrate sim-to-real transfer of ReSYNC, where it performs real-world non-prehensile manipulation skills and generalizes to unseen scenarios through abstract planning. Overall, ReSYNC represents a significant step toward robots that autonomously acquire abstractions for scalable, failure-aware planning in the physical world.

Transcript

Introduction to the show: ident: Robotics Radio. Generated commentary on the latest robotics and control papers.

Rosa: I'm Rosa, and with me are Dev and Taro, guest researcher.

Dev: Today's paper: "Recover, Discover, Plan".

Rosa: The gist: ReSYNC introduces Recovery-Driven Synthesis of Relational Concepts (ReSYNC), a method that progressively discovers and refines state abstractions from failure-recovery experience to support abstract planning,

Dev: First, who's behind it and why it matters.

Title and authors: Rosa: So this paper is called "Recover, Discover, Plan: Learning Skills and Concepts from Robot Failures". It sounds like it’s about getting robots better at handling when things go wrong by not just fixing them but also learning the bigger picture of what's going on.

Dev: Yeah, the authors are Li and Mishra along with Liu, Tao, Kumar, Gray, Wickramarachchi and Francis. They’re coming from a few different places in AI research.

Taro: It’s interesting seeing researchers from different areas working together on something like this because it hints at a bigger problem in autonomy—how to handle the messy reality of physical interaction when things don't go according to plan.

Rosa: Exactly, and the title suggests they aren't just making robots that can recover; they are making them acquire abstract knowledge so they can avoid those failures in the future, which is a big difference from just reacting on the spot.

Dev: They’re aiming for something incremental, building up skills and concepts together rather than trying to solve everything at once.

Taro: That gradual learning approach sounds like it addresses a core limitation where most existing methods are either too reactive or require hand-designed skills upfront, which is something I see a lot in the autonomy space.

Rosa: It seems they’re pushing past just having a policy for every single failure mode because that gets inefficient quickly when you have many potential problems.

The paper's summary: Dev: In the paper, they introduce something called Recovery-Driven Synthesis of Relational Concepts, or ReSYNC. Essentially, they are proposing a method that progressively discovers and refines state abstractions—those relational predicates—directly from the experience of failures and recoveries.

Rosa: So instead of just learning how to recover from one specific failure, ReSYNC learns a new concept about the world that helps explain *why* that recovery worked or what to do next.

Taro: The summary breaks it down into two main parts: first, the skill-learning phase where the robot uses reinforcement learning to learn how to recover from failures seen in training tasks.

Dev: Then there’s the concept-learning phase, where they discover new relational predicates and refine their abstract planning model so they can explain and generalize those learned recovery behaviors.

Rosa: The key idea here is that this interaction allows ReSYNC to take local recoveries that happened during training and convert them into global failure avoidance when the robot is actually tested.

Taro: It moves beyond just having a skill for a specific situation; it builds a framework where those skills are grounded in a new, more abstract way of thinking about the environment itself.

The paper's improvements: Rosa: One major improvement they suggest is this incremental dual-learning process where the robot jointly learns skills and concepts. It starts with learning recovery skills from failures, and then discovers concepts that ground those skills in a new abstract planning model.

Dev: They show how this allows them to generalize to a combinatorial number of unseen failures by recomposing these new skills and concepts they’ve discovered.

Taro: That generalization capability is what really stands out because it means the robot doesn't just know how to fix the drawer being open, but it can apply that learned idea to entirely different types of objects or situations.

Rosa: They also have two designs for this progressive learning: compositional dreaming and concept fine-tuning, which let them expand their library for increasingly complex unseen tasks.

Dev: Compositional dreaming involves composing failure objects from previous stages with new objects, while concept fine-tuning incrementally expands the abstraction library using newly generated data while tuning classifiers for existing concepts before new skills change things.

Taro: And they even show that the system is robust to how you order your training tasks; reversing the sequence still gives it comparable performance to when you did it in the original order.

Conclusion: Rosa: So, to wrap up, ReSYNC is about robots acquiring those abstract knowledge structures from failure experience so they can use that planning model at test time to anticipate and avoid failures globally. It’s a significant step toward robots that can handle complex physical tasks without needing a specific policy for every single problem.

Dev: The system achieves success rates of seventy percent on compositionally novel tasks across four different physical domains, and it shows three things: it outperforms baselines by over fifty percent in the Blocked Stacking domain, it learns from multiple training stages without performance drops as the task horizon grows, and it doesn't forget previous tasks when learning new ones.

Taro: For me, what this means for autonomy is that we’re getting closer to systems that don't just rely on reactive recovery skills but can genuinely compose new solutions by reusing discovered concepts across entirely different manipulation scenarios.

Rosa: It does show sim-to-real transfer for non-prehensile recovery skills, meaning these learned behaviors can actually work in the physical world when things get messy and unexpected.

Dev: The paper "Recover, Discover, Plan: Learning Skills and Concepts from Robot Failures" shows how incremental learning of skills and concepts can lead to robust planning that generalizes better than previous reactive methods.

Taro: It’s a solid paper that shows the path toward robots that are fundamentally better at handling the unknown by building up their understanding of the world through its own mistakes.

More episodes

← Home