Experience-Guided Initiation Search for Learned Skills in Skill Composition

summary

Video file (mp4)

The gist

The gist: EVIS, an Experience-Guided and BehaviorValidated Initiation Search framework, discovers reliable initiation configurations for frozen learned skills in new environments by using historical

In short

EVIS is a framework that uses historical execution data to suggest promising ways to start learned skills in new environments. It prioritizes these suggestions and then uses limited real-world tests to confirm if they actually work. This method helps find reliable skill configurations with fewer expensive interactions, making it efficient for deploying pre-trained skills.

Key concepts

Initiation Configuration
This refers to the specific settings or starting conditions required before a learned skill can begin executing. It's crucial because changing this configuration affects not only the current skill but also the physical state passed to all subsequent skills in a sequence, meaning local success doesn't guarantee overall task success.
Experience-Guided Proposal
EVIS uses past execution records to generate a list of potential starting configurations. It ranks these candidates based on historical performance, transferring the ordering information from old data rather than relying on the predictions themselves. This guides the search toward configurations that have been successful in similar contexts.
Behavioral Validation
Once a configuration is proposed by history, it must be tested in a new environment. EVIS uses these target-environment rollouts to validate if the historical candidate is still effective. This step separates historical guidance (where to look) from behavioral evidence (whether to trust the candidate).
Budgeted Return Rule
EVIS employs a specific rule to decide whether a provisional skill configuration should be accepted, even after initial validation. This rule uses a finite budget of target interactions ($K_{max}$) and assesses the trade-off between accepting a promising candidate and continuing the search.

Terminology used across episodes

This episode discusses

The paper

Experience-Guided Initiation Search for Learned Skills in Skill Composition · Read on arXiv

Qixuan Li, Yanhong Zhao, Jincheng Yu

Deploying frozen learned skills, such as Vision-Language-Action (VLA) policies, in new environments requires identifying initiation configurations that support reliable execution. In skill composition, an initiation configuration affects not only the current skill but also the physical state passed to subsequent skills, so successful execution of an individual skill does not necessarily imply successful completion of the composed task. Estimating target-specific capability through extensive rollouts is costly in real-world deployment, while directly reusing historical experience can be unreliable under environment changes. We propose EVIS, an Experience-Guided and Behavior-Validated Initiation Search framework for discovering reliable initiation configurations under limited target interaction. EVIS uses historical execution experience to prioritize promising candidates and target-environment behavior to validate whether they remain effective. We evaluate EVIS on single-skill and two-stage manipulation tasks with frozen VLA policies. EVIS reduces mean target-environment queries and improves reliable candidate discovery under small interaction budgets. These results show that combining historical guidance with target-side behavioral validation can reduce the interaction cost of deploying frozen learned skills in new environments.

Transcript

Introduction to the show: ident: Robotics Radio. Generated commentary on the latest robotics and control papers.

Rosa: Today's paper: "Experience-Guided Initiation Search for Learned Skills in Skill Composition".

Dev: The gist: EVIS, an Experience-Guided and BehaviorValidated Initiation Search framework,

Rosa: First, who's behind it and why it matters.

Title and authors: Rosa: So we’re looking at this paper today called "Experience-Guided Initiation Search for Learned Skills in Skill Composition". It’s by Qixuan Li, Yanhong Zhao, and Jincheng Yu. Basically, they're tackling the problem of using frozen learned skills in new places. The core issue is that just because a sequence of skills looks valid symbolically doesn't mean it will actually work when you move to a different environment.

Dev: Yeah, and the authors are proposing this framework called EVIS to find those reliable initiation configurations without needing tons of costly target interactions. They’re focusing on how historical execution data can guide the search process first before testing anything in the new world.

Taro: It sounds like they’re trying to bridge that gap between what worked before and what will work now, which is a big challenge when dealing with complex physical states passed between skills, you know?

Rosa: Exactly. The paper suggests that instead of just looking at historical data as a final answer, you use it to propose candidates for initiation configurations and then you test those proposals in the new environment to see if they actually hold up.

Dev: Right, so the main idea is separating the search strategy from the validation step; history tells us where to look first, and target outcomes tell us if we trust that candidate.

The paper's summary: Rosa: The EVIS framework uses historical execution records to create a set of promising candidates, which they call historical proposals. These proposals are based on context-specific Gaussian process classifiers and a retrieval-weighted predictor, which essentially just ranks the initiation configurations based on what worked previously.

Dev: And then they take those ranked candidates and run them in the target environment to see if they lead to a successful rollout under the objective G. If it succeeds, it gets validated as a provisional candidate, but you don't accept it immediately.

Taro: That sounds smart because instead of throwing every possible configuration at the wall in the new place, you’re narrowing down the search based on past performance first. But what happens when those historical proposals aren't good enough for the new environment?

Rosa: Well, that’s where they have a recovery search stage. If a provisional candidate gets rejected or if there aren't enough good historical proposals, EVIS doesn't just stop; it modifies where the search proceeds to try and find more options.

Dev: They use target-side observations accumulated during the discovery process to redirect the remaining search, which means they can broaden their search geometrically by selecting a configuration that is far away from what they've already found so far.

The paper's improvements: Rosa: One of the key contributions here is treating the initiation configuration as an explicit interface for task-level planning over frozen learned skills, which means it makes finding reliable configurations in new environments much more structured. They aim to do this with as few costly target-environment rollouts as possible.

Dev: They also introduced a finite-budget decision rule, Assessk(z), which acts like a screening mechanism. It tells the system whether to accept a provisional candidate based on some score and whether it should keep searching or reject it based on failure counts and acceptance rates.

Taro: That sounds like they’re building in robustness for when things go wrong; if the initial historical guidance fails, the recovery search is supposed to pull them back into a promising area instead of just giving up.

Rosa: Precisely. They show that combining historical guidance with target-side behavioral validation helps reduce mean target-environment queries significantly. For example, in single-skill discovery, they reduced those queries by fifty-nine point four percent compared to a baseline method using fixed historical reuse while keeping the confirmation outcomes the same.

Dev: And for multi-stage tasks, they reported reliable rates of forty-one point seven percent and fifty-eight point three percent at a budget of B = ten which is much better than the history-agnostic Scratch-Sobol method that only hit an eight point three percent reliable rate.

Conclusion: Rosa: To wrap this up, the paper on Experience-Guided Initiation Search for Learned Skills in Skill Composition shows that we can use historical data to prioritize where to look and target outcomes to validate if those candidates still work when transferring skills. This combination really cuts down on how much interaction is needed in a new environment.

Dev: The main implication is that you don't need perfect transfer; you just need a reliable way to screen past experience against the current environment's behavior, and EVIS does that by using a search policy z t = At(DH, D t-one) based on historical data DH and accumulated target observations D t-one <ref:2610.11418#pg1>.

Taro: From my side, it’s about building a system that can actually handle when the world misbehaves. The recovery search part is important because it ensures that if the initial historical proposals fail, we don't get stuck in a local optimum and we can explore configurations far from what we already found.

Rosa: So, by using this Experience-Guided and BehaviorValidated Initiation Search framework, they’ve shown how historical experience can be reused effectively without having to assume perfect transfer to a new environment. That's the summary of the work on Experience-Guided Initiation Search for Learned Skills in Skill Composition.

More episodes

← Home