SearchSkill: Teaching LLMs to Use Search Tools with Evolving Skill Banks

Teaching language models to use search tools is not only a question of whether they search, but also of whether they issue good queries. This framework proposes SEARCHSKILL, which makes query planning explicit through reusable search skills and an evolving skill bank, improving retrieval behavior across open-source and closed-source models by yielding better query formulation and more correct answers within a small search budget.

How it works

SEARCHSKILL introduces a framework that factorizes each interaction step into explicit skill selection and tool execution. Concretely, the model first emits a keyword tag to declare the skill(s) it will use at the current turn, and then either issues a query or returns an answer; retrieved evidence is fed back through evidence. This token-level interface makes the skill choice an explicit, trainable control variable before every retrieval or answer action.

The framework operates in two main stages of supervised fine-tuning (SFT). The first stage teaches the basic SearchSkill action format: each assistant turn emits keyword followed by one query or answer using a validated trajectory pool. The second stage continues from the first-stage checkpoint and repacks the same trajectory supervision into the select-read-act protocol, teaching the model to choose skill IDs from the SkillBank index and condition the next action on that selected card content.

The Evolving SkillBank

The SkillBank is not fixed; it is an evolving SkillBank that expands or refines itself from recurrent failure patterns. This evolution is managed through a stage-wise process aligned with increasingly difficult QA regimes:

1. The bank begins with a compact seed bank (B0) containing general retrieval and grounding skills.

2. It expands through four rounds (B1 to B4), where each round focuses on specific question types, such as single-hop data, HOTPOTQA, 2WikiMultiHopQA, and MUSIQUE.

3. In each round, the bank is updated by proposing additions and targeted refinements using GPT-5.4 under round-specific constraints. Accepted updates are merged into the next bank (Bt+1).

This staged evolution ensures that simpler skills like direct lookup are stabilized first before introducing more complex skills such as bridge-entity search, relation-chain decomposition, and reconstructed-chain verification. The final B4 is a cumulative skill prior covering single-hop lookup, two-hop bridge reasoning, compositional comparison, and longer decomposition.

Data Sampling and Teacher Synthesis

Supervision data is constructed in two steps. First, coverage-oriented training pools are built from benchmarks like NQ, TriviaQA, HotpotQA, 2WikiMultiHopQA, and MuSiQue. Rather than uniform sampling, examples are grouped by coarse reasoning signature, and GPT-5.4 annotates representative groups with reasoning and skill-demand labels.

Second, a trajectory manifest is built by mixing these sampled training examples with failure replay from earlier evaluation traces. GPT-5.4 acts as a teacher under the current SkillBank, rolling out the select-read-act interaction to predict a structured next action including a primary skill, optional support skills, an action type, a query when retrieval is needed, and a checkpoint describing the currently resolved intermediate state. Only trajectories that pass executable validation—including answer correctness and legal skill identifiers—form the supervision pool.

Key Contributions and Results

The main contributions include:

"We propose a skill-conditioned search-tool interface that improves query formulation by making the model choose a reusable search strategy before each tool call, addressing the common failure of issuing broad or copied multi-hop queries."

We introduce an evolvable Search SkillBank that is updated through failure analysis, add/refine operations, and trajectory reconstruction, providing a reusable tool-use prior that can extend beyond the QA setting.

Experiments show that SEARCHSKILL "improves exact match on knowledge-intensive QA benchmarks and yields better retrieval behavior, including fewer copied first queries, more atomic hop-focused queries, and more correct answers within a small search budget. The results demonstrate that the two-stage SFT recipe can obtain performance close to or stronger than large-scale RL-based search agents while avoiding the high rollout and optimization cost of training search behavior primarily through RL. Furthermore, the frozen SkillBank benefits proprietary models as an external planning guide."

Query Planning Diagnostics

The framework addresses query quality failure by explicitly teaching structured decomposition. Diagnostics show that SEARCHSKILL improves metrics such as First Query Copy (↓), increases the Atomic Hop Query (↑), and boosts "Correct@≤ 3 Searches (↑).

Here are specific improvements that can be made to AI systems based on the SEARCHSKILL framework, and what these improved systems can achieve:

)1. Transition from Undifferentiated Search to Explicit Query Planning (The Core Improvement)

Currently, many search-augmented LLMs treat search as a monolithic action. The improvement is implementing the explicit skill selection mechanism where the model must choose a reusable strategy before every tool call.

• The improved system will emit a structured tag:

bridge-entity-search followed by the query string.

• This forces the LLM to decompose complex, multi-hop questions into sequential, logical steps (e.g., identifying an intermediate entity first).

• It prevents query copying or overly broad initial searches that waste retrieval budget.

)2. Automated Discovery and Evolution of Search Strategies (The SkillBank Mechanism)

Instead of relying on human prompt engineering to define search logic, the system will maintain and evolve a dynamic SkillBank based on observed failure patterns.

• The system will undergo an iterative refinement loop:

1. Roll out current policy.

2. Analyze where queries failed (e.g., Query copying, Wrong entity selection).

3. Update the SkillBank by adding or refining skills that address these failures (e.g., creating a new skill for bridge-disambiguate-then-hop).

4. Reconstruct affected trajectories under the new bank structure before retraining.

• This creates a self-improving search prior that adapts to novel reasoning patterns and domain shifts without requiring full model retraining for every change in search behavior.

)3. Structured Reasoning Trajectories (The Two-Stage SFT Protocol)

By using a two-stage Supervised Fine-Tuning (SFT) recipe, the system moves from simply learning how to search to learning how to plan and execute.

• The first stage teaches the model the basic format: emit skill tag, then search or answer.

• The second stage teaches the model to condition its next action on the specific contents of a selected SkillBank card.

• This results in a policy that is robust against prompt variations because it learns to route through an explicit index (the SkillBank) rather than relying on implicit instruction following.

)4. Enhanced Multi-Hop and Compositional Reasoning (Specific Skill Application)

The system will explicitly employ specialized skills tailored for complex reasoning tasks, which are often missed by general models.

• For multi-hop questions, the model will use:

• relation-chain-decomposition: To break down long relational chains into atomic steps.

• bridge-comparison-planning: To resolve hidden entities on both sides of a comparison before retrieving endpoint attributes.

• re-anchored-long-hop-decomposition: To manage the carryforward of intermediate entities across multiple hops, preventing drift.

)5. Improved Evidence Utilization and Grounding (The Final Check)

The system will integrate explicit verification steps into the search protocol to ensure high fidelity in the final answer.

• The system will use:

• answer-grounding-checkverbatim-evidence-span: To copy the shortest faithful answer span directly from evidence, ensuring the final output is strictly supported.

• grounding/verification conflict-check: To run targeted disambiguation searches if retrieved evidence surfaces competing entities or inconsistent checkpoints, preventing errors caused by noisy top-3 retrieval results.

)6. Robustness Across Model Scales and Sources (Transferability)

The framework is designed to be highly transferable:

• The system can take an evolved SkillBank from one model (e.g., a 7B Instruct model) and use it as an external planning guide for a different, frozen, proprietary model (e.g., Claude or Gemini).

• This allows proprietary systems to benefit from the learned search skills without requiring expensive fine-tuning on their specific data distribution.

Teaching language models to use search tools is not only a question of whether they search, but also of whether they issue good queries. This is especially important in open-domain question answering, where broad or copied queries often waste retrieval budget and derail later reasoning. We propose, a framework that makes query planning explicit through reusable search skills. At each step, the model first selects a skill, then generates a search or answer action conditioned on the selected skill card. The skill inventory itself is not fixed: SearchSkill maintains an evolving SkillBank, expands or refines it from recurrent failure patterns, and reconstructs affected trajectories before supervised training. The resulting two-stage SFT recipe aligns training with the inference-time protocol of skill selection followed by skill-grounded execution. Across open-source and closed-source models, SearchSkill improves exact match on knowledge-intensive QA benchmarks and yields better retrieval behavior, including fewer copied first queries, more atomic hop-focused queries, and more correct answers within a small search budget. These results suggest that explicit skill-conditioned query planning is a lightweight alternative to treating search as an undifferentiated action.