Learning Globally Reusable Skills for Coding Agents

The paper addresses the challenge of automated skill evolution for Large Language Model (LLM) coding agents. While such agents can automate software engineering tasks like test generation and defect analysis, they are often limited by static designs — fixed prompts, tool policies, and manually curated skills become insufficient as tasks vary across programming languages, project conventions, and failure modes. Skill evolution is appealing because it extracts and refines reusable abstractions of reasoning and action patterns from execution histories, enabling agents to continuously adapt their behavior through very lightweight updates.

However, the authors identify a fundamental limitation in existing approaches: they largely treat skill evolution as a sequence of local updates, whereas both the generation and integration stages fundamentally require a global perspective. Locally derived updates may violate constraints assumed by other skills, and locally successful updates may be overly specialized to the observed traces and can introduce regressions when reused in future tasks. As the skill bank grows, such locally optimized updates can accumulate and gradually degrade the overall quality and coherence of the agent’s skill set.

The paper formulates two key challenges:

• Challenge 1: Compatibility. A coding agent operates as a tightly coupled system in which the output of one skill often serves as the input assumption of another. Updates made in isolation may inadvertently alter assumptions relied upon by other skills, and relationships among skills (complementary or conflicting) are rarely represented explicitly.

• Challenge 2: Generalization. Updates derived from individual traces are often highly specific to the contexts in which they were generated, and different traces may encode the same underlying capability gap through different low-level observations. Directly merging such updates risks introducing overly specific and fragmented skills into the skill bank.

The authors propose GSE, a Globalized Skill Evolution framework that optimizes the skill bank as an interconnected and reusable system, rather than applying isolated updates. The workflow comprises three stages: trace collection, evolution proposal generation, and skill generalization.

GSE first instruments the agent to record all actions, including internal reasoning steps, invoked tools, command parameters, environment feedback, and the final task outcome. Because raw traces are extremely long and verbose, GSE employs trace simplification to filter out token-heavy command outputs that do not contribute to high-level causal analysis — removing outputs from file-search commands, raw file contents, and low-value logs — while preserving the exact commands executed, the targeted file paths, key error or warning messages, and the agent’s internal reasoning blocks.

Given a simplified trace of a failure, an update agent diagnoses the capability deficiency underlying the observed failure through a hierarchical diagnosis: navigation failure (deficient context exploration), reasoning failure (incorrect codebase reasoning), or execution failure (improper execution behavior). The diagnosis distinguishes between general capability enhancement (improving high-level behaviors like context exploration and validation strategies) and reusable problem pattern (capturing recurring code/task characteristics like bug-triggering patterns, library constraints, or framework-specific strategies).

Skill Relation Graph (SRG): To address Challenge 1, GSE maintains a Skill Relation Graph (SRG), which explicitly represents and evolves the dependencies among skills. Formally, it is a directed graph G = (V, E), where nodes represent skills and edges represent semantic relationships of three types:

• dependency: skill vj relies on assumptions, or results produced by skill vi

• co-usage: skills are complementary and perform better when applied together

• conflict: skills provide incompatible guidance or encode contradictory assumptions, and therefore should not be activated simultaneously

Given a proposed update, GSE identifies affected skills by examining all neighboring skills connected to the updated skill in the SRG, plus candidate related skills retrieved via metadata and semantic analysis. If a modification changes assumptions, intermediate outputs, applicability conditions, or interaction patterns, GSE generates corresponding auxiliary evolution proposals that jointly evolve both the affected skills and their relationships. This transforms skill evolution from a sequence of independent local updates into a globalized optimization process over the entire skill bank.

DSL-Based Evolution Proposals: Each proposal is strictly structured using a specialized Domain-Specific Language (DSL) specifying the target skill, the edit operation, the concrete content to be updated, the rationale, the expected behavior after the update, and the relation changes. Before generalization, GSE performs local verification: it applies proposed changes to a temporary candidate skill bank and replays the originating failure case, discarding proposals that fail to resolve it.

To address Challenge 2, GSE performs two stages: cluster-based skill consolidation and replay-driven validation (summarized in Algorithm 1).

Skill Consolidation: Proposals modifying existing skills are grouped using the affected skill set as the clustering key, since failures requiring changes to the same skills likely expose similar capability deficiencies. Proposals introducing new skills are clustered according to semantic similarity of their rationale and expected behavioral effects. A merge agent then analyzes the proposals within each cluster to extract their common patterns, remove case-specific details, and resolve redundant or conflicting updates, producing generalized skill abstractions.

Replay-Driven Validation: For each candidate skill bank, GSE replays a set of relevant historical cases — historical cases where the affected skills were previously activated for existing-skill modifications, or the originating cases for new skills. A candidate skill bank is accepted only if it maintains or improves performance across the replay cases, ensuring the resulting skills generalize beyond individual failures and prevents overfitted or regressive updates from entering the skill bank.

• Bug-revealing test generation: Built on Multi-SWE-Bench's Java portion, yielding 108 buggy focal methods spanning 9 actively maintained Java projects. The focal methods are non-trivial: 4∼206 lines of code, on average 6 branches, 8 direct callees, and references to symbols defined in 5 external files.

• False-positive bug report filtering: Constructed in collaboration with an international IT company (ByteDance) as IndustrialBugs, containing 500 bug reports from 8 production repositories, including 132 confirmed bugs and 368 false alarms. Reports span 3 internal business lines, repositories range from 12.8∼186.3 KLoC, and cover diverse bug types including security issues, concurrency errors, data-flow/integration bugs, and functional correctness bugs.

A one-project-held-out evaluation protocol is used: all cases from one project are held out for evaluation, while cases from the remaining projects are used to evolve the skill bank, ensuring no evaluated case is used during evolution and testing cross-project generalization.

Two state-of-the-art coding agents serve as base harnesses: OpenHands and mini-SWE-agent (extended with a lightweight skill-loading interface). All techniques use the same tools, execution environment, base LLM, and per-task budget.

Compared techniques include:

• Human Skills: 17 skills authored by 6 senior software-quality engineers from our partner company, each with 6∼10 years of engineering experience

• Live-SWE-agent: a state-of-the-art evolution framework that automatically evolves software engineering agent scaffolds adapted to skill evolution

• Trace2Skill: a state-of-the-art evolution framework that leverages multiple analyst sub-agents to propose skill updates and merges them at once into one skill bank

• GSE, plus vanilla base agents without skills

Both tasks use precision, recall, and F1-score. For test generation, outcomes are classified as True Positive (test fails due to ground-truth bug), False Positive (test fails for unrelated reasons), or False Negative (no bug revealed). For filtering, predictions are compared against developer-confirmed labels.

GSE consistently outperforms all baselines on both OpenHands and mini-SWE-agent. On OpenHands, GSE achieves precision 0.35, recall 0.28, F1 0.31, outperforming the strongest baseline scores of 0.33, 0.17, and 0.22 by 6.1%, 64.7%, and 40.9%, respectively. On mini-SWE-agent, GSE achieves precision 0.55, recall 0.29, F1 0.38, improving over the strongest baselines by 27.9%, 31.8%, and 31.0%.

Human-written skills provide limited benefits and do not consistently improve different base agents — they help the weaker OpenHands agent (F1 from 0.08 to 0.16) but slightly hurt mini-SWE-agent (F1 from 0.28 to 0.26). Existing evolution methods also provide limited gains because both of them perform evolution through local updates, making it difficult to coordinate them with existing skills or verify whether they remain useful beyond the cases that produced them.

A case study illustrates the difference: local updates learn isolated rules (e.g., minimize collection size for collection bugs, preserve states for permission bugs) that overfit and conflict, causing failure on a novel cache-state bug. GSE instead abstracts the fragmented local updates into a unified, higher-level skill: Determine Bug Triggering Mechanisms, which classifies defects as either value-triggered or state-triggered, and uses the SRG to establish a formal dependency relationship between this newly generalized diagnostic skill and the existing Construct Stateful/Stateless Tests skills, enabling success on the unseen bug. The RQ1 summary states GSE improves F1-score by 31.0%∼93.8%.

On OpenHands, GSE achieves precision 0.55, recall 0.95, F1 0.70, substantially outperforming the strongest baselines, which achieve only 0.29 precision, 0.84 recall, and 0.43 F1-score. On mini-SWE-agent, GSE improves precision from 0.26 to 0.30, recall from 0.84 to 0.97, and F1 from 0.39 to 0.46. Across both agents, GSE improves precision by 15.4%∼96.4%, recall by 13.1%∼19.8%, and F1 by 17.9%∼66.7%.

Human skills provide only modest improvements because false-positive filtering in enterprise environments requires reasoning about complex business logic, API contracts, and project-specific engineering conventions. Existing evolution methods improve recall but remain limited in precision (precision ranges only 0.25 to 0.29) because both methods rely on local updates and do not explicitly verify whether the learned behaviors generalize across cases. A case study shows that Live-SWE-agent and Trace2Skill learned multiple project-specific nullability skills with conflicting guidance, whereas GSE generalized them into a higher-level capability for reasoning about nullability guarantees and propagated the resulting abstraction to related bug-validation skills through the SRG.

Two variants are tested on OpenHands: GSEw/o graph (removes SRG, disables relationship evolution and propagation) and GSEw/o gen (removes generalization; proposals merged after local validation only). Removing the SRG drops F1 from 0.31 to 0.27 on bug-revealing test generation and from 0.70 to 0.59 on false-positive filtering. Removing generalization drops F1 from 0.31 to 0.27 on test generation and from 0.70 to 0.54 on false-positive filtering. The paper concludes both relationship-aware evolution and skill generalization are essential for maintaining an effective skill bank.

GSE's evolution process consumes an average of 401.55K tokens per case, compared with 357.64K for Trace2Skill, an increase of 12.28%. For downstream execution, GSE-enhanced agents consume 593.21K tokens per case, requiring fewer tokens than all other methods, reducing token consumption by 4.4%, 3.3%, and 4.3% compared with human skills, Live-SWE-agent, and Trace2Skill, respectively. The paper notes the additional prompt budget introduced by evolved skills can be offset by more effective reasoning and exploration strategies.

GSE is instantiated on the industrial partner's proprietary bug-triage agent for false-positive filtering. This agent is specifically designed for enterprise bug analysis and incorporates customized tools, proprietary APIs, and domain-specific prompting strategies, with base performance already exceeding both open-source agents (F1 0.43 vs. 0.37/0.36). Results show GSE achieves precision 0.56, recall 0.97, F1 0.71, improving precision by 93.1%, recall by 16.9%, and F1 by 65.1% over the base agent, and by 86.7%, 16.9%, and 61.4% over human-written skills. This demonstrates the benefits of globalized skill evolution are not limited to relatively lightweight open-source agents.

The paper addresses external validity via evaluation across two open-source agents plus an industrial deployment, two tasks, and both Java and Go. Internal validity is addressed by using officially maintained OpenHands and mini-SWE-agent as base agents and strictly followed the procedures described in their papers for baselines. Construct validity concerns are mitigated by setting the LLM temperature to zero for all techniques, using the one-project-held-out protocol, confirming none of the generated tests align with the reference tests, and having the industrial partner confirm proprietary codebases are excluded from the training corpora of public LLMs.

The paper situates GSE within two research lines: skills for LLM agents (noting SkillsBench finds well-designed skills improve agents, while SWE-Skills-Bench reports software engineering skills provide uneven gains and may even degrade performance) and agent evolution (including parameter-update methods like ReTool, WebRL, and OpenClaw-RL, and externalized-experience methods like AFlow, Reflexion, ExpeL, ReasoningBank, Memento, and Trace2Skill). The authors conclude that existing methods typically perform skill evolution through isolated local updates and do not consider their global impact, whereas GSE performs globalized skill evolution by jointly evolving inter-skill relationships and generalizing locally effective updates into reusable skills.

In conclusion, GSE formulates skill evolution as a global optimization problem over an interconnected and reusable skill bank, co-evolving inter-skill relationships through the SRG and generalizing local proposals through cluster-based consolidation and replay-driven validation. The evaluation shows GSE consistently outperforms existing techniques across diverse scenarios, improving precision by 6.1%∼34.1% and recall by 31.8%∼180.0% for test generation, and precision by 15.4%∼96.4% and recall by 13.1%∼19.8% for false-positive filtering, confirming the effectiveness of globalized skill evolution.

Based on the GSE framework, I can improve AI systems by implementing globalized skill evolution instead of isolated local updates. The improved system can:

1. Maintain a Skill Relation Graph (SRG) that explicitly tracks dependency, co-usage, and conflict relationships between skills. When updating one skill, the system automatically identifies affected neighboring skills and co-evolves them, preventing incompatible assumptions and conflicting guidance.

2. Generate evolution proposals through a structured DSL that specifies the target skill, edit operation, concrete content, rationale, expected behavior, and relation changes. This makes skill updates verifiable and reversible, reducing the risk of introducing regressions.

3. Perform cluster-based skill consolidation to abstract fragmented, case-specific updates into generalized, reusable skills. Instead of storing many narrow rules like minimize collection size or preserve permission state, the system learns higher-level capabilities such as classify defects as value-triggered vs. state-triggered.

4. Run replay-driven validation before accepting any skill-bank change. The system replays historical failure cases and relevant past tasks, accepting an update only if it maintains or improves performance across the replay set. This prevents overfitting to a single trace and ensures the skill generalizes.

5. Use hierarchical failure diagnosis that distinguishes navigation, reasoning, and execution failures, and separates general capability enhancements from reusable problem patterns. This improves the precision of the learned skills.

6. Simplify execution traces by removing token-heavy command outputs while preserving exact commands, file paths, key errors/warnings, and internal reasoning. This makes skill extraction more efficient and causally focused.

7. Propagate skill abstractions through the relation graph to related validation or test-construction skills, enabling the system to apply newly generalized knowledge consistently across the whole skill bank.

As a result, the improved AI system can:

• Achieve substantially higher F1 scores on bug-revealing test generation and false-positive bug report filtering, with gains of 31–94% and 18–67% respectively over state-of-the-art evolution methods.

• Maintain performance across different coding agents (OpenHands, mini-SWE-agent) and proprietary enterprise agents, showing cross-agent generalizability.

• Reduce downstream token consumption by 3–4% because evolved skills lead to more effective reasoning and exploration, offsetting the prompt overhead of larger skill banks.

• Avoid skill conflicts and overfitting — for example, instead of conflicting nullability rules, the system learns a unified nullability-guarantee reasoning skill and propagates it to related bug-validation skills.

• Continuously adapt to new projects without manual skill authoring, using a held-out project protocol to confirm that evaluated cases were never used during evolution.

Automated skill evolution enables Large Language Model (LLM) agents to continuously improve without expensive retraining. However, existing approaches typically treat skill evolution as a sequence of local updates, overlooking relationships among skills and often producing overfitted skill updates that fail to generalize across tasks. We propose GSE, a globalized skill evolution framework that jointly optimizes skill compatibility and skill generalization. To preserve consistency across the skill bank, GSE maintains a Skill Relation Graph (SRG) that explicitly models and co-evolves inter-skill relationships. To improve generalization, GSE performs cluster-based skill consolidation to abstract reusable capabilities from local updates and employs replay-driven verification to prevent overfitting and behavioral regressions. We evaluate GSE on two representative software engineering tasks: bug-revealing test generation and false-positive bug report filtering. Across two state-of-the-art coding agents, OpenHands and mini-SWE-agent, GSE consistently achieves the best precision, recall, and F1-score. Compared with existing evolution techniques, GSE improves precision and recall by 6.1% 34.1% and 31.8% 180.0% for test generation, and by 15.4% 96.4% and 13.1% 19.8% for false-positive filtering. Deployment on an internal industrial agent further yields a 61.4% improvement in F1-score, demonstrating the effectiveness and generalizability of GSE for evolving effective skills.