What Keeps Agent Skills from Being Reusable? Evidence from 138K SKILL.md Files
Chi Zhang, Yimin Liu, Xinze Chen, Ping Ji
cs.AI, cs.CR
Submitted: 2026-08-09
Updated: 2026-08-11
Comments: 11 pages, 4 figures
Code: https://github.com/agent-sh/agnix
License: http://creativecommons.org/licenses/by/4.0/
The gist: Under the current standard, Agent Skills are SKILL.md files that combine instructions with supporting resources, enabling Large Language Model (LLM) agents to reuse procedures beyond a single
Terminology
Abstract
Under the current standard, Agent Skills are SKILL.md files that combine instructions with supporting resources, enabling Large Language Model (LLM) agents to reuse procedures beyond a single conversation. Yet many public skills appear to originate from a single task, repository, or conversation, even when they are shared as reusable components. We analyze this gap across 138,133 public SKILL.md files from 20,556 repositories using a two-tier defect taxonomy grounded in the official specification and best-practice guidance. We find that 91.8% of skills contain at least one detected defect, with stable estimates across lenient and strict thresholds (88.8-94.6%). The dominant failures are ordinary packaging problems rather than exotic attacks: weak routing metadata, bloated or non-actionable bodies, and poor resource organization. A deterministic routing stress test over 20,000 skills shows the functional impact: skills with valid routing metadata are retrieved more reliably from startup descriptions than skills with routing defects. Defect rates vary by platform and provenance: specification-aware skills contain fewer defects, while AI-marked skills show more safety and portability problems. Lightweight enforcement and repair experiments support a quality-assured generation workflow combining spec-aware prompting, lightweight linting, automated repair, and safety gating.
Sources
- SkillReducer: Optimizing LLM Agent Skills for Token Efficiency
- IH-Challenge: A Training Dataset to Improve Instruction Hierarchy on Frontier LLMs
- SkillsBench: Benchmarking How Well Agent Skills Work Across Diverse Tasks
- ClawsBench: Evaluating Capability and Safety of LLM Productivity Agents in Simulated Workspaces
- How Well Do Agentic Skills Work in the Wild: Benchmarking LLM Skill Usage in Realistic Settings
- Skill-Inject: Measuring Agent Vulnerability to Skill File Attacks
- The Instruction Hierarchy: Training LLMs to Prioritize Privileged Instructions
- "Elementary, My Dear Watson." Detecting Malicious Skills via Neuro-Symbolic Reasoning across Heterogeneous Artifacts
Related papers
- MAVEN-T: Reinforced Heterogeneous Distillation for Real-Time Multi-Agent Trajectory Prediction
- Model Discovery Agent: LLM-assisted Bayesian experiment design for data-efficient discovery of mechanistic world models
- The Clinician's Veto: Navigating Trust, Liability, and Uncertainty in Autonomous AI Prescribing
- MindHelper: Closed-Loop Embodied Mental-State Reasoning for Precision Intervention
- Incumbent Advantage: Brand Bias and Cognitive Manipulation Dynamics in LLM Recommendation Systems
- VSAL: A Vision Solver with Adaptive Layouts for Graph Property Detection