SkillCorpus: Consolidating and Evaluating the Open Skill Ecosystem for Real-World LLM Agents

arXiv:2607.15557 · cs.CL · Submitted 2026-07-17 · Read on arXiv

cs.CL

Submitted: 2026-07-17

Updated: 2026-08-26

Code: https://github.com/EverMind-AI/SkillCorpus

License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/

Terminology

• Benchmark — 54× in this paper · explained in 2 episodes
• Robustness — 18× in this paper · explained in 5 episodes
• Verifier — 10× in this paper · explained in 2 episodes
• LLM judge — 7× in this paper · explained in 2 episodes
• Agent Skills — 6× in this paper · explained in 1 episode
• Trace — 6× in this paper · explained in 2 episodes
• Ablation — 5× in this paper · explained in 1 episode
• Alignment — 5× in this paper · explained in 1 episode
• Core — 5× in this paper · explained in 2 episodes
• Procedural Knowledge — 5× in this paper · explained in 2 episodes
• Benchmarking — 4× in this paper · explained in 4 episodes
• Aggregation — 3× in this paper · explained in 1 episode
• Agent Harness — 2× in this paper · explained in 2 episodes
• Consistency — 2× in this paper · explained in 3 episodes
• Embeddings — 2× in this paper · explained in 2 episodes
• Model Context Protocol — 2× in this paper
• Abundance — 1× in this paper · explained in 1 episode
• Adaptive Selection — 1× in this paper · explained in 1 episode
• Agentic Coding — 1× in this paper · explained in 1 episode
• Calibration — 1× in this paper · explained in 7 episodes
• Clustering — 1× in this paper · explained in 2 episodes
• Contrastive Learning — 1× in this paper · explained in 5 episodes
• Data Provenance — 1× in this paper · explained in 2 episodes
• Fine-Tuning — 1× in this paper · explained in 5 episodes
• Foundation Model — 1× in this paper · explained in 3 episodes
• Model Context Protocol (MCP) — 1× in this paper · explained in 8 episodes
• Ontology — 1× in this paper · explained in 2 episodes
• Provenance — 1× in this paper · explained in 4 episodes
• Reinforcement Learning — 1× in this paper · explained in 4 episodes
• Scaffolding — 1× in this paper · explained in 3 episodes
• Scaling Laws — 1× in this paper · explained in 4 episodes
• Verification — 1× in this paper · explained in 2 episodes

Sources

• The Scaling Laws of Skills in LLM Agent Systems
• SkillRet: A Large-Scale Benchmark for Skill Retrieval in LLM Agents
• Model Context Protocol (MCP): Landscape, Security Threats, and Future Research Directions
• Deduplicating Training Data Makes Language Models Better
• Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks
• SkillFlow: Scalable and Efficient Agent Skill Retrieval System
• Organizing, Orchestrating, and Benchmarking Agent Skills at Ecosystem Scale
• SkillsBench: Benchmarking How Well Agent Skills Work Across Diverse Tasks
• SkillNet: Create, Evaluate, and Connect AI Skills
• Position: Agentic Evolution is the Path to Evolving LLMs
• A Large-Scale Evolvable Dataset for Model Context Protocol Ecosystem and Security Analysis
• Agent Skills: A Data-Driven Analysis of Claude Skills for Extending Large Language Model Functionality
• Graph-of-Skills: Dependency-Aware Structural Retrieval for Massive Agent Skills
• How Well Do Agentic Skills Work in the Wild: Benchmarking LLM Skill Usage in Realistic Settings
• SkillRAE: Agent Skill-Based Context Compilation for Retrieval-Augmented Execution
• Terminal-Bench: Benchmarking Agents on Hard, Realistic Tasks in Command Line Interfaces
• Gorilla: Large Language Model Connected with Massive APIs
• Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena
• SkillRouter: Skill Routing for LLM Agents at Scale
• LIMA: Less Is More for Alignment

Related papers

• Exploring Solution Divergence and Its Effect on Large Language Model Problem Solving
• Ishigaki-IDS-Bench: A Benchmark for Generating Information Delivery Specification from BIM Information Requirements
• Subliminal Steering: Stronger Encoding of Hidden Signals
• MedStruct-S: A Benchmark for Key Discovery, Key-Conditioned QA and Semi-Structured Extraction from OCR Clinical Reports
• The End of Transformers? On Challenging Attention and the Rise of Sub-Quadratic Architectures
• Untangling the Mechanisms of Misleading Context in Medical Question Answering