Rethinking Self-Evolution: A Constrained Exploration-Exploitation Process for Mitigating Skill Overfitting

arXiv:2607.26643 · cs.AI, cs.LG · Submitted 2026-08-19 · Read on arXiv

Hongqiang Lin, Chao Liu, Xiaofan Bai, Xuan Jin, Yuhong Li, Nenggan Zheng, Xipeng Cao

cs.AI, cs.LG

Submitted: 2026-08-19

Updated: 2026-08-20

Terminology

• Benchmark — 52× in this paper · explained in 2 episodes
• Accuracy — 21× in this paper · explained in 1 episode
• Verification — 17× in this paper · explained in 2 episodes
• Core — 13× in this paper · explained in 2 episodes
• Agent Skills — 9× in this paper · explained in 1 episode
• Generalization — 8× in this paper · explained in 13 episodes
• Trace — 8× in this paper · explained in 2 episodes
• Ablation — 5× in this paper · explained in 1 episode
• Adaptation — 3× in this paper · explained in 1 episode
• Benchmarking — 3× in this paper · explained in 3 episodes
• Clustering — 3× in this paper · explained in 2 episodes
• Large Language Model (LLM) — 3× in this paper · explained in 3 episodes
• Reinforcement Learning — 3× in this paper · explained in 4 episodes
• Abstraction — 2× in this paper · explained in 1 episode
• Agentic System — 2× in this paper · explained in 2 episodes
• Consistency — 2× in this paper · explained in 3 episodes
• Cross-Model Transfer — 2× in this paper · explained in 2 episodes
• Hallucination — 2× in this paper · explained in 3 episodes
• Verifier — 2× in this paper · explained in 2 episodes
• AI Agents — 1× in this paper · explained in 2 episodes
• Ablation Studies — 1× in this paper · explained in 1 episode
• Ablation study — 1× in this paper · explained in 1 episode
• Action Space — 1× in this paper · explained in 1 episode
• Adam — 1× in this paper · explained in 2 episodes
• Agentic Evaluation — 1× in this paper · explained in 1 episode
• Agentic Task — 1× in this paper · explained in 1 episode
• Agentic Tasks — 1× in this paper · explained in 1 episode
• Aggregation — 1× in this paper · explained in 1 episode
• Analyzer — 1× in this paper · explained in 1 episode
• Continual Learning — 1× in this paper · explained in 2 episodes
• Failure Modes — 1× in this paper · explained in 3 episodes
• Initial Data — 1× in this paper · explained in 2 episodes
• Multimodal Reasoning — 1× in this paper · explained in 5 episodes
• Robustness — 1× in this paper · explained in 5 episodes
• Scaling Laws — 1× in this paper · explained in 4 episodes
• Self-Evolving Agents — 1× in this paper · explained in 2 episodes
• Thinking mode — 1× in this paper · explained in 2 episodes

Sources

• The Scaling Laws of Skills in LLM Agent Systems
• A Comprehensive Survey of Self-Evolving AI Agents: A New Paradigm Bridging Foundation Models and Lifelong Agentic Systems
• LiveMathematicianBench: A Live Benchmark for Research-Level Mathematical Reasoning with Proof Sketches
• A Survey of Agent Memory in the Second Half: Towards Self-Evolving and Long-Horizon Agents
• From Raw Experience to Skill Consumption: A Systematic Study of Model-Generated Agent Skills
• SoK: Agentic Skills -- Beyond Tool Use in LLM Agents
• Organizing, Orchestrating, and Benchmarking Agent Skills at Ecosystem Scale
• MUSE-Autoskill: Self-Evolving Agents via Skill Creation, Memory, Management, and Evaluation
• SkillForge: Forging Domain-Specific, Self-Evolving Agent Skills in Cloud Technical Support
• SkillClaw: Let Skills Evolve Collectively with Agentic Evolver
• Trace2Skill: Distill Trajectory-Local Lessons into Transferable Agent Skills
• SkillOS: Learning Skill Curation for Self-Evolving Agents
• AutoRefine: Compiling Trajectories into Validated Typed Agent Artifacts
• SkillOpt-Lite: Better and Faster Agent Self-evolution via One Line of Vibe
• Skill-R1: Agent Skill Evolution via Reinforcement Learning
• SkillX: Automatically Constructing Skill Knowledge Bases for Agents
• SkillGrad: Optimizing Agent Skills Like Gradient Descent
• Evo-Memory: Benchmarking LLM Agent Test-time Learning with Self-Evolving Memory
• SkillRL: Evolving Agents via Recursive Skill-Augmented Reinforcement Learning
• Agent Skills for Large Language Models: Architecture, Acquisition, Security, and the Path Forward

Related papers

• MAVEN-T: Reinforced Heterogeneous Distillation for Real-Time Multi-Agent Trajectory Prediction
• Model Discovery Agent: LLM-assisted Bayesian experiment design for data-efficient discovery of mechanistic world models
• The Clinician's Veto: Navigating Trust, Liability, and Uncertainty in Autonomous AI Prescribing
• MindHelper: Closed-Loop Embodied Mental-State Reasoning for Precision Intervention
• Incumbent Advantage: Brand Bias and Cognitive Manipulation Dynamics in LLM Recommendation Systems
• VSAL: A Vision Solver with Adaptive Layouts for Graph Property Detection