SafeEvolve: Harness-Policy Co-Evolution from Agent Experience for Safety Alignment

arXiv:2609.02786 · cs.AI, cs.CR · Submitted 2026-09-02 · Read on arXiv

cs.AI, cs.CR

Submitted: 2026-09-02

Updated: 2026-09-02

Code: https://github.com/MaoPopovich/SafeEvolve

Terminology

• Verifier — 33× in this paper · explained in 2 episodes
• Benchmark — 25× in this paper · explained in 2 episodes
• Alignment — 20× in this paper · explained in 1 episode
• GRPO — 14× in this paper · explained in 6 episodes
• Reinforcement Learning — 8× in this paper · explained in 4 episodes
• Ablation — 7× in this paper · explained in 1 episode
• Distillation — 5× in this paper · explained in 2 episodes
• Generalization — 5× in this paper · explained in 13 episodes
• Robustness — 5× in this paper · explained in 5 episodes
• Self-distillation — 3× in this paper · explained in 6 episodes
• Trace — 3× in this paper · explained in 2 episodes
• Agent Harness — 2× in this paper · explained in 2 episodes
• Alignment Methods — 2× in this paper · explained in 1 episode
• Attack Success Rate (ASR) — 2× in this paper · explained in 2 episodes
• Direct Preference Optimization — 2× in this paper
• Failure Modes — 2× in this paper · explained in 3 episodes
• Fine-Tuning — 2× in this paper · explained in 5 episodes
• KL Regularization — 2× in this paper · explained in 2 episodes
• Regularization — 2× in this paper · explained in 3 episodes
• Reward Hacking — 2× in this paper · explained in 8 episodes
• Supervised Fine-Tuning — 2× in this paper
• Supervised Fine-Tuning (SFT) — 2× in this paper · explained in 8 episodes
• Verification — 2× in this paper · explained in 2 episodes
• Ablation Studies — 1× in this paper · explained in 1 episode
• Action Space — 1× in this paper · explained in 1 episode
• Agentic System — 1× in this paper · explained in 2 episodes
• Agentic Task — 1× in this paper · explained in 1 episode
• Agentic Tasks — 1× in this paper · explained in 1 episode
• Benchmarking — 1× in this paper · explained in 3 episodes
• Catastrophic Forgetting — 1× in this paper · explained in 5 episodes
• Core — 1× in this paper · explained in 2 episodes
• Direct Preference Optimization (DPO) — 1× in this paper · explained in 9 episodes
• Grounding — 1× in this paper · explained in 3 episodes
• Group Relative Policy Optimization — 1× in this paper
• Group Relative Policy Optimization (GRPO) — 1× in this paper · explained in 5 episodes
• Large Language Model (LLM) — 1× in this paper · explained in 3 episodes
• Objective Function — 1× in this paper · explained in 2 episodes
• On-Policy Distillation — 1× in this paper · explained in 3 episodes
• Provenance — 1× in this paper · explained in 3 episodes
• Pruning — 1× in this paper · explained in 4 episodes
• Self-Evolving Agents — 1× in this paper · explained in 2 episodes

Sources

• Constitutional AI: Harmlessness from AI Feedback
• EvoTrainer: Co-Evolving LLM Policies and Training Harnesses for Autonomous Agentic Reinforcement Learning
• HarnessForge: Joint Harness and Policy Evolution for Adaptive Agent Systems
• Meta-SecAlign: Training LLMs against Prompt Injection for Robust Agents
• EvoRubrics: Dynamic Rubrics as Rewards via Adversarial Co-Evolution for LLM Reinforcement Learning
• Reducing the Safety Tax in LLM Safety Alignment with On-Policy Self-Distillation
• ReSkill: Reconciling Skill Creation with Policy Optimization in Agentic RL
• RvB: Automating AI System Hardening via Iterative Red-Blue Games
• Recursive Harness Self-Improvement
• SafeSteer: Localized On-Policy Distillation for Efficient Safety Alignment
• AgentDyn: Are Your Agent Security Defenses Deployable in Real-World Dynamic Environments?
• Agentic Harness Engineering: Observability-Driven Automatic Evolution of Coding-Agent Harnesses
• Harness Updating Is Not Harness Benefit: Disentangling Evolution Capabilities in Self-Evolving LLM Agents
• Auditing Agent Harness Safety
• AgentDoG 1.5: A Lightweight and Scalable Alignment Framework for AI Agent Safety and Security
• AgentDoG: A Diagnostic Guardrail Framework for AI Agent Safety and Security
• Self-Distilled Agentic Reinforcement Learning
• Harness-Aware Self-Evolving: Co-Evolving Model Weights, Harness, and Task Solutions
• EvoHarness-RL: Learning Runtime Harness Coordination for Self-Evolving Agents
• SHE: Trajectory-driven Safety Harness Evolution for LLM Agents

Related papers

• MAVEN-T: Reinforced Heterogeneous Distillation for Real-Time Multi-Agent Trajectory Prediction
• Model Discovery Agent: LLM-assisted Bayesian experiment design for data-efficient discovery of mechanistic world models
• The Clinician's Veto: Navigating Trust, Liability, and Uncertainty in Autonomous AI Prescribing
• MindHelper: Closed-Loop Embodied Mental-State Reasoning for Precision Intervention
• Incumbent Advantage: Brand Bias and Cognitive Manipulation Dynamics in LLM Recommendation Systems
• VSAL: A Vision Solver with Adaptive Layouts for Graph Property Detection