Proteus: A Self-Evolving Red Team for Agent Skill Ecosystems

arXiv:2605.11891 · cs.CR, cs.AI · Submitted 2026-05-12 · Read on arXiv

cs.CR, cs.AI

Submitted: 2026-05-12

Updated: 2026-09-26

Code: https://github.com/microsoft/PyRIT

Terminology

• Ablation — 15× in this paper · explained in 1 episode
• Trace — 11× in this paper · explained in 2 episodes
• Alignment — 9× in this paper · explained in 1 episode
• Agent Skills — 5× in this paper · explained in 1 episode
• Benchmark — 4× in this paper · explained in 2 episodes
• International Conference on Learning Representations — 4× in this paper
• Admissibility — 3× in this paper · explained in 2 episodes
• Core — 3× in this paper · explained in 2 episodes
• Verification — 3× in this paper · explained in 2 episodes
• Ablation study — 2× in this paper · explained in 1 episode
• Benchmarking — 2× in this paper · explained in 3 episodes
• Generative AI — 2× in this paper · explained in 2 episodes
• LLM judge — 2× in this paper · explained in 2 episodes
• RAG — 2× in this paper · explained in 3 episodes
• Robustness — 2× in this paper · explained in 5 episodes
• Strategy Diversity Growth — 2× in this paper
• ACM Conference on Computer and Communications Security — 1× in this paper
• Adaptation — 1× in this paper · explained in 1 episode
• Adversarial Attacks — 1× in this paper · explained in 1 episode
• Consistency — 1× in this paper · explained in 3 episodes
• Degeneracy — 1× in this paper · explained in 3 episodes
• Embeddings — 1× in this paper · explained in 2 episodes
• Failure Modes — 1× in this paper · explained in 3 episodes
• Graph Construction — 1× in this paper · explained in 3 episodes
• Information Leakage — 1× in this paper · explained in 2 episodes
• Interpretability — 1× in this paper · explained in 15 episodes
• Iterative Refinement — 1× in this paper · explained in 3 episodes
• Learning Curve Slope — 1× in this paper
• Meeting of the Association for Computational Linguistics — 1× in this paper
• Multi-agent Collaboration — 1× in this paper · explained in 2 episodes
• Multi-agent Collaborative Framework — 1× in this paper · explained in 2 episodes
• Provenance — 1× in this paper · explained in 4 episodes
• Reinforcement Learning — 1× in this paper · explained in 4 episodes
• Sample Efficiency — 1× in this paper · explained in 4 episodes
• Thinking mode — 1× in this paper · explained in 2 episodes

Sources

• Constitutional AI: Harmlessness from AI Feedback
• Red Teaming Language Models to Reduce Harms: Methods, Scaling Behaviors, and Lessons Learned
• RedCodeAgent: Automatic Red-teaming Agent against Diverse Code Agents
• SkillProbe: Security Auditing for Emerging Agent Skill Marketplaces via Multi-Agent Collaboration
• Llama Guard: LLM-based Input-Output Safeguard for Human-AI Conversations
• "Do Not Mention This to the User": Detecting and Understanding Malicious Agent Skills in the Wild
• Structured Security Auditing and Robustness Enhancement for Untrusted Agent Skills
• GAIA: a benchmark for General AI Assistants
• AgenticRed: Evolving Agentic Systems for Red-Teaming
• Active Attacks: Red-teaming LLMs via Adaptive Environments
• Genesis: Evolving Attack Strategies for LLM Web Agent Red-Teaming
• AutoRedTeamer: Autonomous Red Teaming with Lifelong Attack Integration
• Universal and Transferable Adversarial Attacks on Aligned Language Models

Related papers

• SoK: AI-Augmented Binary Reversing
• Relaxed Sender Anonymity for CBDC Interbank Settlement: A Zero-Knowledge Approach on Permissioned EVM
• Calibration-Family Overfit: Why Trusted Sabotage Monitors Don't Transfer Across Lineages
• Efficient Fuzzy PSI under One-Sided Assumptions
• Sealing the Audit-Runtime Gap for LLM Skills
• Token Composition: A Graph Based on EVM Logs