Can We Stop Malicious AI? KILLBENCH: A Benchmark for External AI Kill Switch Feasibility

arXiv:2511.13725 · cs.CR, cs.AI · Submitted 2025-09-26 · Read on arXiv

cs.CR, cs.AI

Submitted: 2025-09-26

Updated: 2026-09-12

Comments: Accepted to Findings of the Association for Computational Linguistics: EMNLP 2026. 48 pages (11 pages main text incl. references, 35 pages appendix), 10 figures, 9 tables. Project page: https://sechan00.github.io/KillBench/

Project page: https://sechan00.github.io/KillBench

License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/

Terminology

• Benchmark — 33× in this paper · explained in 2 episodes
• Alignment — 24× in this paper · explained in 1 episode
• Core — 7× in this paper · explained in 2 episodes
• LLM judge — 7× in this paper · explained in 2 episodes
• AI Agents — 4× in this paper · explained in 2 episodes
• Commentary — 3× in this paper · explained in 2 episodes
• Interpretability — 3× in this paper · explained in 15 episodes
• Adversarial Noise — 2× in this paper · explained in 1 episode
• Character Role Play — 2× in this paper
• Generative AI — 2× in this paper · explained in 2 episodes
• Regret — 2× in this paper · explained in 3 episodes
• Research Experiment — 2× in this paper
• Robustness — 2× in this paper · explained in 5 episodes
• Text Continuation — 2× in this paper
• Ablation — 1× in this paper · explained in 1 episode
• Action Space — 1× in this paper · explained in 1 episode
• Adversarial Attacks — 1× in this paper · explained in 1 episode
• Adversarial Context — 1× in this paper · explained in 1 episode
• Agent Harness — 1× in this paper · explained in 2 episodes
• Agentic Task — 1× in this paper · explained in 1 episode
• Agentic Tasks — 1× in this paper · explained in 1 episode
• Alignment Techniques — 1× in this paper · explained in 1 episode
• Assumed Responsibility — 1× in this paper
• Benchmarking — 1× in this paper · explained in 3 episodes
• Consistency — 1× in this paper · explained in 3 episodes
• Decoupling — 1× in this paper · explained in 2 episodes
• Fine-Tuning — 1× in this paper · explained in 5 episodes
• Fingerprinting — 1× in this paper · explained in 2 episodes
• Indirect Prompt Injection — 1× in this paper
• Prompt Engineering — 1× in this paper · explained in 4 episodes
• Provenance — 1× in this paper · explained in 4 episodes
• Reinforcement Learning — 1× in this paper · explained in 4 episodes
• Reverted Refusal — 1× in this paper
• Trace — 1× in this paper · explained in 2 episodes
• Transferability — 1× in this paper · explained in 4 episodes
• Zero-Shot Transfer — 1× in this paper · explained in 4 episodes

Sources

• Constitutional AI: Harmlessness from AI Feedback
• LLM Agents can Autonomously Hack Websites
• Llama Guard: LLM-based Input-Output Safeguard for Human-AI Conversations
• EIA: Environmental Injection Attack on Generalist Web Agents for Privacy Leakage
• Jailbreaking ChatGPT via Prompt Engineering: An Empirical Study
• Core Safety Values for Provably Corrigible Agents
• Hacking Back the AI-Hacker: Prompt Injection as a Defense Against LLM-driven Cyberattacks
• SafeArena: Evaluating the Safety of Autonomous Web Agents
• Universal and Transferable Adversarial Attacks on Aligned Language Models

Related papers

• SoK: AI-Augmented Binary Reversing
• Relaxed Sender Anonymity for CBDC Interbank Settlement: A Zero-Knowledge Approach on Permissioned EVM
• Calibration-Family Overfit: Why Trusted Sabotage Monitors Don't Transfer Across Lineages
• Efficient Fuzzy PSI under One-Sided Assumptions
• Sealing the Audit-Runtime Gap for LLM Skills
• Token Composition: A Graph Based on EVM Logs