Safeguards Based on Copyable Context Cannot Provide Reliable Safety for LLMs
Pingyu Wu, Lingyao Zhu, Weiming Zhang, Nenghai Yu
cs.CR, cs.AI
Submitted: 2026-07-30
License: http://creativecommons.org/licenses/by-nc-sa/4.0/
The gist: Large language model safeguards decide whether to answer before seeing how an answer will be used.
Terminology
Abstract
Large language model safeguards decide whether to answer before seeing how an answer will be used. This creates a basic problem for dual-use tasks: the same answer can help an authorized professional or an attacker, while an attacker can imitate a benign request and interaction history. We separate the capability released by the model from the evidence available about downstream use. When that evidence is copyable, we derive the exact worst-case floor on attacker assistance while preserving useful answers. The result yields a safety trilemma: Useful Capability, Reliable Safety, and Open Access cannot coexist. We then show how a trusted credential can complement existing safeguards by adding hard-to-copy information that predicts actual downstream use, and identify the stronger condition needed to eliminate the floor. Evidence from dual-use evaluations, adaptive attacks, and deployed trusted-access programs supports the practical relevance of these conditions.
Sources
- Personhood credentials: Artificial intelligence and the value of privacy-preserving tools to distinguish who is real online
- Constitutional AI: Harmlessness from AI Feedback
- On the Impossibility of Separating Intelligence from Judgment: The Computational Intractability of Filtering for AI Alignment
- The Defense Trilemma: Why Prompt Injection Defense Wrappers Fail?
- Defensive Refusal Bias: How Safety Alignment Fails Cyber Defenders
- Uncertainty-Aware Clarification in LLM Agents with Information Gain
- Paved with True Intents: Intent-Aware Training Improves LLM Safety Classification Across Training Regimes
- Breach By A Thousand Leaks: Unsafe Information Leakage in `Safe' AI Responses
- Position Paper: Model Access should be a Key Concern in AI Governance
- No Free Lunch with Guardrails
- Measuring AI Ability to Complete Long Software Tasks
- Beyond Refusal: A Same-Lineage Study of Aligned and Abliterated LLMs for Vulnerability Analysis
- The WMDP Benchmark: Measuring and Reducing Malicious Use With Unlearning
- The Attacker Moves Second: Stronger Adaptive Attacks Bypass Defenses Against Llm Jailbreaks and Prompt Injections
- Deep Ignorance: Filtering Pretraining Data Builds Tamper-Resistant Safeguards into Open-Weight LLMs
- SafeRedirect: Defeating Internal Safety Collapse via Task-Completion Redirection in Frontier LLMs
- Modular Pretraining Enables Access Control
- Constitutional Classifiers: Defending against Universal Jailbreaks across Thousands of Hours of Red Teaming
- OpenSafeIntent: Evaluating Intent-Calibrated Safe Completion Across Dual-Use Prompt Sets
- False Sense of Security: Why Probing-based Malicious Input Detection Fails to Generalize
Related papers
- SoK: AI-Augmented Binary Reversing
- Relaxed Sender Anonymity for CBDC Interbank Settlement: A Zero-Knowledge Approach on Permissioned EVM
- Calibration-Family Overfit: Why Trusted Sabotage Monitors Don't Transfer Across Lineages
- Efficient Fuzzy PSI under One-Sided Assumptions
- Sealing the Audit-Runtime Gap for LLM Skills
- Token Composition: A Graph Based on EVM Logs