Control-Token Injection Suppresses Chain-of-Thought and Defeats Reasoning-Based Oversight in Tool-Using Agents

arXiv:2609.27542 · cs.CR · Submitted 2026-09-23 · Read on arXiv

cs.CR

Submitted: 2026-09-23

Updated: 2026-09-23

Code: https://github.com/Usama1002/deleting-the-trace

Terminology

• Trace — 51× in this paper · explained in 2 episodes
• Commentary — 9× in this paper · explained in 2 episodes
• Provenance — 8× in this paper · explained in 4 episodes
• Decode — 7× in this paper · explained in 2 episodes
• Transformer — 6× in this paper · explained in 2 episodes
• Ablation — 5× in this paper · explained in 1 episode
• Benchmark — 5× in this paper · explained in 2 episodes
• Tokenization — 4× in this paper · explained in 2 episodes
• AI Agents — 1× in this paper · explained in 2 episodes
• Abstraction — 1× in this paper · explained in 1 episode
• Adversarial Attacks — 1× in this paper · explained in 1 episode
• Agent Hijacking — 1× in this paper · explained in 1 episode
• Benchmarking — 1× in this paper · explained in 3 episodes
• Chain-of-Thought (CoT) — 1× in this paper · explained in 4 episodes
• Cooperation — 1× in this paper · explained in 2 episodes
• Empirical Methods in Natural Language Processing — 1× in this paper
• In International Conference on Machine Learning — 1× in this paper
• LLM judge — 1× in this paper · explained in 2 episodes
• Reinforcement Learning — 1× in this paper · explained in 4 episodes

Sources

• Monitoring Reasoning Models for Misbehavior and the Risks of Promoting Obfuscation
• ChatInject: Abusing Chat Templates for Prompt Injection in LLM Agents
• Reasoning Models Struggle to Control their Chains of Thought
• AgentDojo: A Dynamic Environment to Evaluate Prompt Injection Attacks and Defenses for LLM Agents
• DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning
• Automating Agent Hijacking via Structural Template Injection
• In AI Sweet Harmony: Sociopragmatic Guardrail Bypasses and Evaluation-Awareness in OpenAI gpt-oss-20b
• When Chain of Thought is Necessary, Language Models Struggle to Evade Monitors
• Gemma 4 Technical Report
• Chain of Thought Monitorability: A New and Fragile Opportunity for AI Safety
• PromptArmor: Simple yet Effective Prompt Injection Defenses
• The Instruction Hierarchy: Training LLMs to Prioritize Privileged Instructions
• RTBAS: Defending LLM Agents Against Prompt Injection and Privacy Leakage
• MELON: Provable Defense Against Indirect Prompt Injection Attacks in AI Agents
• Universal and Transferable Adversarial Attacks on Aligned Language Models

Related papers

• SoK: AI-Augmented Binary Reversing
• Relaxed Sender Anonymity for CBDC Interbank Settlement: A Zero-Knowledge Approach on Permissioned EVM
• Calibration-Family Overfit: Why Trusted Sabotage Monitors Don't Transfer Across Lineages
• Efficient Fuzzy PSI under One-Sided Assumptions
• Sealing the Audit-Runtime Gap for LLM Skills
• Token Composition: A Graph Based on EVM Logs