When Detection Does Not Guarantee Resistance: Reasoning and Poisoned Context in RAG
cs.CL
Submitted: 2026-08-17
Updated: 2026-09-30
Comments: 7 pages
License: http://creativecommons.org/licenses/by-nc-nd/4.0/
The gist: Retrieval-Augmented Generation (RAG) improves large language models by grounding them in external evidence, but this exposes them to knowledge-poisoning attacks, where misinformation injected into
Terminology
Abstract
Retrieval-Augmented Generation (RAG) improves large language models by grounding them in external evidence, but this exposes them to knowledge-poisoning attacks, where misinformation injected into retrieved documents influences model outputs. We investigate whether deliberative reasoning reduces susceptibility to poisoned evidence using two metrics: Cordon Rate, which measures cases where detected misinformation nevertheless influences the final answer, and Leakage Rate, which measures implicit influence from poisoned context despite explicit instructions to disregard it. We evaluate six model configurations on 200 SciFact questions, including DeepSeek-V4-Flash and Qwen3.6-Plus with reasoning disabled and enabled. Enabling reasoning reduces conditional susceptibility: DeepSeek-V4-Flash reduces Cordon Rate from 0.211 to 0.107 and Leakage Rate from 0.235 to 0.140, despite overall attack success rising from 0.233 to 0.298. These results show that poison detection, attack success, and resistance to contextual influence are distinct capabilities, and that deliberative reasoning reduces behavioral impact of corrupted evidence conditional on detection, even as it renders explicit poison identification less reliable.
Sources
- A Comprehensive Anatomy of Human and DeepSeek-R1 LLM Mathematical Reasoning
- BEIR: A Heterogenous Benchmark for Zero-shot Evaluation of Information Retrieval Models
- Cordon-MAS: Defending RAG against Knowledge Poisoning via Information-Flow Control
- Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilities
- Detecting Is Not Resolving: The Monitoring Control Gap in Retrieval Augmented LLMs
- DeepSeek-V3 Technical Report
- Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks
Related papers
- Exploring Solution Divergence and Its Effect on Large Language Model Problem Solving
- Ishigaki-IDS-Bench: A Benchmark for Generating Information Delivery Specification from BIM Information Requirements
- Subliminal Steering: Stronger Encoding of Hidden Signals
- MedStruct-S: A Benchmark for Key Discovery, Key-Conditioned QA and Semi-Structured Extraction from OCR Clinical Reports
- The End of Transformers? On Challenging Attention and the Rise of Sub-Quadratic Architectures
- Untangling the Mechanisms of Misleading Context in Medical Question Answering