How Do LLMs Cite? A Mechanistic Interpretation of Attribution in Retrieval-Augmented Generation
cs.IR, cs.AI, cs.CL
Submitted: 2026-06-09
Updated: 2026-09-19
Comments: This preprint has not undergone peer review or any post-submission improvements or corrections. The Version of Record of this contribution is published in Advances in Information Retrieval, ECIR 2026, Lecture Notes in Computer Science, vol. 16485, pp. 458-473, and is available online at https://doi.org/10.1007/978-3-032-21324-2_35
Journal ref: Advances in Information Retrieval, ECIR 2026. Lecture Notes in Computer Science, vol. 16485, pp. 458-473. Springer, Cham (2026)
DOI: 10.1007/978-3-032-21324-2_35
Code: https://github.com/TransformerLensOrg/TransformerLens
License: http://creativecommons.org/licenses/by/4.0/
The gist: Retrieval-Augmented Generation (RAG) aims to enhance the trustworthiness of Large Language Models (LLMs) by grounding their outputs in external documents, often using inline citations for
Terminology
Abstract
Retrieval-Augmented Generation (RAG) aims to enhance the trustworthiness of Large Language Models (LLMs) by grounding their outputs in external documents, often using inline citations for verifiability. However, the faithfulness of these citations -- whether the model genuinely uses a source to generate an answer -- remains a critical, unverified assumption. This paper offers the first mechanistic account of how a large language model decides whether to attach an inline citation while answering a factoid question. Using the Llama-3.1-8B-Instruct model in a controlled experimental environment based on the PopQA dataset, we employ an activation patching approach. We map the underlying mechanism responsible for citation, discovering that it is not a single, localized component but a distributed, multi-stage "attributional ensemble" of attention heads and MLP layers. We show that amplifying or attenuating only those critical heads and MLPs repairs over 90% of missed citations and eliminates 69% of spurious ones on PopQA without harming answer accuracy. Although gains on the multi-document HotpotQA benchmark are modest, the same component set still moves citation rates in the intended direction, indicating that the underlying mechanism is not dataset-specific. The results reveal a potential disconnect between the model's apparent reasoning and its internal computational pathway, suggesting that inline citations can create a false sense of security.
Sources
- Creating Trustworthy LLMs: Dealing with Hallucinations in Healthcare AI
- Attributed Question Answering: Evaluation and Modeling for Attributed Large Language Models
- Reasoning Models Don't Always Say What They Think
- ContextCite: Attributing Model Generation to Context
- Retrieval-Augmented Generation for Large Language Models: A Survey
- The Llama 3 Herd of Models
- Attributing Response to Context: A Jensen-Shannon Divergence Driven Mechanistic Study of Context Attribution in Retrieval-Augmented Generation
- Evaluating Verifiability in Generative Search Engines
- Are self-explanations from Large Language Models faithful?
- Hallucination-Free? Assessing the Reliability of Leading AI Legal Research Tools
- Explainable AI: Beware of Inmates Running the Asylum Or: How I Learnt to Stop Worrying and Love the Social and Behavioural Sciences
- Peering into the Mind of Language Models: An Approach for Attribution in Contextual Question Answering
- Model Internals-based Answer Attribution for Trustworthy Retrieval-Augmented Generation
- A Comprehensive Survey of Hallucination Mitigation Techniques in Large Language Models
- Correctness is not Faithfulness in RAG Attributions
- Interpretability in the Wild: a Circuit for Indirect Object Identification in GPT-2 small
- Using Mechanistic Interpretability to Craft Adversarial Attacks against Large Language Models
- HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answering
Related papers
- The Price of Isolation: Estimating the Ecosystem Cost of Symmetric Two-Sided A/B Testing
- SCAR: Semantic Continuity-Aware Retrieval for Efficient Context Expansion in RAG
- MixLoRA-DSI: Dynamically Expandable Mixture-of-LoRA Experts for Rehearsal-Free Generative Retrieval over Dynamic Corpora
- RRCM: Ranking-Driven Retrieval over Collaborative and Meta Memories for LLM Recommendation
- Right Family, Wrong Skill: Evaluating Risk Exposure in Agent Skill Retrieval
- UltRAG: a Universal Simple Scalable Recipe for Knowledge Graph RAG