Hallucination Detection in LLMs with Topological Divergence on Attention Graphs
cs.CL, cs.AI, math.AT, math.MG
Submitted: 2025-04-14
Updated: 2026-05-12
Comments: Accepted to the 64th Annual Meeting of the Association for Computational Linguistics (ACL 2026)
Code: https://github.com/sb-ai-lab/TOHA
License: http://creativecommons.org/licenses/by/4.0/
The gist: Hallucination, i.e., generating factually incorrect content, remains a critical challenge for large language models (LLMs).
Terminology
Abstract
Hallucination, i.e., generating factually incorrect content, remains a critical challenge for large language models (LLMs). We introduce TOHA, a TOpology-based HAllucination detector in the RAG setting, which leverages a topological divergence metric to quantify the structural properties of graphs induced by attention matrices. Examining the topological divergence between prompt and response subgraphs reveals consistent patterns: higher divergence values in specific attention heads correlate with hallucinated outputs, independent of the dataset. Extensive experiments - including evaluation on question answering and summarization tasks - show that our approach achieves state-of-the-art or competitive results on several benchmarks while requiring minimal annotated data and computational resources. Our findings suggest that analyzing the topological structure of attention matrices can serve as an efficient and robust indicator of factual reliability in LLMs.
Sources
- Fact-Checking the Output of Large Language Models via Token-Level Uncertainty Quantification
- Hallucination Detection in LLMs Using Spectral Features of Attention Maps
- RAGTruth: A Hallucination Corpus for Developing Trustworthy Retrieval-Augmented Language Models
- Siren's Song in the AI Ocean: A Survey on Hallucination in Large Language Models
- Efficient Hallucination Detection for LLMs Using Uncertainty-Aware Attention Heads
Related papers
- Exploring Solution Divergence and Its Effect on Large Language Model Problem Solving
- Ishigaki-IDS-Bench: A Benchmark for Generating Information Delivery Specification from BIM Information Requirements
- Subliminal Steering: Stronger Encoding of Hidden Signals
- MedStruct-S: A Benchmark for Key Discovery, Key-Conditioned QA and Semi-Structured Extraction from OCR Clinical Reports
- The End of Transformers? On Challenging Attention and the Rise of Sub-Quadratic Architectures
- Untangling the Mechanisms of Misleading Context in Medical Question Answering