Two-Token Features and Small-Large Ensembles for VLM Hallucination Detection
cs.CL
Submitted: 2026-09-09
Updated: 2026-09-09
License: http://creativecommons.org/licenses/by/4.0/
The gist: We present our system for the SHROOM-Visions 2026 shared task on character-level VLM hallucination detection.
Terminology
Abstract
We present our system for the SHROOM-Visions 2026 shared task on character-level VLM hallucination detection. A small (4 B-parameter) VLM is fine-tuned as a per-token classifier reading a two-token feature from its own hidden states, and is ensembled with a about 400B zero-shot VLM judge at prediction time. Both components see off-the-shelf OCR of any visible in-image text. We use synthetic hallucination data generated by the large model as a source of ensemble diversity, and use validation to select feature layer, training data and OCR grounding. Our official entry reaches mean Cor 0.487 / Cor-lbl 0.387 on the hidden test set, placing 6 th/ 28 (EN), 6 th/ 21 (FR), 8 th/ 21 (IT) and 7 th/ 22 (ZH) on the task's primary Cor-lbl metric.
Related papers
- Exploring Solution Divergence and Its Effect on Large Language Model Problem Solving
- Ishigaki-IDS-Bench: A Benchmark for Generating Information Delivery Specification from BIM Information Requirements
- Subliminal Steering: Stronger Encoding of Hidden Signals
- MedStruct-S: A Benchmark for Key Discovery, Key-Conditioned QA and Semi-Structured Extraction from OCR Clinical Reports
- The End of Transformers? On Challenging Attention and the Rise of Sub-Quadratic Architectures
- Untangling the Mechanisms of Misleading Context in Medical Question Answering