LitTraceQA: A Benchmark for Multi-Stage Grounding and Verification in Scientific Question Answering

arXiv:2608.07370 · cs.CL · Submitted 2026-08-15 · Read on arXiv

Xuye Liu, Yimu Wang, Peng Shi, Bo Xue, Xiangrui Ke, Songcheng Cai, Kath Choi, Di Wu, Freda Shi, Krzysztof Czarnecki

cs.CL

Submitted: 2026-08-15

Updated: 2026-08-18

Comments: Work in Progress

License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/

Terminology

• Benchmark — 42× in this paper · explained in 2 episodes
• Grounding — 32× in this paper · explained in 3 episodes
• Trace — 25× in this paper · explained in 2 episodes
• Verification — 22× in this paper · explained in 2 episodes
• Quantization — 12× in this paper · explained in 12 episodes
• Accuracy — 6× in this paper · explained in 1 episode
• Adaptation — 6× in this paper · explained in 1 episode
• Image-to-Sphere Policy — 6× in this paper
• Verifier — 5× in this paper · explained in 2 episodes
• Calibration — 4× in this paper · explained in 7 episodes
• Distillation — 4× in this paper · explained in 2 episodes
• Pruning — 4× in this paper · explained in 4 episodes
• Alignment — 3× in this paper · explained in 1 episode
• Benchmarking — 3× in this paper · explained in 3 episodes
• Core — 2× in this paper · explained in 2 episodes
• Knowledge Distillation — 2× in this paper · explained in 11 episodes
• 3D Gaussian Splatting — 1× in this paper · explained in 2 episodes
• Fidelity — 1× in this paper · explained in 2 episodes
• Foundation Model — 1× in this paper · explained in 3 episodes
• Foundation Models — 1× in this paper · explained in 7 episodes
• Gaussian Splatting — 1× in this paper · explained in 4 episodes

Sources

• Attributed Question Answering: Evaluation and Modeling for Attributed Large Language Models
• A Dataset of Information-Seeking Questions and Answers Anchored in Research Papers
• Evidence Inference 2.0: More Data, Better Models
• Enabling Large Language Models to Generate Text with Citations
• PaperQA: Retrieval-Augmented Generative Agent for Scientific Research
• M3SciQA: A Multi-Modal Multi-Document Scientific QA Benchmark for Evaluating Foundation Models
• S2ORC: The Semantic Scholar Open Research Corpus
• VISTAQA: Benchmarking Joint Visual Question Answering and Pixel-Level Evidence
• SciRepEval: A Multi-Format Benchmark for Scientific Document Representations
• Fact or Fiction: Verifying Scientific Claims
• ResearchQA: Evaluating Scholarly Question Answering at Scale Across 75 Fields with Survey-Mined Questions and Rubrics

Related papers

• Exploring Solution Divergence and Its Effect on Large Language Model Problem Solving
• Ishigaki-IDS-Bench: A Benchmark for Generating Information Delivery Specification from BIM Information Requirements
• Subliminal Steering: Stronger Encoding of Hidden Signals
• MedStruct-S: A Benchmark for Key Discovery, Key-Conditioned QA and Semi-Structured Extraction from OCR Clinical Reports
• The End of Transformers? On Challenging Attention and the Rise of Sub-Quadratic Architectures
• Untangling the Mechanisms of Misleading Context in Medical Question Answering