Where Does Retrieval-Based Open-Ended Evaluation Fail? Automatic Taxonomy Induction from Long-Form Medical Answer Factuality Verification

arXiv:2609.30467 · cs.CL, cs.IR · Submitted 2026-09-24 · Read on arXiv

cs.CL, cs.IR

Submitted: 2026-09-24

Updated: 2026-09-24

Code: https://github.com/gaojie058/Act-onomy

Terminology

• Verifier — 90× in this paper · explained in 2 episodes
• Verification — 48× in this paper · explained in 2 episodes
• Benchmark — 18× in this paper · explained in 2 episodes
• Trace — 16× in this paper · explained in 2 episodes
• Ablation — 14× in this paper · explained in 1 episode
• Alignment — 13× in this paper · explained in 1 episode
• Grounding — 8× in this paper · explained in 3 episodes
• Consistency — 7× in this paper · explained in 3 episodes
• Reasoning Traces — 7× in this paper · explained in 3 episodes
• Core — 6× in this paper · explained in 2 episodes
• Fine-Tuning — 6× in this paper · explained in 5 episodes
• RAG — 5× in this paper · explained in 3 episodes
• Adam — 3× in this paper · explained in 2 episodes
• Error Taxonomy — 3× in this paper · explained in 2 episodes
• Calibration — 2× in this paper · explained in 7 episodes
• Hallucination — 2× in this paper · explained in 4 episodes
• LLM judge — 2× in this paper · explained in 2 episodes
• Question Answering — 2× in this paper
• Transformer — 2× in this paper · explained in 2 episodes
• Accuracy — 1× in this paper · explained in 1 episode
• Adaptation — 1× in this paper · explained in 1 episode
• Annotator Agreement — 1× in this paper · explained in 1 episode
• Benchmarking — 1× in this paper · explained in 4 episodes
• Contrastive Learning — 1× in this paper · explained in 5 episodes
• Cosine Similarity — 1× in this paper · explained in 3 episodes
• Domain Adaptation — 1× in this paper · explained in 5 episodes
• F1 Score — 1× in this paper · explained in 3 episodes
• Failure Modes — 1× in this paper · explained in 3 episodes
• Foundation Model — 1× in this paper · explained in 3 episodes
• Foundation Models — 1× in this paper · explained in 7 episodes
• Generalizability — 1× in this paper · explained in 6 episodes
• Generalization — 1× in this paper · explained in 13 episodes
• Generative Model — 1× in this paper · explained in 2 episodes
• Hallucination Detection — 1× in this paper · explained in 2 episodes
• Human-in-the-Loop — 1× in this paper · explained in 3 episodes
• Mental Health — 1× in this paper
• Model-Agnostic — 1× in this paper · explained in 2 episodes
• Pairwise Comparisons — 1× in this paper · explained in 2 episodes
• Prenatal Care — 1× in this paper
• Representation Learning — 1× in this paper · explained in 2 episodes
• Scaffolding — 1× in this paper · explained in 3 episodes
• Test-time Scaling — 1× in this paper · explained in 2 episodes

Sources

• LLM-as-a-qualitative-judge: automating error analysis in natural language generation
• How to Interpret Agent Behavior
• Unsupervised Dense Information Retrieval with Contrastive Learning
• MedRAGChecker: Claim-Level Verification for Biomedical Retrieval-Augmented Generation
• Rethinking Retrieval-Augmented Generation for Medicine: A Large-Scale, Systematic Expert Evaluation and Practical Insights
• RAGChecker: A Fine-grained Framework for Diagnosing Retrieval-Augmented Generation
• MedGemma Technical Report
• OpenAI GPT-5 System Card
• RAG-X: Systematic Diagnosis of Retrieval-Augmented Generation for Medical Question Answering
• Qwen3 Embedding: Advancing Text Embedding and Reranking Through Foundation Models

Related papers

• Exploring Solution Divergence and Its Effect on Large Language Model Problem Solving
• Ishigaki-IDS-Bench: A Benchmark for Generating Information Delivery Specification from BIM Information Requirements
• Subliminal Steering: Stronger Encoding of Hidden Signals
• MedStruct-S: A Benchmark for Key Discovery, Key-Conditioned QA and Semi-Structured Extraction from OCR Clinical Reports
• The End of Transformers? On Challenging Attention and the Rise of Sub-Quadratic Architectures
• Untangling the Mechanisms of Misleading Context in Medical Question Answering