Can Jev Judge Radiology Reports? Evaluating a System One Model for Clinical Factuality
cs.CL, cs.AI
Submitted: 2026-09-23
Updated: 2026-09-23
Terminology
Sources
- CRIMSON: A Clinically-Grounded LLM-Based Metric for Generative Radiology Report Evaluation
- MAIRA-2: Grounded Radiology Report Generation
- VERT: Reliable LLM Judges for Radiology Report Evaluation
- Radiology-Aware Model-Based Evaluation Metric for Report Generation
- Length-Controlled AlpacaEval: A Simple Way to Debias Automatic Evaluators
- SummEval: Re-evaluating Summarization Evaluation
- On Calibration of Modern Neural Networks
- RadGraph: Extracting Clinical Entities and Relations from Radiology Reports
- GREEN: Generative Radiology Report Evaluation and Error Notation
- AtomiMed: Hierarchical Atomic Fact-Checking for Universal Clinical-Aware Medical Report Evaluation
- RadEval: A framework for radiology text evaluation
- RadSEM: A Finding-by-Finding Metric for Clinical Consistency in Radiology Reports
- BERTScore: Evaluating Text Generation with BERT
- RaTEScore: A Metric for Radiology Report Generation
- Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena
Related papers
- Exploring Solution Divergence and Its Effect on Large Language Model Problem Solving
- Ishigaki-IDS-Bench: A Benchmark for Generating Information Delivery Specification from BIM Information Requirements
- Subliminal Steering: Stronger Encoding of Hidden Signals
- MedStruct-S: A Benchmark for Key Discovery, Key-Conditioned QA and Semi-Structured Extraction from OCR Clinical Reports
- The End of Transformers? On Challenging Attention and the Rise of Sub-Quadratic Architectures
- Untangling the Mechanisms of Misleading Context in Medical Question Answering