EviDx: Evidence-Aware Active Diagnosis with Scaffolded LLM Agents
cs.AI
Submitted: 2026-08-25
Updated: 2026-08-25
Code: https://github.com/LihangZeng/EviDx
License: http://creativecommons.org/licenses/by/4.0/
The gist: Clinical diagnosis is an active evidence-seeking process in which clinicians acquire evidence, update competing hypotheses, and decide when the available evidence is sufficient for diagnosis.
Terminology
Abstract
Clinical diagnosis is an active evidence-seeking process in which clinicians acquire evidence, update competing hypotheses, and decide when the available evidence is sufficient for diagnosis. Yet many medical diagnosis systems built around large language models (LLMs) still formulate diagnosis as static case-to-answer prediction, with limited support for evidence acquisition. Agentic LLMs offer a dynamic alternative through tool use and intermediate diagnostic trajectories, but existing systems often under-specify how patient evidence should be exposed, scaffolded, and controlled at runtime. We introduce EviDx, an evidence-aware active diagnosis framework that pairs patient-specific diagnostic environments with a clinical diagnostic scaffold and an observer-guided runtime harness. In EviDx, E-Synthesis constructs interactive environments from raw clinical cases; the scaffold organizes role-specialized agents, evidence tools, and evolving evidence states; and the harness regulates diagnostic termination by tracking uncertainty and evidence coverage. A 3-level evaluation pyramid assesses execution robustness, reasoning dynamics, and diagnostic outcomes. Experiments show that EviDx improves diagnostic performance and process stability while revealing model-dependent capability boundaries.
Sources
- ToolForge: A Data Synthesis Pipeline for Multi-Hop Search without Real-World APIs
- The Llama 3 Herd of Models
- MedAgentBench: A Realistic Virtual EHR Environment to Benchmark Medical LLM Agents
- MDAgents: An Adaptive Collaboration of LLMs for Medical Decision-Making
- Agent Hospital: A Simulacrum of Hospital with Evolvable Medical Agents
- DeepSeek-V3.2: Pushing the Frontier of Open Large Language Models
- Ministral 3
- PhysicianBench: Evaluating LLM Agents in Real-World EHR Environments
- AutoMedic: An Automated Evaluation Framework for Clinical Conversational Agents with Medical Dataset Grounding
- MedAgent-Pro: Towards Evidence-based Multi-modal Medical Diagnosis via Reasoning Agentic Workflow
- A Survey of Agentic Reasoning for Large Language Models: Towards Recursively Self-Improving and Collective Agents
- Qwen3 Technical Report
- Survey on Evaluation of LLM-based Agents
- GLM-5: from Vibe Coding to Agentic Engineering
- Qwen3 Embedding: Advancing Text Embedding and Reranking Through Foundation Models
- AgentClinic: a multimodal agent benchmark to evaluate AI in simulated clinical environments
- DiagnosisArena: Benchmarking Diagnostic Reasoning for Large Language Models
- AgentSpec: Customizable Runtime Enforcement for Safe and Reliable LLM Agents
- ProbGuard: Proactive Runtime Monitoring for LLM Agent Safety via Probabilistic Prediction
Related papers
- MAVEN-T: Reinforced Heterogeneous Distillation for Real-Time Multi-Agent Trajectory Prediction
- Model Discovery Agent: LLM-assisted Bayesian experiment design for data-efficient discovery of mechanistic world models
- The Clinician's Veto: Navigating Trust, Liability, and Uncertainty in Autonomous AI Prescribing
- MindHelper: Closed-Loop Embodied Mental-State Reasoning for Precision Intervention
- Incumbent Advantage: Brand Bias and Cognitive Manipulation Dynamics in LLM Recommendation Systems
- VSAL: A Vision Solver with Adaptive Layouts for Graph Property Detection