MedGround: Bridging the Evidence Gap in Medical Vision-Language Models with Verified Grounding Data
cs.CV, cs.AI
Submitted: 2026-01-11
Updated: 2026-09-06
License: http://creativecommons.org/licenses/by/4.0/
The gist: Vision-Language Models (VLMs) can generate convincing clinical narratives, yet frequently struggle to visually ground their statements.
Terminology
Abstract
Vision-Language Models (VLMs) can generate convincing clinical narratives, yet frequently struggle to visually ground their statements. We posit that this limitation arises from the scarcity of high-quality, large-scale clinical referring-localization pairs. To address this, we introduce MedGround, an automated pipeline that transforms segmentation resources into high-quality medical referring grounding data. Leveraging expert masks as spatial anchors, MedGround precisely derives localization targets, extracts shape and spatial cues, and guides VLMs to synthesize natural, clinically grounded queries that reflect morphology and location. To ensure data rigor, a multi-stage verification system integrates strict formatting checks, geometry- and medical-prior rules, and image-based visual judging to filter out ambiguous or visually unsupported samples. Finally, we present MedGround-35K, a novel multimodal medical dataset. Extensive experiments demonstrate that VLMs trained with MedGround-35K consistently achieve improved referring grounding performance, enhance multi-object semantic disambiguation, and exhibit strong generalization to unseen grounding settings. This work highlights MedGround as a scalable, data-driven approach to anchor medical language to verifiable visual evidence.
Sources
- M3D: Advancing 3D Medical Image Analysis with Multi-Modal Large Language Models
- Qwen3-VL Technical Report
- Qwen2.5-VL Technical Report
- Med-GLIP: Advancing Medical Language-Image Pre-training with Large-scale Grounded Dataset
- MAIRA-2: Grounded Radiology Report Generation
- RAFT: Reward rAnked FineTuning for Generative Foundation Model Alignment
- ClinKD: Cross-Modal Clinical Knowledge Distiller For Multi-Task Medical Images
- Enhancing Radiology Report Generation and Visual Grounding using Reinforcement Learning
- MedCLM: Learning to Localize and Reason via a CoT-Curriculum in Medical Vision-Language Models
- BriFiSeg: a deep learning-based method for semantic and instance segmentation of nuclei in brightfield images
- Evaluating Reasoning Faithfulness in Medical Vision-Language Models using Multimodal Perturbations
- MosMedData: Chest CT Scans With COVID-19 Related Findings Dataset
- Qilin-Med-VL: Towards Chinese Large Vision-Language Model for General Healthcare
- RaDialog: A Large Vision-Language Model for Radiology Report Generation and Conversational Assistance
- Lingshu: A Generalist Foundation Model for Unified Multimodal Medical Understanding and Reasoning
- MedGemma Technical Report
- Advancing Multimodal Medical Capabilities of Gemini
- Weakly-supervised segmentation of referring expressions
- MedSG-Bench: A Benchmark for Medical Image Sequences Grounding
- Generalised Medical Phrase Grounding
Related papers
- Loss Knows Best: Detecting Annotation Errors in Videos via Loss Trajectories
- AnchorWeave: World-Consistent Video Generation with Retrieved Local Spatial Memories
- Benchmarking the Robustness of Foundation Models for Mammography under Domain Shift
- MambaX-Net: Dual-Input Mamba-Enhanced Cross-Attention Network for Longitudinal MRI Segmentation
- TeleOCR: Navigating Document Parsing Across Digital and Camera-Captured Documents
- A Survey on Efficient Vision-Language-Action Models