Diffusion Model in Latent Space for Medical Image Segmentation Task
cs.CV, cs.AI
Submitted: 2025-12-01
Updated: 2026-09-21
License: http://creativecommons.org/licenses/by/4.0/
The gist: Medical image segmentation is crucial for clinical diagnosis and treatment planning.
Terminology
Abstract
Medical image segmentation is crucial for clinical diagnosis and treatment planning. Traditional methods typically produce a single segmentation mask, failing to capture inherent uncertainty. Recent generative models enable the creation of multiple plausible masks per image, mimicking the collaborative interpretation of several clinicians. However, these approaches remain computationally heavy. We propose MedSegLatDiff, a diffusion based framework that combines a variational autoencoder (VAE) with a latent diffusion model for efficient medical image segmentation. The VAE compresses the input into a low dimensional latent space, reducing noise and accelerating training, while the diffusion process operates directly in this compact representation. We further replace the conventional MSE loss with weighted cross entropy in the VAE mask reconstruction path to better preserve tiny structures such as small nodules. MedSegLatDiff is evaluated on ISIC-2018 (skin lesions), CVC-Clinic (polyps), and LIDC-IDRI (lung nodules). It achieves state of the art or highly competitive Dice and IoU scores while simultaneously generating diverse segmentation hypotheses and confidence maps. This provides enhanced interpretability and reliability compared to deterministic baselines, making the model particularly suitable for clinical deployment.
Sources
- TransUNet: Transformers Make Strong Encoders for Medical Image Segmentation
- Auto-Encoding Variational Bayes
- Denoising Diffusion Implicit Models
- Classifier-Free Diffusion Guidance
- SegDiff: Image Segmentation with Diffusion Probabilistic Models
- Latent Diffusion for Medical Image Segmentation: End to end learning for fast sampling and accuracy
- ProGiDiff: Prompt-Guided Diffusion-Based Medical Image Segmentation
- Skin Lesion Analysis Toward Melanoma Detection 2018: A Challenge Hosted by the International Skin Imaging Collaboration (ISIC)
- Decoupled Weight Decay Regularization
- Pseudo Numerical Methods for Diffusion Models on Manifolds
Related papers
- Loss Knows Best: Detecting Annotation Errors in Videos via Loss Trajectories
- AnchorWeave: World-Consistent Video Generation with Retrieved Local Spatial Memories
- Benchmarking the Robustness of Foundation Models for Mammography under Domain Shift
- MambaX-Net: Dual-Input Mamba-Enhanced Cross-Attention Network for Longitudinal MRI Segmentation
- TeleOCR: Navigating Document Parsing Across Digital and Camera-Captured Documents
- A Survey on Efficient Vision-Language-Action Models