DC-Gen: Post-Training Diffusion Acceleration with Deeply Compressed Latent Space
cs.CV, cs.AI
Submitted: 2025-09-29
Updated: 2026-09-07
Comments: Accepted to ECCV 2026. The first three authors contributed equally to this work
Code: https://github.com/dc-ai-projects/DC-Gen
License: http://creativecommons.org/licenses/by-nc-sa/4.0/
The gist: Existing text-to-image diffusion models excel at generating high-quality images, but face significant efficiency challenges when scaled to high resolutions, like 4K image generation.
Terminology
Abstract
Existing text-to-image diffusion models excel at generating high-quality images, but face significant efficiency challenges when scaled to high resolutions, like 4K image generation. While previous research accelerates diffusion models in various aspects, it seldom handles the inherent redundancy within the latent space. To bridge this gap, this paper introduces DC-Gen, a general framework that accelerates text-to-image diffusion models by leveraging a deeply compressed latent space. Rather than a costly training-from-scratch approach, DC-Gen uses an efficient post-training pipeline to preserve the quality of the base model. A key challenge in this paradigm is the representation gap between the base model's latent space and a deeply compressed latent space, which can lead to instability during direct fine-tuning. To overcome this, DC-Gen first bridges the representation gap with a lightweight embedding alignment training. Once the latent embeddings are aligned, only a small amount of LoRA fine-tuning is needed to unlock the base model's inherent generation quality. We verify DC-Gen's effectiveness on SANA and FLUX.1-Krea. The resulting DC-Gen-SANA and DC-Gen-FLUX models achieve quality comparable to their base models but with a significant speedup. Specifically, DC-Gen-FLUX reduces the latency of 4K image generation by 53x on the NVIDIA H100 GPU. When combined with NVFP4 SVDQuant, DC-Gen-FLUX generates a 4K image in just 3.5 seconds on a single NVIDIA 5090 GPU, achieving a total latency reduction of 138x compared to the base FLUX.1-Krea model. Code: https://github.com/dc-ai-projects/DC-Gen.
Sources
- PixArt-\Sigma: Weak-to-Strong Training of Diffusion Transformer for 4K Text-to-Image Generation
- SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesis
- SVDQuant: Absorbing Outliers by Low-Rank Components for 4-Bit Diffusion Models
- Latent Consistency Models: Synthesizing High-Resolution Images with Few-Step Inference
- Deep Compression Autoencoder for Efficient High-Resolution Diffusion Models
- DC-AE 1.5: Accelerating Diffusion Model Convergence with Structured Latent Space
- SANA: Efficient High-Resolution Image Synthesis with Linear Diffusion Transformers
- SANA 1.5: Efficient Scaling of Training-Time and Inference-Time Compute in Linear Diffusion Transformer
- DPM-Solver++: Fast Solver for Guided Sampling of Diffusion Probabilistic Models
- ViDiT-Q: Efficient and Accurate Quantization of Diffusion Transformers for Image and Video Generation
- Improved Distribution Matching Distillation for Fast Image Synthesis
- LinFusion: 1 GPU, 1 Minute, 16K Image
- SiT: Exploring Flow and Diffusion-based Generative Models with Scalable Interpolant Transformers
- One Step Diffusion via Shortcut Models
- PipeFusion: Patch-level Pipeline Parallelism for Diffusion Transformers Inference
- Inductive Moment Matching
- Mean Flows for One-step Generative Modeling
- Jet-Nemotron: Efficient Language Model with Post Neural Architecture Search
- Playground v2.5: Three Insights towards Enhancing Aesthetic Quality in Text-to-Image Generation
- Playground v3: Improving Text-to-Image Alignment with Deep-Fusion Large Language Models
Related papers
- Loss Knows Best: Detecting Annotation Errors in Videos via Loss Trajectories
- AnchorWeave: World-Consistent Video Generation with Retrieved Local Spatial Memories
- Benchmarking the Robustness of Foundation Models for Mammography under Domain Shift
- MambaX-Net: Dual-Input Mamba-Enhanced Cross-Attention Network for Longitudinal MRI Segmentation
- TeleOCR: Navigating Document Parsing Across Digital and Camera-Captured Documents
- A Survey on Efficient Vision-Language-Action Models