Early-Bird Decoding: Accelerating Diffusion LLMs with Learnable Block Sizes and Parallel Sampling
cs.CL, cs.AI, cs.LG
Submitted: 2026-09-15
Updated: 2026-09-15
Comments: 23 pages, 4 figures
Code: https://github.com/project-numina/aimo-progress-prize
License: http://creativecommons.org/licenses/by/4.0/
The gist: Diffusion large language models (dLLMs) offer a promising parallel decoding paradigm as an alternative to autoregressive generation through iterative unmasking.
Terminology
Abstract
Diffusion large language models (dLLMs) offer a promising parallel decoding paradigm as an alternative to autoregressive generation through iterative unmasking. However, dLLMs typically require many steps before token confidence reaches the decoding threshold, resulting in inefficient inference even with block-wise KV caching. To accelerate dLLM inference, we for the first time propose an "early-bird (EB)" decoding framework, motivated by the observation that tokens with similarly low entropy tend to cluster and can be jointly decoded earlier, before reaching the confidence threshold. In particular, our EB-Decode framework integrates two key enablers: (1) a learnable network that adaptively groups tokens with similar uncertainty into variable-length blocks, rather than relying on fixed block sizes; (2) a position-aware sampler that learns to unmask tokens in parallel using fewer decoding steps within predicted variable-length blocks. Both components are developed without modifying pretrained dLLM weights and can therefore be directly deployed as plug-ins during serving, with negligible training and inference overhead. Extensive experiments across three models and four benchmarks consistently validate our observation and the effectiveness of EB-Decode, achieving 3.53-18.76 times higher throughput than the vanilla decoding method and up to 1.58 times higher throughput over the strongest baseline, Fast-dLLM, with comparable accuracy.
Sources
- Block Diffusion: Interpolating Between Autoregressive and Diffusion Language Models
- Program Synthesis with Large Language Models
- Learning to Parallel: Accelerating Diffusion Large Language Models via Learnable Parallel Decoding
- LLaDA2.0: Scaling Up Diffusion Language Models to 100B
- Evaluating Large Language Models Trained on Code
- DPad: Efficient Diffusion Language Models with Suffix Dropout
- dParallel: Learnable Parallel Decoding for dLLMs
- SDAR: A Synergistic Diffusion-AutoRegression Paradigm for Scalable Sequence Generation
- Training Verifiers to Solve Math Word Problems
- Beyond Autoregression: Fast LLMs via Self-Distillation Through Time
- The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networks
- The Llama 3 Herd of Models
- Adaptive Computation Time for Recurrent Neural Networks
- Distillation of Discrete Diffusion through Dimensional Correlations
- Wide-In, Narrow-Out: Revokable Decoding for Efficient and Effective DLLMs
- FlashDLM: Accelerating Diffusion Language Model Inference via Efficient KV Caching and Guided Diffusion
- Empirical Analysis of Decoding Biases in Masked Diffusion Models
- Accelerating Diffusion LLMs via Adaptive Parallel Decoding
- ParallelBench: Understanding the Trade-offs of Parallel Decoding in Diffusion LLMs
- Diffusion Language Models Know the Answer Before Decoding
Related papers
- Exploring Solution Divergence and Its Effect on Large Language Model Problem Solving
- Ishigaki-IDS-Bench: A Benchmark for Generating Information Delivery Specification from BIM Information Requirements
- Subliminal Steering: Stronger Encoding of Hidden Signals
- MedStruct-S: A Benchmark for Key Discovery, Key-Conditioned QA and Semi-Structured Extraction from OCR Clinical Reports
- The End of Transformers? On Challenging Attention and the Rise of Sub-Quadratic Architectures
- Untangling the Mechanisms of Misleading Context in Medical Question Answering