Prefilling-dLLM: Predictive Prefilling for Long-Context Inference in Diffusion Language Models
cs.CL
Submitted: 2026-06-09
Updated: 2026-08-29
Code: https://github.com/menik1126/Prefilling-dLLM
Terminology
Sources
- Flex Attention: A Programming Model for Generating Optimized Attention Kernels
- Scaling Diffusion Language Models via Adaptation from Autoregressive Models
- DiffuSeq: Sequence to Sequence Text Generation with Diffusion Models
- UltraLLaDA: Scaling the Context Length to 128K for Diffusion Large Language Models
- semi-PD: Towards Efficient LLM Serving via Phase-Wise Disaggregated Computation and Unified Storage
- d$^2$Cache: Accelerating Diffusion-Based LLMs via Dual Adaptive Caching
- FlexPrefill: A Context-Aware Sparse Attention Mechanism for Efficient Long-Sequence Inference
- LLaDA2.0: Scaling Up Diffusion Language Models to 100B
- Diffusion LLMs Can Do Faster-Than-AR Inference via Discrete Diffusion Forcing
- SparseD: Sparse Attention for Diffusion Language Models
- LongLLaDA: Unlocking Long Context Capabilities in Diffusion LLMs
- Fast-dLLM v2: Efficient Block-Diffusion LLM
- dLLM-Cache: Accelerating Diffusion Large Language Models with Adaptive Caching
- Fast-dLLM: Training-free Acceleration of Diffusion LLM by Enabling KV Cache and Parallel Decoding
- Focus-dLLM: Accelerating Long-Context Diffusion LLM Inference via Confidence-Guided Context Focusing
- LoSA: Locality Aware Sparse Attention for Block-Wise Diffusion Language Models
- Attention Is All You Need for KV Cache in Diffusion LLMs
- Large Language Diffusion Models
- YaRN: Efficient Context Window Extension of Large Language Models
- Dream 7B: Diffusion Large Language Models
Related papers
- Exploring Solution Divergence and Its Effect on Large Language Model Problem Solving
- Ishigaki-IDS-Bench: A Benchmark for Generating Information Delivery Specification from BIM Information Requirements
- Subliminal Steering: Stronger Encoding of Hidden Signals
- MedStruct-S: A Benchmark for Key Discovery, Key-Conditioned QA and Semi-Structured Extraction from OCR Clinical Reports
- The End of Transformers? On Challenging Attention and the Rise of Sub-Quadratic Architectures
- Untangling the Mechanisms of Misleading Context in Medical Question Answering