Know When to Hold 'em: Correct-Token Retention in Uniform-State Diffusion Language Models

arXiv:2610.01275 · cs.CL · Submitted 2026-10-01 · Read on arXiv

cs.CL

Submitted: 2026-10-01

Updated: 2026-10-01

Project page: http://skylion007.github.io/OpenWebTextCorpus

Terminology

• Accuracy — 49× in this paper · explained in 1 episode
• Perplexity — 18× in this paper · explained in 3 episodes
• Diffusion Model — 16× in this paper · explained in 8 episodes
• Diffusion Models — 16× in this paper · explained in 6 episodes
• Regularization — 9× in this paper · explained in 3 episodes
• Ablation — 4× in this paper · explained in 1 episode
• Benchmark — 3× in this paper · explained in 2 episodes
• Trace — 3× in this paper · explained in 2 episodes
• Consistency — 2× in this paper · explained in 3 episodes
• Adam — 1× in this paper · explained in 2 episodes
• Computational overhead — 1× in this paper · explained in 2 episodes
• Decode — 1× in this paper · explained in 2 episodes
• Downstream Tasks — 1× in this paper · explained in 2 episodes
• Generalization — 1× in this paper · explained in 13 episodes
• Generative AI — 1× in this paper · explained in 2 episodes
• Generative Modeling — 1× in this paper · explained in 3 episodes
• Iterative Refinement — 1× in this paper · explained in 3 episodes
• Latent Diffusion Model — 1× in this paper · explained in 2 episodes
• Latent Diffusion Models — 1× in this paper · explained in 2 episodes
• Transformer — 1× in this paper · explained in 2 episodes

Sources

• Semi-Parametric Neural Image Synthesis
• One Billion Word Benchmark for Measuring Progress in Statistical Language Modeling
• Denoising Diffusion Probabilistic Models
• Noise2Music: Text-conditioned Music Generation with Diffusion Models
• Mercury: Ultra-Fast Language Models Based on Diffusion
• Discrete Diffusion Modeling by Estimating the Ratios of the Data Distribution
• Inference-Time Scaling for Diffusion Models beyond Scaling Denoising Steps
• Movie Gen: A Cast of Media Foundation Models
• Simple Guidance Mechanisms for Discrete Diffusion Models
• Consistency Models
• Seed Diffusion: A Large-Scale Diffusion Language Model with High-Speed Inference
• Character-level Convolutional Networks for Text Classification

Related papers

• Exploring Solution Divergence and Its Effect on Large Language Model Problem Solving
• Ishigaki-IDS-Bench: A Benchmark for Generating Information Delivery Specification from BIM Information Requirements
• Subliminal Steering: Stronger Encoding of Hidden Signals
• MedStruct-S: A Benchmark for Key Discovery, Key-Conditioned QA and Semi-Structured Extraction from OCR Clinical Reports
• The End of Transformers? On Challenging Attention and the Rise of Sub-Quadratic Architectures
• Untangling the Mechanisms of Misleading Context in Medical Question Answering