Training Parallel Speculative Draft Models by Directly Minimizing Expected Decoding Rounds

arXiv:2610.10411 · cs.LG, cs.AI, cs.CL, stat.ML · Submitted 2026-10-07 · Read on arXiv

cs.LG, cs.AI, cs.CL, stat.ML

Submitted: 2026-10-07

Updated: 2026-10-07

Code: https://github.com/y-x-zhao/AngelSpec-EDR

Terminology

• Speculative Decoding — 39× in this paper · explained in 2 episodes
• Verification — 16× in this paper · explained in 2 episodes
• Benchmark — 9× in this paper · explained in 2 episodes
• Expected Decoding Rounds — 9× in this paper
• Diffusion Model — 6× in this paper · explained in 8 episodes
• Diffusion Models — 6× in this paper · explained in 6 episodes
• Distillation — 4× in this paper · explained in 2 episodes
• Verifier — 4× in this paper · explained in 2 episodes
• Consistency — 3× in this paper · explained in 3 episodes
• Distribution Matching — 2× in this paper · explained in 2 episodes
• Flow Matching — 2× in this paper · explained in 8 episodes
• Accuracy — 1× in this paper · explained in 1 episode
• Adaptation — 1× in this paper · explained in 1 episode
• Alignment — 1× in this paper · explained in 1 episode
• Block Diffusion — 1× in this paper · explained in 2 episodes
• Embeddings — 1× in this paper · explained in 2 episodes
• Generative Modeling — 1× in this paper · explained in 3 episodes
• LLM-as-a-Judge — 1× in this paper · explained in 3 episodes
• Large Language Model (LLM) — 1× in this paper · explained in 3 episodes
• On-Policy Distillation — 1× in this paper · explained in 3 episodes
• Thinking mode — 1× in this paper · explained in 2 episodes
• Transformer — 1× in this paper · explained in 2 episodes
• Unified Framework — 1× in this paper · explained in 5 episodes

Sources

• Hydra: Sequentially-Dependent Draft Heads for Medusa Decoding
• Program Synthesis with Large Language Models
• Confidence-Based Decoding is Provably Efficient for Diffusion Language Models
• Medusa: Simple LLM Inference Acceleration Framework with Multiple Decoding Heads
• Accelerating Large Language Model Decoding with Speculative Sampling
• DFlash: Block Diffusion for Flash Speculative Decoding
• Evaluating Large Language Models Trained on Code
• Optimal Inference Schedules for Masked Diffusion Models
• Analog Bits: Generating Discrete Data using Diffusion Models with Self-Conditioning
• LangFlow: Continuous Diffusion Rivals Discrete in Language Modeling
• DSpark: Confidence-Scheduled Speculative Decoding with Semi-Autoregressive Generation
• Training Verifiers to Solve Math Word Problems
• Efficient Sampling with Discrete Diffusion Models: Sharp and Adaptive Guarantees
• Verification-Aware Training for Speculative Decoding
• Measuring Mathematical Problem Solving With the MATH Dataset
• Classifier-Free Diffusion Guidance
• ECHO: Elastic Speculative Decoding with Sparse Gating for High-Concurrency Scenarios
• SpecDec++: Boosting Speculative Decoding via Adaptive Candidate Lengths
• Mercury: Ultra-Fast Language Models Based on Diffusion
• Draft-OPD: On-Policy Distillation for Speculative Draft Models

Related papers

• Polynomial-Augmented Neural Networks (PANNs) with Weak Orthogonality Constraints for Enhanced Function and PDE Approximation
• AIRL-S: Unifying Reinforcement Learning and Search-Based Test-Time Scaling via Adversarial Inverse Reinforcement Learning
• Transformers as Bayesian In-Context Experimenters: Smoothness-Adaptive Efficient ATE Estimation
• Convergence issues in Relational Concept Analysis based on AOC-posets
• Beliefs Beyond Posteriors: Local-Consistency Optimisation for Bayesian Neural Networks
• Understanding Diffusion Models via Ratio-Based Function Approximation with SignReLU Networks