GrowMTP: Can RL Grow Its Own Draft Head?
cs.LG, cs.CL
Submitted: 2026-09-15
Updated: 2026-09-15
Project page: https://growmtp.github.io
License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
Terminology
Sources
- Hydra: Sequentially-Dependent Draft Heads for Medusa Decoding
- Medusa: Simple LLM Inference Acceleration Framework with Multiple Decoding Heads
- Qwen3-Coder-Next Technical Report
- Nemotron 3 Super: Open, Efficient Mixture-of-Experts Hybrid Mamba-Transformer Model for Agentic Reasoning
- Accelerating Large Language Model Decoding with Speculative Sampling
- Sequoia: Scalable, Robust, and Hardware-aware Speculative Decoding
- Better & Faster Large Language Models via Multi-token Prediction
- DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning
- History Rhymes: Accelerating LLM Reinforcement Learning with RhymeRL
- OpenAI o1 System Card
- Draft-OPD: On-Policy Distillation for Speculative Draft Models
- TACO: Topics in Algorithmic COde generation dataset
- Breaking Entropy Bounds: Accelerating RL Training via MTP with Rejection Sampling
- EAGLE: Speculative Sampling Requires Rethinking Feature Uncertainty
- Scaling Data Difficulty: Improving Coding Models via Reinforcement Learning on Fresh and Challenging Problems
- DeepSeek-V3 Technical Report
- Online Speculative Decoding
- NeMo-Aligner: Scalable Toolkit for Efficient Model Alignment
- OpenAI GPT-5 System Card
- Joint Training of Multi-Token Prediction in Reinforcement Learning via Optimal Coefficient Calibration
Related papers
- Polynomial-Augmented Neural Networks (PANNs) with Weak Orthogonality Constraints for Enhanced Function and PDE Approximation
- AIRL-S: Unifying Reinforcement Learning and Search-Based Test-Time Scaling via Adversarial Inverse Reinforcement Learning
- Transformers as Bayesian In-Context Experimenters: Smoothness-Adaptive Efficient ATE Estimation
- Convergence issues in Relational Concept Analysis based on AOC-posets
- Beliefs Beyond Posteriors: Local-Consistency Optimisation for Bayesian Neural Networks
- Understanding Diffusion Models via Ratio-Based Function Approximation with SignReLU Networks