Beyond Tokens: Semantic-Aware Speculative Decoding for Efficient Inference by Probing Internal States
cs.CL, cs.PF
Submitted: 2026-02-03
Updated: 2026-08-31
Terminology
Sources
- Qwen Technical Report
- Medusa: Simple LLM Inference Acceleration Framework with Multiple Decoding Heads
- A Survey on LLM-as-a-Judge
- DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning
- Measuring Mathematical Problem Solving With the MATH Dataset
- OpenAI o1 System Card
- Semantic Entropy Probes: Robust and Cheap Hallucination Detection in LLMs
- Semantic Uncertainty: Linguistic Invariances for Uncertainty Estimation in Natural Language Generation
- Evaluating Scoring Bias in LLM-as-a-Judge
- Reward-Guided Speculative Decoding for Efficient LLM Reasoning
- DeepSeek-V3 Technical Report
- Decoupled Weight Decay Regularization
- SpecInfer: Accelerating Generative Large Language Model Serving with Tree-based Speculative Inference and Verification
- s1: Simple test-time scaling
- SpecReason: Fast and Accurate Inference-Time Compute via Speculative Reasoning
- GPT-4 Technical Report
- Accelerating LLM Inference with Staged Speculative Decoding
- Gemma: Open Models Based on Gemini Research and Technology
- Judging the Judges: Evaluating Alignment and Vulnerabilities in LLMs-as-Judges
- LLaMA: Open and Efficient Foundation Language Models
Related papers
- Exploring Solution Divergence and Its Effect on Large Language Model Problem Solving
- Ishigaki-IDS-Bench: A Benchmark for Generating Information Delivery Specification from BIM Information Requirements
- Subliminal Steering: Stronger Encoding of Hidden Signals
- MedStruct-S: A Benchmark for Key Discovery, Key-Conditioned QA and Semi-Structured Extraction from OCR Clinical Reports
- The End of Transformers? On Challenging Attention and the Rise of Sub-Quadratic Architectures
- Untangling the Mechanisms of Misleading Context in Medical Question Answering