FlexEE: Self-Speculative and KV-Compatible Early Exiting for Offloading-Aware LLM Inference
cs.AI
Submitted: 2026-09-15
Updated: 2026-09-15
License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
Terminology
Sources
- Program Synthesis with Large Language Models
- Scaling Inference-Efficient Language Models
- $\pi_0$: A Vision-Language-Action Flow Model for General Robot Control
- EE-LLM: Large-Scale Training and Inference of Early-Exit Large Language Models with 3D Parallelism
- BoolQ: Exploring the Surprising Difficulty of Natural Yes/No Questions
- Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challenge
- Not All Layers of LLMs Are Necessary During Inference
- GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformers
- TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehension
- Depth-Adaptive Transformer
- Scaling Laws for Neural Language Models
- RACE: Large-scale ReAding Comprehension Dataset From Examinations
- Scalable Vision-Language-Action Model Pretraining for Robotic Manipulation with Real-Life Human Activity Videos
- Pointer Sentinel Mixture Models
- A Simple and Effective Pruning Approach for Large Language Models
- LLaMA: Open and Efficient Foundation Language Models
- Llama 2: Open Foundation and Fine-Tuned Chat Models
- Sheared LLaMA: Accelerating Language Model Pre-training via Structured Pruning
- DeeBERT: Dynamic Early Exiting for Accelerating BERT Inference
Related papers
- MAVEN-T: Reinforced Heterogeneous Distillation for Real-Time Multi-Agent Trajectory Prediction
- Model Discovery Agent: LLM-assisted Bayesian experiment design for data-efficient discovery of mechanistic world models
- The Clinician's Veto: Navigating Trust, Liability, and Uncertainty in Autonomous AI Prescribing
- MindHelper: Closed-Loop Embodied Mental-State Reasoning for Precision Intervention
- Incumbent Advantage: Brand Bias and Cognitive Manipulation Dynamics in LLM Recommendation Systems
- VSAL: A Vision Solver with Adaptive Layouts for Graph Property Detection