GroupKV: Hierarchical KV Cache Management for Long-Context Diffusion LLM Inference

arXiv:2609.17573 · cs.OS, cs.DC, cs.LG · Submitted 2026-07-30 · Read on arXiv

cs.OS, cs.DC, cs.LG

Submitted: 2026-07-30

Updated: 2026-07-30

License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/

Terminology

• KV cache — 3× in this paper · explained in 3 episodes
• Accuracy — 1× in this paper · explained in 1 episode
• Clustering — 1× in this paper · explained in 2 episodes
• Consistency — 1× in this paper · explained in 3 episodes

Related papers

• MemSpec: Memory-Aware Runtime for Adaptive Draft Scheduling in Speculative Decoding on Edge Devices
• Planarian: Managing Agent State with Statepoints
• VUDA: Enabling Controlled Spatial Sharing of Graphics and Compute on NVIDIA GPUs
• ActKV: Efficient LLM Agents through Action-Guided KV Cache Management
• SeqMoE: Toward Full-Load Performance via Predictive and Graph-Compatible MoE Offloading
• Continuum: Efficient and Robust Multi-Turn LLM Agent Scheduling with KV Cache Time-to-Live