DeltaS: Reading the Gated Linear Attention State for KV Cache Eviction in Streaming Video

arXiv:2609.27470 · cs.CV, cs.CL, cs.LG · Submitted 2026-09-23 · Read on arXiv

cs.CV, cs.CL, cs.LG

Submitted: 2026-09-23

Updated: 2026-09-23

Code: https://github.com/MaumAI-Company/DeltaS

Terminology

• Benchmark — 40× in this paper · explained in 2 episodes
• KV cache — 34× in this paper · explained in 3 episodes
• Accuracy — 19× in this paper · explained in 1 episode
• Aggregation — 8× in this paper · explained in 1 episode
• Adaptation — 4× in this paper · explained in 1 episode
• Hybrid Architectures — 3× in this paper · explained in 3 episodes
• Ablation — 2× in this paper · explained in 1 episode
• Foundation Model — 2× in this paper · explained in 3 episodes
• Foundation Models — 2× in this paper · explained in 7 episodes
• Redundancy — 2× in this paper · explained in 2 episodes
• Transformer — 2× in this paper · explained in 2 episodes
• Benchmarking — 1× in this paper · explained in 3 episodes
• Cosine Similarity — 1× in this paper · explained in 3 episodes
• Hallucination — 1× in this paper · explained in 3 episodes
• Quantization — 1× in this paper · explained in 12 episodes
• Sparse Attention — 1× in this paper · explained in 5 episodes
• Temporal Reasoning — 1× in this paper · explained in 2 episodes
• Tokenization — 1× in this paper · explained in 2 episodes
• Verification — 1× in this paper · explained in 2 episodes

Sources

• Stateful Token Reduction for Long-Video Hybrid VLMs
• Kimi Linear: An Expressive, Efficient Attention Architecture
• MiniMax-01: Scaling Foundation Models with Lightning Attention
• A Hippocampus for Linear Attention: An Exact Memory for What the Recurrent State Forgets
• Video-MME-v2: Towards the Next Stage in Benchmarks for Comprehensive Video Understanding
• A Simple Baseline for Streaming Video Understanding
• InfiniteVL: Synergizing Linear and Sparse Attention for Highly-Efficient, Unlimited-Input Vision-Language Models
• StreamMem: Query-Agnostic KV Cache Memory for Streaming Video Understanding

Related papers

• Loss Knows Best: Detecting Annotation Errors in Videos via Loss Trajectories
• AnchorWeave: World-Consistent Video Generation with Retrieved Local Spatial Memories
• Benchmarking the Robustness of Foundation Models for Mammography under Domain Shift
• MambaX-Net: Dual-Input Mamba-Enhanced Cross-Attention Network for Longitudinal MRI Segmentation
• TeleOCR: Navigating Document Parsing Across Digital and Camera-Captured Documents
• A Survey on Efficient Vision-Language-Action Models