Near-Oracle KV Selection via Pre-hoc Sparsity for Long-Context Inference
cs.LG, cs.AI, cs.IT, math.IT
Submitted: 2026-02-09
Updated: 2026-09-23
Terminology
Sources
- GPT-4o System Card
- DeepSeek-V3 Technical Report
- Multi-modal Retrieval Augmented Multi-modal Generation: Datasets, Evaluation Metrics and Strong Baselines
- Multimodal Reasoning Agent for Zero-Shot Composed Image Retrieval
- A Survey of Automatic Evaluation Methods on Text, Visual and Speech Generations
- MLLM-Guided VLM Fine-Tuning with Joint Inference for Zero-Shot Composed Image Retrieval
- LLaMA: Open and Efficient Foundation Language Models
- Linformer: Self-Attention with Linear Complexity
- VORTA: Efficient Video Diffusion via Routing Sparse Attention
- HashEvict: A Pre-Attention KV Cache Eviction Strategy using Locality-Sensitive Hashing
- Mistral 7B
- Post-Training Sparse Attention with Double Sparsity
- Training Verifiers to Solve Math Word Problems
Related papers
- Polynomial-Augmented Neural Networks (PANNs) with Weak Orthogonality Constraints for Enhanced Function and PDE Approximation
- AIRL-S: Unifying Reinforcement Learning and Search-Based Test-Time Scaling via Adversarial Inverse Reinforcement Learning
- Transformers as Bayesian In-Context Experimenters: Smoothness-Adaptive Efficient ATE Estimation
- Convergence issues in Relational Concept Analysis based on AOC-posets
- Beliefs Beyond Posteriors: Local-Consistency Optimisation for Bayesian Neural Networks
- Understanding Diffusion Models via Ratio-Based Function Approximation with SignReLU Networks