HeadGuard: Selective Head Protection for Low-Bit VLM KV-Cache Quantization
cs.LG, cs.CV
Submitted: 2026-09-18
Updated: 2026-09-18
Terminology
Sources
- Pixtral 12B
- GQA: Training Generalized Multi-Query Transformer Models from Multi-Head Checkpoints
- Qwen2.5-VL Technical Report
- Microsoft COCO Captions: Data Collection and Evaluation Server
- Not All Heads Matter: A Head-Level KV Cache Compression Method with Integrated Retrieval and Reasoning
- Gemma 3 Technical Report
- Gemma 4 Technical Report
- KVQuant: Towards 10 Million Context Length LLM Inference with KV Cache Quantization
- KVmix: Gradient-Based Layer Importance-Aware Mixed-Precision Quantization for KV Cache
- Visual Instruction Tuning
- Improved Baselines with Visual Instruction Tuning
- MMBench: Is Your Multi-modal Model an All-around Player?
- KIVI: A Tuning-Free Asymmetric 2bit Quantization for KV Cache
- Learn to Explain: Multimodal Reasoning via Thought Chains for Science Question Answering
- Are Sixteen Heads Really Better than One?
- Towards VQA Models That Can Read
- Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolution
- MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGI
- InternVL3: Exploring Advanced Training and Test-Time Recipes for Open-Source Multimodal Models
- RateQuant: Optimal Mixed-Precision KV Cache Quantization via Rate-Distortion Theory
Related papers
- Polynomial-Augmented Neural Networks (PANNs) with Weak Orthogonality Constraints for Enhanced Function and PDE Approximation
- AIRL-S: Unifying Reinforcement Learning and Search-Based Test-Time Scaling via Adversarial Inverse Reinforcement Learning
- Transformers as Bayesian In-Context Experimenters: Smoothness-Adaptive Efficient ATE Estimation
- Convergence issues in Relational Concept Analysis based on AOC-posets
- Beliefs Beyond Posteriors: Local-Consistency Optimisation for Bayesian Neural Networks
- Understanding Diffusion Models via Ratio-Based Function Approximation with SignReLU Networks