Interface-Aware KV Cache Quantization for Dense On-Chip NVM in Long-Context LLM Decoding

arXiv:2609.05764 · cs.AR, cs.AI · Submitted 2026-09-04 · Read on arXiv

cs.AR, cs.AI

Submitted: 2026-09-04

Updated: 2026-09-04

Comments: IEEE/ACM International Conference on Computer-Aided Design (ICCAD 2026)

DOI: 10.1145/3831252.3845860

License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/

Terminology

• KV cache — 7× in this paper · explained in 3 episodes
• Quantization — 4× in this paper · explained in 12 episodes
• Accuracy — 2× in this paper · explained in 1 episode
• Large Language Model (LLM) — 1× in this paper · explained in 3 episodes

Related papers

• WitCert: Sound Runtime Risk Observability and Gating for KV-Cache Quantization
• Golden Ruler: A Numeric Format Catalog with Bit-Exact Conformance Vectors for FP8, BF16, MXFP4, and Microscaling Formats
• PoisonCap: Efficient Hierarchical Temporal Safety for CHERI
• Provisioning to Runtime Optimization of a 100 MW-Scale AI Cluster
• Bit-Accurate Modeling of GPU Matrix Multiply-Accumulate Units: Demystifying Numerical Discrepancy and Accuracy
• Optimizing Polynomial Multiplication and Fixed-Weight Sampling for HQC on ARM Cortex-M4