DeepSeek-V4.1-Flash: Pushing the Limits of KV Cache Compression

summary

The gist

DeepSeek-V4.1-Flash is a 552B-parameter multimodal Mixture-of-Experts model that cuts its global KV cache footprint to 890 bytes per token—roughly one quarter of DeepSeek-V4-Flash's—while

In short

DeepSeek-V4.1-Flash is a 552B-parameter multimodal MoE model that cuts global KV cache to 890 bytes per token, about a quarter of its predecessor's, while improving performance. It combines a 20-layer causal encoder with a 20-layer decoder, compressed sparse attention with cross-layer KV reuse, FP4 main-KV quantization, and bounded-replay SWA recovery, reaching 74.2% on DeepSWE v1.1.

Key concepts

Causal Encoder-Decoder (CED): A 40-layer backbone split into a 20-layer causal encoder and a 20-layer decoder, where decoder global KV is projected from the encoder's final hidden state. This lets prefill activate only 8B parameters per token versus 16B during decode, and halves prefill computation from O(NL) to roughly O(NL/2).
Compressed Sparse Attention 2 (CSA2): An attention scheme that shares main KV and indexer K across layers and lets layers reuse Top-K indices. Each layer runs in Full Mode (computes its own KV and indices), Reindex Mode (reuses KV but rescores with its own query), or Reuse Mode (reuses both KV and indices with no indexer computation).
MXFP4 KV quantization: The main KV cache is stored in the OCP-standard MXFP4 format, using E2M1 values with one E4M3 scale per 16 channels, quantized after RoPE. The more quantization-sensitive sliding-window-attention KV stays in FP8, keeping accuracy while shrinking the dominant cache.
SWA Bounded Replay: Instead of persisting sliding-window KV to SSD, the system holds it in a distributed host-memory pool with a short TTL. On a miss, it replays only the most recent n_win tokens rather than L times n_win, accepting approximate states and cutting the persistent KV footprint to about one eighth of the prior model's.

Terminology used across episodes

Reinforcement Learning (RL) — explained in 25 episodes
Retrieval-Augmented Generation (RAG) — explained in 20 episodes
Sample Complexity — explained in 20 episodes
Flow Matching — explained in 18 episodes
Federated Learning — explained in 17 episodes
Quantum Fisher Information (QFI) — explained in 17 episodes

This episode discusses

The paper

DeepSeek-V4.1-Flash: Pushing the Limits of KV Cache Compression · Read on arXiv

Anyi Xu, B. Li, Bangcai Lin, Bing Xue, BingCheng Xian, Bingzheng Xu, Bochao Wu, Bowei Zhang, Boyi Deng, C.C. Yu, Chao Jin, Chaofan Lin, Chen Dong, Chenbing Wang, Chenfan Feng, Chengda Lu, Chenggang Zhao, Chengqi Deng, Chengyuan Zhang, Chenhao Xu, Chenqi Zhao, Chenze Shao, Chuhao Wang, Chuqi Zhang, Damai Dai, Dejian Yang, Deli Chen, Di Huang, Di Wu, Donghao Li, Erhang Li, Eric Fu, F. Zhou, Fangwei Zhou, Fangyun Lin, Fangzhou Yuan, Feiyu Xia, Fucong Dai, Guangbo Hao, Guanglin Li, Guanting Chen, Guoai Cao, Guofan Fan, Guolai Meng, Guowei Li, Haichuan Zhang, Haiyang Ma, Haiyang Shen, Han Li, Han Yu, Han Zhang, Hangyuan Deng, Hanwei Xu, Hanxiang Xu, Hanxun Zhong, Hao Guo, Hao Jiang, Hao Li, Hao Qin, Haodong Wen, Haofen Liang, Haofeng Huang, Haohua Liu, Haoling Zhang, Haoming Luo, Haoran Yang, Haotian Xu, Haotian Yuan, Haoting Huang, Haowen Luo, Haoyang Cai, Haoyu Chen, Haozhe Ji, Hengran Zhang, Hengrui Wang, Hengxu Wu, Honghui Ding, Hongxuan Tang, Huadong Wang, Huanqi Cao, Huazuo Gao, Hui Qu, Hui Zeng, J. Yang, J.H. Jin, J.H. Zhang, J.X. Zou, Jia Yu, Jiahui Zhou, Jiajun Chen, Jialiang Huang, Jialin Zhao, Jiamin Tang, Jian Zhou, Jianan Tong, Jianwen Li, Jiaqi Zhu, Jiarui Wang, Jiasheng Ye, Jiashi Li, Jiaxin Xu, Jiaying Ding, Jibai Lu, Jiewen Hu, Jin Yan, Jincheng Zhai, Jingchang Chen, Jingcheng Hu, Jingli Zhou, Jingsheng Xu, Jingting Xiang, Jingyan Yun, Jingyang Yuan, Jingyuan Cheng, Jinhua Zhu, Jinpeng Wang, Jinyi Chen, Jinyi Hu, Jiping Yu, Jueliang Guo, Junbo Pei, Junbo Sun, Junguang Jiang, Junjie Qiu, Junkang Zhou, Junqi Liu, Junren Li, Junxian Li, Junxiao Song, Junyi Guo, Kai Dong, Kaifeng Chen, Kaige Gao, Kang Guan, Kangdong Yuan, Ke Hong, Ke Xu, Kefan Zhao, Kexin Ji, Kexin Zhang, Kexing Zhou, Kuai Yu, Lan Zhang, Lean Wang, Lecong Zhang, Lei Wang, Letian Gao, Liang Zhao, Liansheng Xu, Lihua Guo, Lingxiao Luo, Lingyue Fu, Litao Deng, Litong Wang, Liyue Zhang, Longhao Chen, Lu Chen, Luotian Huang, Luyao Ma, Luyao Wang, M.S. Di, Max Mei, Menghao Ye, Miao Cui, Mingchuan Zhang, Minghua Zhang, Minghui Tang, Mingjing Zhang, Mingqi Wei, Mingshu Chen, Mingxing Liu, Mingxu Zhou, Mingyu Xu, Mingyu Yang, Mingze Wang, Muyang Chen, Ni Shentu, Ning Wang, Niufang Ning, Panpan Huang, Peixin Cong, Peiyi Wang, Peiyuan Xin, Pengfei Ren, Pengfei Yan, Pengle Zhang, Qi Kang, Qi Tang, Qiancheng Wang, Qiang Li, Qihao Zhu, Qingyang Li, Qinyu Chen, Qiushi Du, Qizhou Guo, Rongxian Xu, Rui Ding, Rui Hu, Rui Tian, Rui Yu, Ruidong Zhu, Ruifan Xu, Ruihan Yang, Ruihang Xia, Ruijie Lu, Ruilin Geng, Ruipeng Hong, Ruiqi Ge, Ruisong Zhang, Ruize Sun, Ruizhe Pan, Runji Wang, Runqian Chen, Runxin Xu, Ruohong Tian, Ruomeng Shen, Ruoyu Zhang, Ryan X., S.H. Liu, Shanghao Lu, Shangyan Zhou, Shanhuang Chen, Shaofei Cai, Shaoheng Nie, Shaoyuan Chen, Shengding Hu, Shengkai Lin, Shengwen Ran, Shengyu Liu, Shengyuan Jia, Shi Bai, Shi Feng, Shicheng Xu, Shichun Liu, Shiqiang Hu, Shirong Ma, Shiyu Wang, Shiyuan Feng, Shufan Gong, Shuhan Lin, Shuiping Yu, Shunfeng Zhou, Shuo Yang, Shuomeng Wang, Shuting Guo, Shuting Pan, Shuying Yu, Sinuo Cao, Siyi Lin, Sizhe Chen, Songyang Chen, Songyang Zhou, Tao Ni, Tao Yun, Tian Jin, Tian Pei, Tian Ye, Tianle Lin, Tianran Ji, Tianyi Cui, Tianyuan Yue, Tingting Yu, Tongrui Xiong, Wangding Zeng, Wei Liu, Wei Zhang, Weibin Xu, Weihao Zeng, Weilin Zhao, Wen Liu, Wenfeng Liang, Wenjie Pang, Wenjing Luo, Wenjing Yao, Wenjun Gao, Wenkai Shao, Wenkai Yang, Wenli Zhang, Wenlu Wang, Wenlve Huang, Wenqian Yan, Wentao Zhang, Xi Gao, Xiang He, Xiang Li, Xiangli Li, Xiangwen Wang, Xiangying Zhang, Xiankui Wei, Xiao Bi, Xiaodong Liu, Xiaohan Wang, Xiaojian Qu, Xiaokang Chen, Xiaokang Zhang, Xiaotao Nie, Xiaoyao Zou, Xiaoyuan Li, Xicheng Guo, Xieting Chu, Xin Cheng, Xin Liu, Xin Xie, Xinbo Xu, Xingchao Liu, Xingchen Liu, Xingkai Yu, Xingyou Li, Xintong Yao, Xinyang Chen, Xinyong Jiang, Xinyu Yang, Xu Chen, Xuanyu Wang, Xubei Zhong, Xuecheng Su, Xuejie Liu, Xuheng Lin, Xujie Fan, Xuncheng Zhao, Xuwei Fu, Y.C. Yan, Y.H. Jiang, Y.T. Wu, Y.W. M., Y.Z. Wang, Yafei Gao, Yang Yang, Yang Zhang, Yanru Ma, Yanwen Huang, Yao Li, Yao Meng, Yao Zhao, Yaofeng Sun, Yaohui Wang, Yaoyang Ye, Yehang Yin, Yexinrui Wu, Yi Qian, Yi Tao, Yi Yu, Yichao Zhang, Yichen Jiang, Yicheng Wang, Yifan Ding, Yifan Shi, Yifeng Peng, Yifeng Zhai, Yijia Wu, Yiliang Xiong, Yilun Wang, Ying He, Ying Zhou, Yingjia Luo, Yinmin Zhong, Yiping Wang, Yisong Wang, Yixiang Zhang, Yixiao Chen, Yixuan Tan, Yixuan Wei, Yiyang Ma, Yiyao Yang, Yiyuan Liu, Yizai Cai, Yizhen Wei, Yizhi Wang, Yonglun Yang, Yongqi Zhuo, Yongqiang Guo, Yongtong Wu, Yu Wu, Yu Zhang, Yuan Bian, Yuan Cheng, Yuan Ou, Yuan Sun, Yuanfan Xu, Yuanhang Sun, Yuanhao Li, Yuchen Liu, Yuchen Yao, Yudong Han, Yuduan Wang, Yuhan Wu, Yuhao Meng, Yuheng Zou, YuKun Li, Yunchuan Wang, Yunfan Xiao, Yunfan Xiong, Yupeng Chen, Yuqian Cao, Yuqian Wang, Yuqing Chen, Yushun Zhang, Yutong Lin, Yuwei Xiao, Yuxian Gu, Yuxiang Chen, Yuxiang Huang, Yuxiang Luo, Yuxiang You, Yuxin Chen, Yuxin Xiang, Yuxuan Liu, Yuxuan Zhou, Yuyang Zhou, Yuzhe Guo, Yuzhen Huang, Yuzhuo Bai, Z.Y. Z., Zanlin Ni, Zehao Wang, Zehua Zhao, Zehui Ren, Zejun Zhao, Zhangli Sha, Zhanying Wang, Zhaochen Zhang, Zhaoshuai Du, Zhe Fu, Zhean Xu, Zhenda Xie, Zheng Liu, Zhengyan Zhang, Zhenhua Dong, Zhewen Hao, Zhibang Wang, Zhibin Gou, Zhicheng Ma, Zhihao Li, Zhihong Shao, Zhihuan Huang, Zhijie Li, Zhirui Lu, Zhixian Huang, Zhixuan Chen, Zhixuan Pan, Zhiyu Wu, Zhizhou Ren, Zhu He, Zhuoshu Li, Zhuping Zhang, Zian Xu, Zihao Wang, Zihui Gu, Zijia Zhu, Zili Zhang, Zilin Li, Zilong Hou, Zilong Lyu, Ziqiao Wang, Ziwei Xie, Ziya Zhang, Ziyi Gao, Zizheng Pan, Zonglin Li, Zongqing Yao, Zui Chen, Zuofan Wu, Chenchen Ling, Chengyu Hou, Chong Chen, D. Li, Di Qi, Dongjie Ji, Fang Wei, Fanyi Xia, Fei Xie, Feiyi Tan, Hailong Guo, Haiyan Zhai, Hui Zhou, Huihui Tan, Huijie Li, Jia Luo, Jia Song, Jialu Cai, Jian Liang, Jiangting Zhou, Jiaqi Gao, Jiayi Shao, Jie Chen, Jieyu Yang, Jin Chen, Jingde Zhang, Jingzi Zhou, Jinqian Wang, Jinyang Liu, JinZhao Sun, Junhua Ling, Junmin Zheng, Kaicheng Yang, Ke Xu, Le Su, Leyi Xia, Liangfeng Ding, Lin Zhuo, Linwang Ma, Linyan Zhu, Liyu Cai, Luqi Yao, M.K. Zhang, Meng Li, Miao Lin, Miaojun Wang, Min Zhang, Mingming Li, Mingming Wang, Mingze Yin, Minmin Han, Nan Cao, Ning Wang, Ningxin Ma, Panpan Wang, Peihan Lin, Peng Sun, Peng Zhang, Qian Ying, Qiang Xiang, Qiao Wang, Qingmiao Mao, Qiwei Jiang, Rongli Jin, Ruyi Chen, Sha Tao, Shangmian Sun, Shaoqing Wu, Shichao Zou, Si Lei, Tianyang Zhang, Tianyu Sun, Tingting Yin, W.L. Xiao, Wei An, Wei Li, Wei Wang, Weiwei Lin, Wenqing Hou, X. Lin, Xiangfei Meng, Xianzhu Huang, Xiao Peng, Xiaoqian Li, Xiaoting Zhang, Xiaowen Sun, Xiaoxiang Wang, Xiaoyu Ye, Xinrou Zhang, Xinyu Zhang, Xue Cao, Xueyin Chen, Yanan Zhou, Yanhong Xu, Yao Xia, Yao Xu, Yi Shao, Yihong Zhang, Yiling Ma, Ying Tang, Yining Lou, Yiru Chen, Yishi Piao, Yixuan Chen, Yong Xiong, Yuchen Xuan, Yuehan Yang, Yuer Xu, Yukun Zha, Yunxian Ma, Yuping Lin, Yuting Yan, Yutong Xie, Yuwen Sheng, Yuxuan Zhu, Zekai Zhang, Zhe Ju, Zhenzhen Lin, Zheren Gao, Zheyang Sun, Zhigang Yan, Zhongyu Wu, Zi Wang, Zihua Qu, Ziling Yan, Ziyi Wan

DeepSeek-AI

Transcript

Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.

Tom: We're looking at the paper: "DeepSeek-V4.1-Flash: Pushing the Limits of KV Cache Compression".

Jane: DeepSeek-V4.1-Flash is a 552B-parameter multimodal Mixture-of-Experts model that cuts its global KV cache footprint to 890 bytes per token—roughly one quarter of DeepSeek-V4-Flash's—while delivering better overall performance,

Tom: First, who's behind it and why it matters.

Title and authors: Tom: We're back with "DeepSeek-V4 point 1-Flash: Pushing the Limits of KV Cache Compression," and this time we're reading the cover properly <ref:2609.19969#pg1,DeepSeek-V4.1-Flash: Pushing the Limits of KV Cache Compression>. The title comes from DeepSeek-AI, and Jane, a listener just got in the car, so start with the thing the title assumes everyone knows.

Jane: The KV cache is the model's working memory of the conversation so far, stored as key and value vectors so it doesn't reread the whole prompt every time it produces a word. This model supports contexts up to one million tokens, and that memory grows with the context <ref:2609.19969#pg1,support for contexts of up to one million tokens>. So the title is making a promise about memory, and that's a strange thing to put on a cover.

Tom: It's the right thing to put there, because the authors say the bottleneck to cheaper deployment is exactly this: prefill is still expensive, and the caches strain HBM and SSD capacity and data-transfer bandwidth <ref:2609.19969#pg1,prefill remains computationally expensive, and large KV caches continue to strain HBM and SSD capacity and data-transfer bandwidth>. So the title is a claim about where the cost sits, not about what the model can do.

Lu: And the framing is deliberate, because the model underneath is a 552B-parameter multimodal Mixture-of-Experts <ref:2609.19969#pg1,a multimodal Mixture-of-Experts (MoE) model with 552B backbone parameters>. A title about compression on a model that size is really a title about deployment economics — the compression is what makes the size affordable to serve.

Meng: That's the part I care about. When I size a deployment, the question is how much memory each live session holds, and a paper that leads with bytes per token is speaking my language. The catch is that a title tells you the target, not whether they hit it, and I want the actual number before I believe the word "limits."

Lalam: The paper does give a number: it cuts the global KV cache footprint to eight hundred ninety bytes per token, roughly a quarter of the previous Flash model's <ref:2609.19969#pg1,These designs reduce its global KV cache footprin>. I'd hold the word "limits" loosely, though, because frontiers move. What's solid is that eight hundred ninety is a measured figure you can plan against.

Jane: So the title stacks two claims — a big model and a small memory per token — and Tom, you're saying those aren't separate achievements.

Tom: They're the same achievement. The model is large and the per-token memory is small, and those only coexist because of the architecture. That's what the summary lays out, and it's where we go next.

The paper's summary: Jane: We've got the title and the authors, and the headline number is eight hundred ninety bytes of global KV cache per token. Now let's open the summary the authors write of their own paper for "DeepSeek-V4 point 1-Flash: Pushing the Limits of KV Cache Compression" and see what they claim <ref:2609.19969#pg1>. Tom, what are the pieces?

Tom: Three pieces. First, a Causal Encoder-Decoder backbone — forty layers, split into a twenty-layer causal encoder followed by a twenty-layer decoder <ref:2609.19969#pg7,organized into a 20-layer causal encoder followed by a 20-layer decoder>. Second, Compressed Sparse Attention two which reuses KV and indexer keys across layers <ref:2609.19969#pg1>. Third, FP4 quantization of the main KV cache. Together they give the eight hundred ninety bytes.

Jane: Slow down on the encoder-decoder split, because that's the piece a listener would trip over. Why does cutting the model in half save anything?

Tom: Because during prefill, when the model reads the whole prompt, it only activates 8B parameters per token, while decode uses 16B <ref:2609.19969#pg1,activates 16B parameters per token during decode but only 8B parameters during prefill>. The decoder's global KV is projected from the final encoder hidden state instead of being computed layer by layer, and for a long sequence that roughly halves the prefill computation <ref:2609.19969#pg9,CED reduces the prefill complexity from>.

Lu: That's the right place to save, because agent workloads are input-heavy — the prompt is enormous and the answer is short <ref:2609.19969#pg1,The widespread adoption of long-horizon agents has made model workloads increasingly inputheavy>. Halving prefill is halving the dominant cost. The decoder still pays full price per generated token, and that's the correct trade.

Meng: The number I'd flag is the cache itself. At one million tokens, eight hundred ninety bytes per token is under a gigabyte of global KV for one sequence, against roughly four times that before. That's the difference between a session fitting on one node and spilling across several, and it's the number I'd put in a capacity plan.

Lalam: There's a multimodal claim buried in the summary too — it's pretrained on a 45T-token multimodal corpus <ref:2609.19969#pg1,on a multimodal corpus comprising 45T tokens>. Worth saying, because most people picture this compression work as text-only, and images are in the training mix from the start.

Jane: And the caveat, because every summary has one?

Tom: The authors say it themselves: a gap with giant models remains on science-oriented agentic tasks that need expert-level domain knowledge <ref:2609.19969#pg6,a gap with giant models remains on science-oriented agentic tasks>. So the compression is real, but it doesn't hand you a giant model's knowledge on hard science. Next, the paper goes further and proposes improvements on top of its own design.

The paper's improvements: Jane: We've covered the title and the summary of "DeepSeek-V4 point 1-Flash: Pushing the Limits of KV Cache Compression" — the split backbone, the cross-layer reuse, the eight hundred ninety bytes. Now the part I find most interesting: the paper doesn't stop at describing the design, it proposes improvements on top of it. Tom, where does it start?

Tom: It starts with the cache layout. CSA2 already shares main KV and indexer keys across layers, but Reindex layers still rescore the reused keys with their own query <ref:2609.19969#pg11,reuses the most recent available main KV from a preceding layer>. The improvement is to store one FP4 main KV block per group plus per-layer low-rank RoPE deltas, so those layers don't reread full latents and reads stay at eight hundred ninety bytes per token.

Lu: The second is the sparse indexer. The paper fixes a candidate pool of up to sixteen thousand three hundred eighty-four positions that later layers search <ref:2609.19969#pg12,number of positions scored per query by each subsequent indexer is bounded independently of context length>. The improvement makes the pool adaptive — size it from the Full-Mode indexer's score entropy, so a sharp distribution gets a small pool and a flat one gets a large pool. That trims the constant per-query cost without losing recall on ambiguous queries.

Meng: Then the deployment side, which is where I live. SWA KV stops being persisted to SSD and sits in a distributed host-memory pool with a short TTL, and misses are recovered by SWA Bounded Replay, which replays only the most recent n win tokens <ref:2609.19969#pg19,SWA KV is no longer cached in the persistent KV cache>. The improvement adds a check at resumption: compare the replayed SWA KV against a cheap low-rank sketch, and trigger a full replay only when divergence crosses a threshold.

Jane: That one turns an admitted weakness into something watched. The authors say the robustness boundaries of bounded replay haven't been fully characterized, so making the failure bounded and visible is a real step.

Lalam: There's a scheduling improvement I like as well. DSpark's scheduler picks a verification length per request to maximize throughput, but the effort scalar already predicts output length <ref:2609.19969#pg30,the public API exposes three preset reasoning-effort tiers>. Feeding the effort tier in means low-effort requests use shorter verification and high-effort ones keep speculation budget for long trajectories.

Tom: And one on storage policy. The paper keeps global KV with a guaranteed lifetime of at least seventy-two hours under LRU <ref:2609.19969#pg19,SWA KV is no longer cached in the persistent KV cache>. The improvement replaces that with a value-of-reuse eviction policy — score prefixes by observed downstream reuse, evict low-value ones early, and free capacity for the SWA pool's short turnover.

Jane: So all of these are the same move: making the compression adaptive instead of fixed. Which raises the question of what it all adds up to.

Conclusion: Tom: We've gone through the title, the summary, and the proposed improvements to "DeepSeek-V4 point 1-Flash: Pushing the Limits of KV Cache Compression" <ref:2609.19969#pg1,DeepSeek-V4.1-Flash: Pushing the Limits of KV Cache Compression>. Before we say goodbye, let's hear what each of you takes away. Lu, you first.

Lu: The idea worth keeping is that compression and capability get designed together instead of traded off. The eight hundred ninety bytes per token isn't a storage trick on the side — it's what lets a 552B model run long agent sessions at a cost that makes sense. What I'd want next is evidence that the adaptive pieces hold under real distribution shift, not just the synthesized tasks <ref:2609.19969#pg25,we refrain from introducing novel post-training algorithms>.

Meng: For me it's a deployment paper. The number I'll carry is the cache footprint — eight hundred ninety bytes per token, roughly a quarter of the previous model's — because that decides how many sessions fit on a node <ref:2609.19969#pg1,These designs reduce its global KV cache footprin>. The bounded-replay check is what I'd ship first, since it turns an approximate recovery into something I can actually monitor.

Lalam: I'd flag the honesty of the limitations. The authors say plainly there's still a gap with giant models on science-oriented agentic tasks <ref:2609.19969#pg6,a gap with giant models remains on science-oriented agentic tasks>. That's the sentence I'd keep — the compression is real and well-evidenced, but it doesn't turn a smaller model into a giant one on hard domain knowledge. I'm fairly confident that's the right read.

Jane: So the paper's contribution is a set of numbers you can plan against, plus a clear statement of where it stops. That combination is genuinely useful to anyone running these models.

Tom: It is, and the improvements are the part that gets tested next. That's "DeepSeek-V4 point 1-Flash: Pushing the Limits of KV Cache Compression" — thanks for listening, and we'll see you with the next paper <ref:2609.19969#pg1,DeepSeek-V4.1-Flash: Pushing the Limits of KV Cache Compression>.

More episodes

← Home