HySparse2: Hybrid Sparse Attention with Two-Level KV Sharing
cs.CL
Submitted: 2026-09-22
Updated: 2026-09-22
License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
Terminology
Sources
- gpt-oss-120b & gpt-oss-20b Model Card
- Program Synthesis with Large Language Models
- DFlash: Block Diffusion for Flash Speculative Decoding
- Evaluating Large Language Models Trained on Code
- Generating Long Sequences with Sparse Transformers
- Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challenge
- Training Verifiers to Solve Math Word Problems
- DeepSeek-V3.2: Pushing the Frontier of Open Large Language Models
- DeepSeek-V4: Towards Highly Efficient Million-Token Context Intelligence
- What is Wrong with Perplexity for Long-context Language Modeling?
- SeerAttention: Learning Intrinsic Sparse Attention in Your LLMs
- HySparse: A Hybrid Sparse Attention Architecture with Oracle Token Selection and KV Cache Sharing
- Are We Done with MMLU?
- Measuring Massive Multitask Language Understanding
- Measuring Mathematical Problem Solving With the MATH Dataset
- RULER: What's the Real Context Size of Your Long-Context Language Models?
- CMMLU: Measuring massive multitask language understanding in Chinese
- DeepSeek-V2: A Strong, Economical, and Efficient Mixture-of-Experts Language Model
- Hidden Decoding at Scale: Latent Computation Scaling for Large Language Models
- NoLiMa: Long-Context Evaluation Beyond Literal Matching
Related papers
- Exploring Solution Divergence and Its Effect on Large Language Model Problem Solving
- Ishigaki-IDS-Bench: A Benchmark for Generating Information Delivery Specification from BIM Information Requirements
- Subliminal Steering: Stronger Encoding of Hidden Signals
- MedStruct-S: A Benchmark for Key Discovery, Key-Conditioned QA and Semi-Structured Extraction from OCR Clinical Reports
- The End of Transformers? On Challenging Attention and the Rise of Sub-Quadratic Architectures
- Untangling the Mechanisms of Misleading Context in Medical Question Answering