Empowering Hybrid Attention Models on NPUs

arXiv:2609.32114 · cs.DC, cs.AI · Submitted 2026-09-26 · Read on arXiv

cs.DC, cs.AI

Submitted: 2026-09-26

Updated: 2026-09-26

Code: https://github.com/yinyuanzhang/HA-NPU

Project page: https://intel.github.io/intel-npu-accelerationlibrary/npu.html

Terminology

• Core — 94× in this paper · explained in 2 episodes
• Flash Linear Attention — 10× in this paper
• Transformer — 8× in this paper · explained in 2 episodes
• Decoupling — 6× in this paper · explained in 2 episodes
• Global Memory — 6× in this paper
• Benchmark — 4× in this paper · explained in 2 episodes
• Ablation — 3× in this paper · explained in 1 episode
• Edge Devices — 3× in this paper · explained in 2 episodes
• KV cache — 3× in this paper · explained in 3 episodes
• Ablation study — 2× in this paper · explained in 1 episode
• Accuracy — 2× in this paper · explained in 1 episode
• Attention Mechanism — 2× in this paper · explained in 3 episodes
• Hybrid Architectures — 2× in this paper · explained in 3 episodes
• Large Language Models (LLMs) — 2× in this paper · explained in 6 episodes
• Pruning — 2× in this paper · explained in 4 episodes
• Adaptation — 1× in this paper · explained in 1 episode
• CVF International Conference on Computer Vision — 1× in this paper
• Computer Vision and Pattern Recognition Conference — 1× in this paper
• Decode — 1× in this paper · explained in 2 episodes
• Embeddings — 1× in this paper · explained in 2 episodes
• Energy Efficiency — 1× in this paper · explained in 3 episodes
• F1 Score — 1× in this paper · explained in 3 episodes
• Granularity Mismatch — 1× in this paper · explained in 2 episodes
• Hybrid Models — 1× in this paper · explained in 2 episodes
• Kimi Delta Attention — 1× in this paper
• Sparsity — 1× in this paper · explained in 4 episodes
• Speculative Decoding — 1× in this paper · explained in 2 episodes
• Topological Order — 1× in this paper · explained in 2 episodes
• Trace — 1× in this paper · explained in 2 episodes

Sources

• Accelerating Mobile Language Model via Speculative Decoding and NPU-Coordinated Execution
• Gated DeltaNet-2: Decoupling Erase and Write in Linear Attention
• Kimi Linear: An Expressive, Efficient Attention Architecture
• Kimi K3: Open Frontier Intelligence
• On the Design of Qwen3.8-Next Architecture: Evaluation, Efficiency, and Training Stability
• Retentive Network: A Successor to Transformer for Large Language Models
• T-MAN: Enabling End-to-End Low-Bit LLM Inference on NPUs via Unified Table Lookup
• Agent.xpu: Efficient Scheduling of Agentic LLM Workloads on Heterogeneous SoC
• PowerInfer-2: Fast Large Language Model Inference on a Smartphone
• Parallelizing Linear Transformers with the Delta Rule over Sequence Length

Related papers

• iScheduler: Reinforcement Learning-Driven Continual Optimization for Large-Scale Resource Investment Problems
• SAMM: Sharded Automated Market Maker
• InferScale: GPU-Native KV Injection for Personalized LLM Serving
• Vigil: Accountable Liveness against Selective Silence
• Steelhead: Interleaving Partially Synchronous and Asynchronous Commit Rules on a Shared DAG
• Pushing CPU Speech Synthesis to the Wall: Extreme Inference Tuning under Serverless Architecture and Billing