Audio Token Attention Is Predictable Before the Language Model Runs
cs.SD, cs.AI, cs.CL
Submitted: 2026-09-30
Updated: 2026-09-30
Code: https://github.com/DabDans/HeadRouter
Project page: https://audio-triage.github.io
Terminology
Sources
- Towards Audio Token Compression in Large Audio Language Models
- HeadRouter: Dynamic Head-Weight Routing for Task-Adaptive Audio Token Pruning in Large Audio Language Models
- Locality Matters for Training-Free Audio Token Compression in Audio-Language Models
- AudioKV: KV Cache Eviction in Efficient Large Audio Language Models
- Entropy-based Coarse and Compressed Semantic Speech Representation Learning
- Qwen2.5-Omni Technical Report
- Qwen3-Omni Technical Report
- Qwen2-Audio Technical Report
- Voxtral
- Phi-4-Mini Technical Report: Compact yet Powerful Multimodal Language Models via Mixture-of-LoRAs
- AudioMarathon: A Comprehensive Benchmark for Long-Context Audio Understanding and Efficiency in Audio LLMs
- Qwen-Audio: Advancing Universal Audio Understanding via Unified Large-Scale Audio-Language Models
- Learning to Predict Middle-Layer Attention in MLLMs for Visual Token Pruning
- SeaLLMs-Audio: Large Audio-Language Models for Southeast Asia
- MiDashengLM: Efficient Audio Understanding with General Audio Captions
Related papers
- Few-Shot Open-Set Audio Classification via Transductive Prototype Refinement and Class Logit Enhancement
- Spectral Masking and Interpolation Attack (SMIA): A Black-box Adversarial Attack against Voice Authentication and Anti-Spoofing Systems
- AVMeme Exam: A Multimodal Multilingual Multicultural Benchmark for LLMs' Contextual and Cultural Knowledge and Thinking
- SoundWeaver: Compositional Warm-Starting for Text-to-Audio Diffusion Serving
- WASIL: In-the-Wild Arabic Spoken Interactions with LLMs
- Efficient Audiovisual Speech Processing via MUTUD: Multimodal Training and Unimodal Deployment