EXAM squared: tending udio Understanding in ultilingual and ultimodal Analysis
cs.SD, cs.AI
Submitted: 2026-08-24
Updated: 2026-08-27
Comments: 8 pages, 2 figures
Code: https://github.com/werywjw/EXAM-2
License: http://creativecommons.org/licenses/by/4.0/
The gist: Recent large audio language models (LALMs) have achieved impressive progress in audio understanding.
Terminology
Abstract
Recent large audio language models (LALMs) have achieved impressive progress in audio understanding. However, existing evaluations remain largely constrained to English and narrow audio domains. Prior benchmarks typically focus on a single audio modality, i.e., speech, sound, or music, limiting the systematic investigation into how these models generalize across diverse visual scenarios. In this paper, we introduce EXAM squared, a benchmark for multilingual and multimodal audio understanding spanning six languages and multiple modalities, including speech, sound, music, mixed-audio settings, and visual images. By incorporating visual information alongside heterogeneous audio inputs, EXAM squared enables more realistic evaluation of scene-aware audio reasoning and cross-modal comprehension. EXAM squared comprises 5,667 multiple-choice questions, 22,614 image instances, and 135,684 multilingual translations. We evaluate state-of-the-art open-source and proprietary LALMs as well as multimodal LLMs, revealing substantial performance gaps in multilingual and cross-modal understanding. Furthermore, we propose Gemma3n-EXAM squared, a lightweight fusion-model fine-tuned on EXAM squared-train, achieves up to 12.4% improvement in multilingual settings and 21.7% gains in multimodal evaluation over a strong baseline. Empirical results establish EXAM squared as a challenging benchmark and pioneer future multilingual and multimodal audio intelligence research.
Sources
- Ming-Omni: A Unified Multimodal Model for Perception and Generation
- UNO-Bench: A Unified Benchmark for Exploring the Compositional Law Between Uni-modal and Omni-modal in Omni Models
- Qwen2-Audio Technical Report
- Audio Flamingo 2: An Audio-Language Model with Long-Audio Understanding and Expert Reasoning Abilities
- Measuring Audio's Impact on Correctness: Audio-Contribution-Aware Post-Training of Large Audio Language Models
- Audio Flamingo: A Novel Audio Language Model with Few-Shot Learning and Dialogue Abilities
- Baichuan-Omni-1.5 Technical Report
- MMAR: A Challenging Benchmark for Deep Reasoning in Speech, Audio, Music, and Their Mix
- ERM-MinMaxGAP: Benchmarking and Mitigating Gender Bias in Multilingual Multimodal Speech-LLM Emotion Recognition
- Qwen3.5-Omni Technical Report
- MMSU: A Massive Multi-task Spoken Language Understanding and Reasoning Benchmark
- MuChoMusic: Evaluating Music Understanding in Multimodal Audio-Language Models
- Qwen3-Omni Technical Report
Related papers
- Few-Shot Open-Set Audio Classification via Transductive Prototype Refinement and Class Logit Enhancement
- Spectral Masking and Interpolation Attack (SMIA): A Black-box Adversarial Attack against Voice Authentication and Anti-Spoofing Systems
- AVMeme Exam: A Multimodal Multilingual Multicultural Benchmark for LLMs' Contextual and Cultural Knowledge and Thinking
- SoundWeaver: Compositional Warm-Starting for Text-to-Audio Diffusion Serving
- WASIL: In-the-Wild Arabic Spoken Interactions with LLMs
- Efficient Audiovisual Speech Processing via MUTUD: Multimodal Training and Unimodal Deployment