Silence is Golden: Mitigating Hallucinations in Large Audio-Language Models via Layer-Weighted Vector Steering

arXiv:2510.12851 · cs.SD, cs.LG, eess.AS · Submitted 2025-10-14 · Read on arXiv

cs.SD, cs.LG, eess.AS

Submitted: 2025-10-14

Updated: 2026-08-31

Terminology

• Steering — 101× in this paper · explained in 2 episodes
• Hallucination — 29× in this paper · explained in 3 episodes
• Adaptive Vector Steering — 22× in this paper
• Benchmark — 8× in this paper · explained in 2 episodes
• Adaptive Steering — 7× in this paper · explained in 1 episode
• Accuracy — 6× in this paper · explained in 1 episode
• Cosine Similarity — 6× in this paper · explained in 3 episodes
• Audio Question Answering — 2× in this paper
• Automatic Speech Recognition — 2× in this paper
• Automatic Speech Recognition (ASR) — 2× in this paper · explained in 2 episodes
• Fine-Tuning — 2× in this paper · explained in 5 episodes
• Grounding — 2× in this paper · explained in 3 episodes
• Hallucination Detection — 2× in this paper · explained in 2 episodes
• Hallucination Mitigation — 2× in this paper · explained in 2 episodes
• RAG — 2× in this paper · explained in 3 episodes
• Transformer — 2× in this paper · explained in 2 episodes
• Activation Steering — 1× in this paper · explained in 4 episodes
• Core — 1× in this paper · explained in 2 episodes
• Generalizability — 1× in this paper · explained in 6 episodes
• Generalization — 1× in this paper · explained in 13 episodes
• Interpretability — 1× in this paper · explained in 15 episodes
• Latent Representation — 1× in this paper · explained in 2 episodes
• Latent Representations — 1× in this paper · explained in 3 episodes
• Representation Learning — 1× in this paper · explained in 2 episodes
• Retrieval-Augmented Generation — 1× in this paper
• Retrieval-Augmented Generation (RAG) — 1× in this paper · explained in 16 episodes
• Robustness — 1× in this paper · explained in 5 episodes
• Tokenization — 1× in this paper · explained in 2 episodes

Sources

• Audio Flamingo 3: Advancing Audio Intelligence with Fully Open Large Audio Language Models
• Gemma 3 Technical Report
• TASTE: Text-Aligned Speech Tokenization and Embedding for Spoken Language Modeling
• Step-Audio 2 Technical Report
• Reducing Object Hallucination in Large Audio-Language Models via Audio-Aware Decoding

Related papers

• Few-Shot Open-Set Audio Classification via Transductive Prototype Refinement and Class Logit Enhancement
• Spectral Masking and Interpolation Attack (SMIA): A Black-box Adversarial Attack against Voice Authentication and Anti-Spoofing Systems
• AVMeme Exam: A Multimodal Multilingual Multicultural Benchmark for LLMs' Contextual and Cultural Knowledge and Thinking
• SoundWeaver: Compositional Warm-Starting for Text-to-Audio Diffusion Serving
• WASIL: In-the-Wild Arabic Spoken Interactions with LLMs
• Efficient Audiovisual Speech Processing via MUTUD: Multimodal Training and Unimodal Deployment