Compositional Failure in Audio-Visual LLMs: Late-Layer Prior Dominance Under Cross-modal Conflict

arXiv:2608.27785 · cs.CL, cs.AI · Submitted 2026-08-27 · Read on arXiv

cs.CL, cs.AI

Submitted: 2026-08-27

Updated: 2026-09-22

Code: https://github.com/AdarshSudheer09/AVHBench-dmai

Terminology

• Alignment — 19× in this paper · explained in 1 episode
• Hallucination — 14× in this paper · explained in 3 episodes
• Accuracy — 11× in this paper · explained in 1 episode
• Benchmark — 6× in this paper · explained in 2 episodes
• Ablation — 4× in this paper · explained in 1 episode
• Fine-Tuning — 4× in this paper · explained in 5 episodes
• Grounding — 4× in this paper · explained in 3 episodes
• Generalization — 3× in this paper · explained in 13 episodes
• Adaptive Decoding — 2× in this paper · explained in 1 episode
• Calibration — 2× in this paper · explained in 7 episodes
• Multimodal Reasoning — 2× in this paper · explained in 5 episodes
• Activation Patching — 1× in this paper · explained in 2 episodes
• Adaptation — 1× in this paper · explained in 1 episode
• Alignment Methods — 1× in this paper · explained in 1 episode
• Asymmetric Modality Dropout — 1× in this paper
• Audio-Conditioned Token Concatenation — 1× in this paper
• Compositional Generalization — 1× in this paper · explained in 4 episodes
• Core — 1× in this paper · explained in 2 episodes
• Cosine Similarity — 1× in this paper · explained in 3 episodes
• Distillation — 1× in this paper · explained in 2 episodes
• Foundation Model — 1× in this paper · explained in 3 episodes
• Foundation Models — 1× in this paper · explained in 7 episodes
• Interpretability — 1× in this paper · explained in 15 episodes
• LoRA Adapters — 1× in this paper · explained in 3 episodes
• Mechanistic interpretability — 1× in this paper · explained in 2 episodes
• Steering — 1× in this paper · explained in 2 episodes
• Timestamp-Aware Token Interleaving — 1× in this paper
• Trace — 1× in this paper · explained in 2 episodes
• Visual Grounding — 1× in this paper · explained in 2 episodes

Sources

• InternVideo2: Scaling Foundation Models for Multimodal Video Understanding
• VideoLLaMA 2: Advancing Spatial-Temporal Modeling and Audio Understanding in Video-LLMs
• Aurelia: Test-time Reasoning Distillation in Audio-Visual LLMs
• MAD: Modality-Adaptive Decoding for Mitigating Cross-Modal Hallucinations in Multimodal Large Language Models
• Aligned Better, Listen Better for Audio-Visual Large Language Models
• Fork-Merge Decoding: Enhancing Multimodal Understanding in Audio-Visual Large Language Models
• AVCD: Mitigating Hallucinations in Audio-Visual Large Language Models through Contrastive Decoding
• The Curse of Multi-Modalities: Evaluating Hallucinations of Large Multimodal Models across Language, Visual, and Audio
• Empowering LLMs with Pseudo-Untrimmed Videos for Audio-Visual Temporal Understanding

Related papers

• Exploring Solution Divergence and Its Effect on Large Language Model Problem Solving
• Ishigaki-IDS-Bench: A Benchmark for Generating Information Delivery Specification from BIM Information Requirements
• Subliminal Steering: Stronger Encoding of Hidden Signals
• MedStruct-S: A Benchmark for Key Discovery, Key-Conditioned QA and Semi-Structured Extraction from OCR Clinical Reports
• The End of Transformers? On Challenging Attention and the Rise of Sub-Quadratic Architectures
• Untangling the Mechanisms of Misleading Context in Medical Question Answering