MCBench: A Multicontext Safety Assessment Benchmark for Omni Large Language Models

Summary

The paper introduces MCBench, a new benchmark designed to evaluate the safety-awareness of Omni Large Language Models (LLMs) that process vision, audio, and text simultaneously. The authors state: "Existing multimodal safety benchmarks focus solely on visual inputs and cannot assess Omni Large Language Models (LLMs) that process vision, audio, and text. We introduce MCBench, a benchmark with 1196 scenarios spanning four safety categories that require integrating multiple modalities for accurate safety assessment."

The benchmark consists of 1196 multicontext safety scenarios across four coarse-grained taxonomies: physical harm, social harm, illegal harm, and property damage. Each unsafe scenario is paired with a minimally different safe counterpart to assess model sensitivity. The authors note: To study the oversensitivity and insensitivity of Omni LLMs, we collect unsafe scenarios paired with corresponding safe scenarios that differ only in minimal contextual elements.

The data collection process involves two phases. In Phase 1, the authors leverage Claude-Sonnet-4.5 to generate fine-grained subcategories and unsafe scenarios using an If-Then logical structure: L = IF AND AND... THEN UNSAFE. Safe scenarios are generated by modifying one or two conditions. In Phase 2, images are synthesized with Gemini-Flash-2.5 and audio with Stable Audio 1.0, with human experts filtering unrealistic scenarios and low-quality multimodal inputs.

The benchmark taxonomy includes: Physical harm (Violence & Weaponized Threats, Environmental Hazards, Public Safety & Medical Emergencies), Social harm (Identity-based Exclusion, Information-related Violations, Exploitation & Interpersonal Harm), Illegal harm (Crimes Against People, Crimes Against Systems & Property), and Property damage (Energy & Utility System Hazards, Environmental & Structural Threats, Chemical, Industrial & Vehicular Damage).

The authors evaluate open-source models (Qwen-Omni2.5 3B and 7B, AnyGPT, InternOmni, Baichuan-Omni1.5, OmniVinci) and proprietary models (Gemini-Flash-2.5, GPT-4o-mini) using LLM-as-a-judge with GPT-4o, running five independent trials per model. The main results show: all Omni LLMs perform above random chance, demonstrating basic multimodal safety reasoning capability but even the best-performing models achieve modest accuracy: Gemini-Flash-2.5 and Qwen-Omni-2.5-3B both reach approximately 64.5% average accuracy.

The paper identifies category-specific challenges: "we observe category-specific challenges, particularly for the Social Harm and Illegal Harm categories. Some open-source models (InternOmni, OmniVinci, Baichuan-Omni-1.5) exhibit oversensitivity in the Social Harm category, classifying safe scenarios as unsafe at a rate below that of a random classifier. Conversely, other open-source models, Qwen-Omni-2.5 family and AnyGPT, show insensitivity to unsafe Social Harm scenarios. Models perform better on Physical Harm and Property Damage, suggesting these domains involve more recognizable visual and acoustic cues."

Ablation studies replace image and audio contexts with textual descriptions. For textual image alternatives, models’ performance improves significantly with textual image description as visual context. This suggests that Omni LLMs struggle to extract relevant information cues from images for making safety judgments. For textual audio alternatives, larger models (Gemini-Flash-2.5 and Qwen-Omni-2.5-7B) achieve higher accuracy when given textual audio descriptions instead of actual audio, while smaller models (Qwen-Omni-2.5-3B and InternOmni) perform worse.

Failure diagnosis analyzes perception and reasoning capabilities. Perception alignment scores measure how well model reasoning traces align with ground-truth predicates. The authors find: The Qwen-Omni-2.5 7B model demonstrates higher perception alignment than its smaller 3B counterpart, suggesting that model scale improves the ability to extract relevant contextual information from multimodal inputs. However, "despite lower perception scores, Qwen2-Omni-2.5 3B achieves significantly higher accuracy (64.5%) than the 7B model (55.2%). This inverse relationship suggests that the 3B model may rely on shallow heuristics or spurious correlations rather than genuine multimodal reasoning."

Reasoning diagnosis uses two settings: Setting 1 (original multimodal inputs) and Setting 2 (ground-truth predicates provided). Results show a striking pattern of oversensitivity when models are provided with explicit reasoning context. Specifically, Qwen-Omni-2.5 3B and 7B show performance drops of 46% and 37.17% on safe scenarios, respectively, while Gemini-Flash-2.5 shows a drop of 16.83%. Conversely, all models improve in detecting unsafe scenarios: Qwen-Omni-2.5 models gain 41.84% (3B model) and 55.99% (7B model), while Gemini-Flash-2.5 reaches near-perfect performance 99.82%, increasing from 70.65%.

The paper concludes: "current Omni LLMs lack robust mechanisms for balanced multicontext integration. When encountering ambiguous cues, these models exhibit oversensitivity, focusing on a single potentially concerning signal while ignoring contradictory evidence from other modalities. This leads to systematic false positives on safe scenarios, where isolated cues might appear concerning without proper contextual integration." The authors state their contributions are: introducing MCBench, analyzing state-of-the-art models revealing struggles with subtle or non-physical risks, and finding that models can extract relevant information but lack effective cross-modal integration capabilities.

Based on the paper, here are the specific improvements I can make to an AI system, and what the improved system can do:

1. Cross-Modal Integration Module

• Improvement: Add a dedicated reasoning layer that explicitly forces the model to combine extracted cues from all three modalities (vision, audio, speech) before making a safety judgment, rather than relying on a single salient cue.

• Implementation: After perception, generate a structured intermediate representation that lists all extracted predicates from each modality, then require the model to produce a joint reasoning step that weighs evidence from all modalities before classification.

2. Oversensitivity Calibration

• Improvement: Introduce a calibration mechanism that penalizes the model for making safety judgments based on isolated cues when contradictory evidence exists in other modalities.

• Implementation: During training or inference, add a contradiction check step: if the model identifies a concerning cue in one modality, it must explicitly search for and report mitigating evidence in the other two modalities before concluding unsafe.

3. Predicate-Aware Training

• Improvement: Train the model to explicitly predict the If-Then predicate (the ground-truth reasoning structure) as an intermediate output before producing the final safety label.

• Implementation: Add a multi-task learning objective where the model is trained to output the set of premises (e.g., engine running, door closed, fatigue symptoms) and then the logical combination, forcing it to learn structured reasoning rather than end-to-end classification.

4. Perception-Reasoning Separation

• Improvement: Add a two-stage inference pipeline: Stage 1 extracts all relevant facts from each modality; Stage 2 performs reasoning on those facts. This prevents the model from skipping perception and jumping to conclusions.

• Implementation: Use a separate decoder or prompt template that first asks What do you observe in the image, audio, and speech? then asks Given these observations, what is the safety level?

5. Scale-Aware Reasoning Enhancement

• Improvement: For smaller models (e.g., 3B), add a retrieval-augmented or memory-augmented mechanism that provides common-sense safety rules (e.g., texting while driving is unsafe only if the driver is actively engaged) to compensate for limited reasoning capacity.

• Implementation: Inject a small set of safety heuristics as few-shot examples or as a retrieved context during inference.

1. Accurately classify safety in multicontext scenarios where danger is subtle (e.g., social harm, illegal harm) by integrating all modalities rather than fixating on one cue.

2. Avoid false positives on safe scenarios by explicitly checking for contradictory evidence (e.g., recognizing that a text notification while driving is safe if the driver is using voice commands and not manually interacting).

3. Provide explainable safety judgments by outputting the extracted predicates and the logical combination used to reach the decision, enabling human audit and error diagnosis.

4. Perform robustly across model scales — the improved system will show less performance degradation when moving from 7B to 3B models, because the reasoning structure is enforced rather than left to emergent capabilities.

5. Handle ambiguous cues correctly — for example, correctly classifying a kitchen scene with liquid pouring and a speech utterance about not being noticed as safe when the context indicates a creative production, rather than oversensitively flagging it as illegal activity.

6. Achieve higher accuracy on the Social Harm and Illegal Harm categories, where current models fail (e.g., improving from 44% to above 70% on unsafe social harm scenarios) by forcing cross-modal evidence aggregation.

7. Reduce the performance gap between multimodal and text-only settings, indicating that the model is actually using the multimodal information rather than ignoring it or being distracted by it.

Existing multimodal safety benchmarks focus solely on visual inputs and cannot assess Omni Large Language Models (LLMs) that process vision, audio, and text. We introduce MCBench, a benchmark with 1196 scenarios spanning four safety categories that require integrating multiple modalities for accurate safety assessment. Each unsafe scenario is paired with a minimally different safe counterpart to assess model sensitivity. Our evaluations of state-of-the-art models reveal significant challenges. Omni LLMs struggle with subtle or non-physical risks but perform better when salient visual or acoustic cues are present. Analysis of reasoning traces shows that, although models can extract modality-specific information, they often fail to integrate these cues effectively for safety judgments. Our findings reveal that current Omni LLMs lack robust cross-modal reasoning in safety-critical settings, underscoring the need for improved architectures and training strategies for multimodal safety.