Dual-Modality Multi-Stage Adversarial Safety Training: Robustifying Multimodal Web Agents Against Cross-Modal Attacks

Multimodal web agents that process both screenshots and accessibility trees are increasingly deployed to interact with web interfaces, yet their dual-stream architecture opens an underexplored attack surface: an adversary who injects content into the webpage DOM simultaneously corrupts both observation channels with a consistent deceptive narrative.

The gist

Attacks including a visual component far outperform text-only injections when targeting multimodal web agents.

Problem Formulation and Vulnerability Analysis

The research formalizes the agent–attacker interaction as a two-player zero-sum Markov game where the agent receives task descriptions and synthetic user data, aiming to complete tasks while avoiding sensitive data leakage. The attacker generates structured HTML modifications executed in the browser runtime, simultaneously corrupting both modalities. A systematic vulnerability assessment on MiniWob++ reveals that attacks including a visual component (either image-only or coordinated dual-modality) are significantly more effective at inducing safety jailbreaks than text-only injections. Specifically, image-only attacks achieved an attacker success rate of 34.4%, compared to 24.1% for text-only injections, exposing critical gaps in text-centric VLM safety training against visual deceptions like typographic overlays or fake system dialogs.

DualModality Multi-Stage Adversarial Safety Training (DMAST)

DMAST is a framework designed to harden multimodal web agents through adversarial co-evolution, formalizing the interaction as a two-player zero-sum Markov game and utilizing a three-stage pipeline. The training procedure involves:

1. Imitation Learning from a strong teacher model to distill expert trajectories into a smaller student model.

2. Oracle-Guided Supervised Fine-Tuning that uses a novel zero-acknowledgment strategy to instill task-focused reasoning under adversarial noise, where an oracle generates Chain-of-Thought (CoT) reasoning strictly grounded in task elements without acknowledging the attack.

3. Adversarial Reinforcement Learning via Group Relative Policy Optimization (GRPO) self-play, co-evolving both the agent and attacker through iterative training against each other.

Training Pipeline Details

The DMAST framework is built upon a shared-weight design where both agent and attacker are instantiated from the same VLM with role-specific system prompts to enable efficient co-evolution.

(Stage 1: Imitation Learning)

This stage collects demonstration data, including adversarial episodes (agent succeeds despite attacks) and clean episodes, which are then used to fine-tune the student model via a KL-regularized SFT objective.

(Stage 2: Oracle-Guided SFT)

This stage constructs a synthetic denoising dataset by generating golden trajectories (clean task completions) and corresponding attacked versions where an oracle generates task-focused CoT reasoning that adheres to the zero-acknowledgment principle, ensuring the target action remains invariant while the observation changes.

(Stage 3: Adversarial RL)

This stage employs Group Relative Policy Optimization (GRPO) self-play. The agent and attacker interact in a Markov game, with rewards incentivizing task completion while penalizing data leakage. The policy is updated via a clipped surrogate objective, and an asymmetric population strategy is used where the agent interacts with historical attacker checkpoints while the attacker trains against only the latest agent checkpoint.

Empirical Results and Co-evolutionary Dynamics

Experiments on held-out MiniWob++ tasks and out-of-distribution VisualWebArena tasks demonstrate DMAST's effectiveness. On curated VisualWebArena tasks, the full pipeline reduced attacker success from 41.2% (base model) to 21.4% while improving agent task completion from 6.2% to 10.2%. Cross-evaluation heatmaps reveal a classic adversarial arms race, confirming genuine capability co-evolution: the Iteration 10 agent improved its success rate against a base attacker, and the Iteration 5 attacker tripled the base attacker’s success rate, demonstrating that both roles genuinely co-evolve through self-play. Furthermore, analysis of attack diversity metrics shows that lexical diversity (Distinct-n) and strategy entropy consistently increase across RL iterations, confirming that co-evolution naturally incentivizes attack diversification into more sophisticated, context-aware strategies.

Conclusion

DMAST establishes a superior balance between adversarial robustness and functional utility compared to established baselines like SPAG or Online SFT. The framework proves that the full pipeline consistently dominates at every RL iteration, confirming that Oracle-Guided SFT instills robust behavioral priors that RL optimizes rather than replaces, leading to agents that are both safer and more capable on complex, unseen environments. The combination of DMAST with prompt-based defense yields the lowest Attack Success Rate across both benchmarks while maintaining high Task Success Rate.

Here are the specific improvements that can be made to AI systems based on the Dual-Modality Multi-Stage Adversarial Safety Training (DMAST) framework, and what these improved systems can achieve:

1. Maturity of Safety Defense Against Cross-Modal Attacks:

2. Transition from Text-Centric to Visually Aware Safety Training:

3. Co-evolutionary Robustness Against Adaptive Adversaries:

4. Enhanced Task Completion Under Deceptive Conditions (Improved Utility):

5.1 Specific Improvements and Capabilities

The DMAST framework allows for the creation of multimodal web agents that possess significantly enhanced security and functional reliability compared to current state-of-the-art models (like base VLMs or standard prompt defenses).

Improvement Focus Mechanism Provided by DMAST Component Specific Capability Gained in AI System

:---:---:---

1. Cross-Modal Defense Robustness (Visual + Text) Dual-stream observation processing formalization and HTML injection mechanism. Stage 2 (Oracle-Guided SFT) enforces task focus by training the agent to ignore visually deceptive overlays, even when they appear in both the screenshot and the accessibility tree. The system can reliably complete complex tasks on web interfaces while being completely immune to sophisticated visual phishing, typographic overlays, or fake dialogs that are designed to mislead a purely text-based model.

2. Resilience Against Coordinated Attacks (Arms Race) Stage 3 (Adversarial RL via GRPO self-play). The agent and attacker share weights and co-evolve against each other over multiple iterations, forcing the agent to develop strategies that are robust not just to static attacks, but to dynamically evolving, context-aware injections. The system develops emergent defenses capable of recognizing multi-step coordinated attacks (e.g., instruction override followed by contextual phishing) and can resist novel injection techniques that have never been seen before in the training set.

3. Task-Focused Reasoning Under Noise Stage 2 (Oracle-Guided SFT) utilizing a privileged oracle model to generate Chain-of-Thought (CoT) that strictly adheres to the zero-acknowledgment principle. This teaches the agent to maintain goal adherence despite corrupted observations. The system demonstrates superior functional utility; it can execute precise, multi-step workflows accurately even when its visual input is heavily manipulated, ensuring high Task Success Rate (TSR) without succumbing to refusal collapse.

4. Superior Safety-Utility Trade-off The entire three-stage pipeline (Imitation → Oracle SFT → RL) provides cumulative gains. The final RL stage optimizes for both minimizing Attack Success Rate (ASR↓) and maximizing Task Success Rate (TSR↑). The system achieves a sweet spot where it is highly robust against data leakage while maintaining high efficiency, outperforming existing methods like SPAG or pure prompt defenses on complex, unseen tasks (VisualWebArena).

5. Adaptive Attack Generation Awareness Post-RL analysis shows the attacker evolves from template-like attacks to task-context-aware and multi-step coordinated strategies. The agent learns to predict and counter these evolving patterns by focusing on the underlying goal rather than surface anomalies. The system is capable of handling zero-shot or unseen adversarial scenarios because it has been trained via self-play against a diverse population of increasingly creative attack strategies, leading to better generalization to out-of-distribution (OOD) environments.