WAInjectBench: Benchmarking Prompt Injection Detections for Web Agents

arXiv:2510.01354 · cs.CR, cs.AI, cs.CL · Submitted 2025-10-01 · Read on arXiv

cs.CR, cs.AI, cs.CL

Submitted: 2025-10-01

Updated: 2026-09-23

Code: https://github.com/Norrrrrrr-lyn/WAInjectBench

Terminology

• Fine-Tuning — 16× in this paper · explained in 5 episodes
• Benchmarking — 9× in this paper · explained in 4 episodes
• Benchmark — 8× in this paper · explained in 2 episodes
• Web Arena — 7× in this paper
• Generalization — 6× in this paper · explained in 13 episodes
• Accuracy — 5× in this paper · explained in 1 episode
• Visual Web Arena — 5× in this paper
• Adaptation — 3× in this paper · explained in 1 episode
• Adversarial Examples — 3× in this paper · explained in 2 episodes
• Robustness — 3× in this paper · explained in 5 episodes
• Domain Adaptation — 2× in this paper · explained in 5 episodes
• Adversarial Robustness — 1× in this paper · explained in 3 episodes
• Core — 1× in this paper · explained in 2 episodes
• Data Augmentation — 1× in this paper · explained in 7 episodes
• Distillation — 1× in this paper · explained in 2 episodes
• Document Object Model — 1× in this paper
• Embeddings — 1× in this paper · explained in 2 episodes
• Ensemble Approach — 1× in this paper · explained in 3 episodes
• False Positive Rate — 1× in this paper
• Parameter Efficiency — 1× in this paper · explained in 3 episodes
• Screenshot with Set-of-Marks — 1× in this paper
• Transferability — 1× in this paper · explained in 4 episodes
• Transformer — 1× in this paper · explained in 2 episodes
• True Positive Rate — 1× in this paper
• Verification — 1× in this paper · explained in 2 episodes

Sources

• Embedding-based classifiers can detect prompt injection attacks
• VPI-Bench: Visual Prompt Injection Attacks for Computer-Use Agents
• WASP: Benchmarking Web Agent Security Against Prompt Injection Attacks
• AugMix: A Simple Data Processing Method to Improve Robustness and Uncertainty
• Llama Guard: LLM-based Input-Output Safeguard for Human-AI Conversations
• VisualWebArena: Evaluating Multimodal Agents on Realistic Visual Web Tasks
• EIA: Environmental Injection Attack on Generalist Web Agents for Privacy Leakage
• Improving Robustness Without Sacrificing Accuracy with Patch Gaussian Augmentation
• PromptArmor: Simple yet Effective Prompt Injection Defenses
• Intriguing properties of neural networks
• MiniLMv2: Multi-Head Self-Attention Relation Distillation for Compressing Pretrained Transformers
• WebInject: Prompt Injection Attack to Web Agents
• Dissecting Adversarial Robustness of Multimodal LM Agents
• Feature Squeezing: Detecting Adversarial Examples in Deep Neural Networks
• JailGuard: A Universal Detection Framework for LLM Prompt-based Attacks
• Attacking Vision-Language Computer Agents via Pop-ups
• WebArena: A Realistic Web Environment for Building Autonomous Agents
• MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Models

Related papers

• SoK: AI-Augmented Binary Reversing
• Relaxed Sender Anonymity for CBDC Interbank Settlement: A Zero-Knowledge Approach on Permissioned EVM
• Calibration-Family Overfit: Why Trusted Sabotage Monitors Don't Transfer Across Lineages
• Efficient Fuzzy PSI under One-Sided Assumptions
• Sealing the Audit-Runtime Gap for LLM Skills
• Token Composition: A Graph Based on EVM Logs