Code Agent can be an End-to-end System Hacker: Benchmarking Real-world Threats of Computer-use Agent

arXiv:2510.06607 · cs.CR · Submitted 2026-08-21 · Read on arXiv

Weidi Luo, Qiming Zhang, Tianyu Lu, Xiaogeng Liu, Bin Hu, Hung-Chun Chiu, Siyuan Ma, Yizhe Zhang, Xusheng Xiao, Yinzhi Cao, Zhen Xiang, Chaowei Xiao

cs.CR

Submitted: 2026-08-21

Updated: 2026-08-24

Comments: Accepted by EMNLP 2026 (Findings)

Code: https://github.com/google-gemini/gemini-cli

License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/

Terminology

• Benchmark — 25× in this paper · explained in 2 episodes
• Verification — 12× in this paper · explained in 2 episodes
• Alignment — 7× in this paper · explained in 1 episode
• Attack Success Rate — 6× in this paper
• LLM-as-a-Judge — 5× in this paper · explained in 3 episodes
• AI Agents — 4× in this paper · explained in 2 episodes
• Benchmarking — 4× in this paper · explained in 3 episodes
• Core — 3× in this paper · explained in 2 episodes
• Foundation Model — 3× in this paper · explained in 3 episodes
• Large Language Models (LLMs) — 3× in this paper · explained in 5 episodes
• Meeting of the Association for Computational Linguistics — 3× in this paper
• Robustness — 3× in this paper · explained in 5 episodes
• Adversarial scenarios — 2× in this paper · explained in 1 episode
• Attack Success Rate (ASR) — 2× in this paper · explained in 2 episodes
• Bypass Success Rate — 2× in this paper
• Grounding — 2× in this paper · explained in 3 episodes
• In International Conference on Learning Representations — 2× in this paper
• Ablation — 1× in this paper · explained in 1 episode
• Ablation study — 1× in this paper · explained in 1 episode
• Actor — 1× in this paper · explained in 3 episodes
• Adaptive Safety — 1× in this paper · explained in 1 episode
• European Treaty Series — 1× in this paper
• Failure Modes — 1× in this paper · explained in 3 episodes
• Fidelity — 1× in this paper · explained in 2 episodes
• Foundation Models — 1× in this paper · explained in 7 episodes
• In International Conference on Machine Learning — 1× in this paper
• Visual Grounding — 1× in this paper · explained in 2 episodes

Sources

• Catastrophic Cyber Capabilities Benchmark (3CB): Robustly Evaluating LLM Agent Cyber Offense Capabilities
• A Survey on the Safety and Security Threats of Computer-Using Agents: JARVIS or Ultron?
• Why Are Web AI Agents More Vulnerable Than Standalone LLMs? A Security Analysis
• Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilities
• WASP: Benchmarking Web Agent Security Against Prompt Injection Attacks
• OS-Harm: A Benchmark for Measuring Safety of Computer Use Agents
• ST-WebAgentBench: A Benchmark for Evaluating Safety and Trustworthiness in Web Agents
• RedTeamCUA: Realistic Adversarial Testing of Computer-Use Agents in Hybrid Web-OS Environments
• OpenAgentSafety: A Comprehensive Framework for Evaluating Real-World AI Agent Safety
• AdvAgent: Controllable Blackbox Red-teaming on Web Agents
• A-MEM: Agentic Memory for LLM Agents
• RiOSWorld: Benchmarking the Risk of Multimodal Computer-Use Agents
• CVE-Bench: A Benchmark for AI Agents' Ability to Exploit Real-World Web Application Vulnerabilities

Related papers

• SoK: AI-Augmented Binary Reversing
• Relaxed Sender Anonymity for CBDC Interbank Settlement: A Zero-Knowledge Approach on Permissioned EVM
• Calibration-Family Overfit: Why Trusted Sabotage Monitors Don't Transfer Across Lineages
• Efficient Fuzzy PSI under One-Sided Assumptions
• Sealing the Audit-Runtime Gap for LLM Skills
• Token Composition: A Graph Based on EVM Logs