SecRespond: Benchmarking AI Agents for Real-World Post-Compromise Incident Response

arXiv:2607.26791 · cs.CR, cs.AI, cs.CL · Submitted 2026-07-29 · Read on arXiv

Lehan Wang, Boli Chen, Ruixue Ding, Pengjun Xie, Jinwei Huang, Zhendong Liu, Shuo Wang, Tao Lei, Xin Ouyang, Xiaomeng Li

cs.CR, cs.AI, cs.CL

Submitted: 2026-07-29

Code: https://github.com/Alibaba-NLP/qqr

License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/

Terminology

Benchmark — 45× in this paper · explained in 2 episodes
Verification — 26× in this paper · explained in 2 episodes
Trace — 18× in this paper · explained in 2 episodes
LLM judge — 17× in this paper · explained in 2 episodes
LLM-as-a-Judge — 13× in this paper · explained in 3 episodes
Benchmarking — 9× in this paper · explained in 3 episodes
Large Language Model — 7× in this paper
AI Agents — 6× in this paper · explained in 2 episodes
Agent Harness — 6× in this paper · explained in 2 episodes
Calibration — 4× in this paper · explained in 7 episodes
Core — 4× in this paper · explained in 2 episodes
Large Language Model (LLM) — 3× in this paper · explained in 3 episodes
Provenance — 3× in this paper · explained in 4 episodes
Aggregation — 2× in this paper · explained in 1 episode
Accuracy — 1× in this paper · explained in 1 episode
Adam — 1× in this paper · explained in 2 episodes
Agent Skills — 1× in this paper · explained in 1 episode
Consistency — 1× in this paper · explained in 3 episodes
Decode — 1× in this paper · explained in 2 episodes
Large Language Models (LLMs) — 1× in this paper · explained in 6 episodes
Security Operations Center — 1× in this paper

Sources

Related papers