VulContextBench: A Benchmark for Security Context Retrieval in Coding Agents

arXiv:2609.32601 · cs.CR · Submitted 2026-09-26 · Read on arXiv

cs.CR

Submitted: 2026-09-26

Updated: 2026-09-26

Code: https://github.com/yikun-li/vul-context-bench

Terminology

• Benchmark — 25× in this paper · explained in 2 episodes
• Core — 15× in this paper · explained in 2 episodes
• Accuracy — 8× in this paper · explained in 1 episode
• Verifier — 7× in this paper · explained in 2 episodes
• Trace — 5× in this paper · explained in 2 episodes
• International Conference on Software Engineering — 3× in this paper
• Conference on Mining Software Repositories — 2× in this paper
• Benchmarking — 1× in this paper · explained in 3 episodes
• Consistency — 1× in this paper · explained in 3 episodes
• Verification — 1× in this paper · explained in 2 episodes

Sources

• Vulnerability Detection with Code Language Models: How Far Are We?
• A Survey on Code Generation with LLM-based Agents
• ContextBench: A Benchmark for Context Retrieval in Coding Agents
• CleanVul: Automatic Function-Level Vulnerability Detection in Code Commits Using LLM Heuristics
• SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineering
• SWE Context Bench: A Benchmark for Context Learning in Coding

Related papers

• SoK: AI-Augmented Binary Reversing
• Relaxed Sender Anonymity for CBDC Interbank Settlement: A Zero-Knowledge Approach on Permissioned EVM
• Calibration-Family Overfit: Why Trusted Sabotage Monitors Don't Transfer Across Lineages
• Efficient Fuzzy PSI under One-Sided Assumptions
• Sealing the Audit-Runtime Gap for LLM Skills
• Token Composition: A Graph Based on EVM Logs