PentestChain: A Cost-Aware, MCP-Orchestrated Framework for Automated Penetration Testing with Free-Tier LLMs
cs.CR, cs.AI, cs.NI
Submitted: 2026-09-16
Updated: 2026-09-18
Comments: 12
Code: https://github.com/Rushabh0508/Pentest-Chain
License: http://creativecommons.org/licenses/by/4.0/
The gist: AI-driven penetration testing has been demonstrated with premium frontier models such as GPT-4, but the per-engagement token cost makes continuous, automated testing unaffordable for the smaller
Terminology
Abstract
AI-driven penetration testing has been demonstrated with premium frontier models such as GPT-4, but the per-engagement token cost makes continuous, automated testing unaffordable for the smaller organisations that need it most. This paper presents PentestChain, a ten-phase automated penetration testing framework that couples a curated, deterministic exploit map with a cost-aware AI cascade-a local Ollama model (qwen2.5-7b) first, then free-tier OpenRouter and Cerebras, with a rule-based fallback that always produces output-and exposes the full pipeline through a Model Context Protocol (MCP) server with eleven tools. We make three contributions. First, we treat US-dollar cost per engagement as a measured, first-class evaluation metric and show that a 7B-parameter local model, kept off the critical path by a deterministic backbone, sustains end-to-end operation at zero measured paid-API cost. Second, we analyse the attack surface that an MCP-exposed offensive engine introduces, grounding a four-position threat model in the 2025 MCP incident record (the CVE-2025-6514 remote-code-execution flaw in mcp-remote, the postmark-mcp supply-chain backdoor, and the tool-poisoning-rug-pull-line-jumping class), and contribute four mitigations. Third, we specify a reproducible, containerised evalua-tion protocol aligned with the standardised testbeds now expected at top-tier venues-AutoPenBench, a Cybench subset, and the PentestGPT 182-sub-task benchmark-with multi-trial statistics (more than 10 trials per configuration, pass-at-k, non-parametric significance tests and effect sizes) and direct, same testbed reproduction of the PentestGPT and PentestAgent baselines rather than citation of their published numbers. On the legacy targets measured to date, the framework detected 26 services, enriched 34 CVEs, produced
Sources
- Recognition Without Mitigation: Ethical Frameworks in Autonomous Offensive-LLM Agent Research
- MCPSecBench: A Systematic Security Benchmark and Playground for Testing Model Context Protocols
- PentestMCP: A Toolkit for Agentic Penetration Testing
- LLM Agents can Autonomously Exploit One-day Vulnerabilities
- LLM Agents can Autonomously Hack Websites
- Teams of LLM Agents can Exploit Zero-Day Vulnerabilities
- AutoPentester: An LLM Agent-based Framework for Automated Pentesting
- AutoPenBench: Benchmarking Generative Agents for Penetration Testing
- Benchmarking Practices in LLM-driven Offensive Security: Testbeds, Metrics, and Experiment Design
- On the Surprising Efficacy of LLMs for Penetration-Testing
- Getting pwn'd by AI: Penetration Testing with Large Language Models
- Model Context Protocol (MCP): Landscape, Security Threats, and Future Research Directions
- PenHeal: A Two-Stage LLM Framework for Automated Pentesting and Optimal Remediation
- Breaking the Protocol: Security Analysis of the Model Context Protocol Specification and Prompt Injection Vulnerabilities in Tool-Integrated LLM Agents
- Qwen2.5 Technical Report
- Autonomous Penetration Testing using Reinforcement Learning
- NYU CTF Bench: A Scalable Open-Source Benchmark Dataset for Evaluating LLMs in Offensive Security
- D-CIPHER: Dynamic Collaborative Intelligent Multi-Agent System with Planner and Heterogeneous Executors for Offensive Security
- Cybench: A Framework for Evaluating Cybersecurity Capabilities and Risks of Language Models
- When MCP Servers Attack: Taxonomy, Feasibility, and Mitigation
Related papers
- SoK: AI-Augmented Binary Reversing
- Relaxed Sender Anonymity for CBDC Interbank Settlement: A Zero-Knowledge Approach on Permissioned EVM
- Calibration-Family Overfit: Why Trusted Sabotage Monitors Don't Transfer Across Lineages
- Efficient Fuzzy PSI under One-Sided Assumptions
- Sealing the Audit-Runtime Gap for LLM Skills
- Token Composition: A Graph Based on EVM Logs