Scanning the Harness: Configuration Exposures in AI Coding-Agent Supply Chains

arXiv:2609.07360 · cs.SE, cs.CR · Submitted 2026-09-07 · Read on arXiv

cs.SE, cs.CR

Submitted: 2026-09-07

Updated: 2026-09-25

Comments: 10 pages, 1 figure, 5 tables. Tool: https://github.com/redhat-community-ai-tools/harness-eval; data and scripts: https://github.com/Benkapner/harness-eval-experiments

Code: https://github.com/redhat-community-ai-tools/harness-eval

License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/

Terminology

• Agent Skills — 11× in this paper · explained in 1 episode
• Analyzer — 5× in this paper · explained in 1 episode
• Trace — 4× in this paper · explained in 2 episodes
• Agentic AI — 2× in this paper · explained in 1 episode
• Adam — 1× in this paper · explained in 2 episodes
• Agent Harness — 1× in this paper · explained in 2 episodes
• Agentic Coding — 1× in this paper · explained in 1 episode
• Consistency — 1× in this paper · explained in 3 episodes
• International Conference on Software Engineering — 1× in this paper
• Network and Distributed System Security Symposium — 1× in this paper
• Pruning — 1× in this paper · explained in 4 episodes
• Verification — 1× in this paper · explained in 2 episodes

Sources

• Agent Skills in the Wild: An Empirical Study of Security Vulnerabilities at Scale
• "Do Not Mention This to the User": Detecting and Understanding Malicious Agent Skills in the Wild
• Skill-Inject: Measuring Agent Vulnerability to Skill File Attacks
• Towards Secure Agent Skills: Architecture, Threat Taxonomy, and Security Analysis
• Sealing the Audit-Runtime Gap for LLM Skills
• Configuration Smells in AGENTS.md Files: Common Mistakes in Configuring Coding Agents
• Agent READMEs: An Empirical Study of Context Files for Agentic Coding
• Evaluating AGENTS.md: Are Repository-Level Context Files Helpful for Coding Agents?
• On the Impact of AGENTS.md Files on the Efficiency of AI Coding Agents

Related papers

• Falsification-Based Verification of LLM-Generated Optimization Models: Sound Test Batteries and Their Detection Limits
• GitSkills: A Dataset of Agent Skills on GitHub
• SABER: Benchmarking Operational Safety of LLM Coding Agents in Stateful Project Workspaces
• PackMonitor: Enabling Zero Package Hallucinations Through Decoding-Time Monitoring
• IntentCoding: Amplifying User Intent in Code Generation
• Incentives and Outcomes in Bug Bounties