Security in the Age of AI Teammates: An Empirical Study of Agentic Pull Requests on GitHub

arXiv:2601.00477 · cs.CR, cs.SE · Submitted 2026-01-01 · Read on arXiv

cs.CR, cs.SE

Submitted: 2026-01-01

Updated: 2026-09-03

Terminology

• AI Agents — 5× in this paper · explained in 2 episodes
• Agentic System — 5× in this paper · explained in 2 episodes
• Agentic AI — 4× in this paper · explained in 1 episode
• Accuracy — 3× in this paper · explained in 1 episode
• Adam — 3× in this paper · explained in 2 episodes
• Agentic AI systems — 3× in this paper · explained in 1 episode
• Agentic Coding — 3× in this paper · explained in 1 episode
• Formal Verification — 3× in this paper · explained in 2 episodes
• Large Language Models (LLMs) — 3× in this paper · explained in 6 episodes
• Verification — 3× in this paper · explained in 2 episodes
• Benchmark — 2× in this paper · explained in 2 episodes
• Multi-Agent System — 2× in this paper · explained in 5 episodes
• Robustness — 2× in this paper · explained in 5 episodes
• Source Code Analysis and Manipulation — 2× in this paper
• Transformer — 2× in this paper · explained in 2 episodes
• Accountability — 1× in this paper · explained in 2 episodes
• Agentic Approaches — 1× in this paper · explained in 1 episode
• Alignment — 1× in this paper · explained in 1 episode
• Calibration — 1× in this paper · explained in 7 episodes
• Common Weakness Enumeration — 1× in this paper
• Consistency — 1× in this paper · explained in 3 episodes
• Failure Modes — 1× in this paper · explained in 3 episodes
• Feature Extraction — 1× in this paper · explained in 2 episodes
• Fine-Tuning — 1× in this paper · explained in 5 episodes
• Generalization — 1× in this paper · explained in 13 episodes
• Interpretability — 1× in this paper · explained in 15 episodes
• Loss Function — 1× in this paper · explained in 2 episodes
• Predictive Modeling — 1× in this paper · explained in 2 episodes
• Steering — 1× in this paper · explained in 2 episodes

Sources

• Training language models to follow instructions with human feedback
• GPT-4 Technical Report
• The Rise of AI Teammates in Software Engineering (SE) 3.0: How Autonomous Coding Agents Are Reshaping Software Engineering
• Agentic Software Engineering: Foundational Pillars and a Research Roadmap
• ReAct: Synergizing Reasoning and Acting in Language Models
• WebArena: A Realistic Web Environment for Building Autonomous Agents
• On the Use of Agentic Coding: An Empirical Study of Pull Requests on GitHub
• Evaluating Large Language Models Trained on Code
• A Survey on Large Language Models for Software Engineering
• Asleep at the Keyboard? Assessing the Security of GitHub Copilot's Code Contributions
• Formalizing the Safety, Security, and Functional Properties of Agentic AI Systems
• Taming Uncertainty via Automation: Observing, Analyzing, and Optimizing Agentic AI Systems
• ProofWright: Towards Agentic Formal Verification of CUDA
• Toward Agentic Software Engineering Beyond Code: Framing Vision, Values, and Vocabulary
• Vibe Coding vs. Agentic Coding: Fundamentals and Practical Implications of Agentic AI
• Agentsway -- Software Development Methodology for AI Agents-based Teams
• CodePori: Large-Scale System for Autonomous Software Development Using Multi-Agent Technology
• Experimenting with Multi-Agent Software Development: Towards a Unified Platform
• AI Agentic Programming: A Survey of Techniques, Challenges, and Opportunities
• ChatCollab: Exploring Collaboration Between Humans and AI Agents in Software Teams

Related papers

• SoK: AI-Augmented Binary Reversing
• Relaxed Sender Anonymity for CBDC Interbank Settlement: A Zero-Knowledge Approach on Permissioned EVM
• Calibration-Family Overfit: Why Trusted Sabotage Monitors Don't Transfer Across Lineages
• Efficient Fuzzy PSI under One-Sided Assumptions
• Sealing the Audit-Runtime Gap for LLM Skills
• Token Composition: A Graph Based on EVM Logs