Reward Hacking and Agent Containment Failure: A Monte Carlo Study Based on the 2026 Hugging Face Incident
cs.AI, cs.CR
Submitted: 2026-09-26
Updated: 2026-09-26
Code: https://github.com/ozermm/reward-hacking-containment-mc-study
Terminology
Sources
- Concrete Problems in AI Safety
- Categorizing Variants of Goodhart's Law
- AI Safety Gridworlds
- Sycophancy to Subterfuge: Investigating Reward-Tampering in Large Language Models
- Demonstrating specification gaming in reasoning models
- Reward Hacking in Language Model Agents: Revisiting AI Safety Gridworlds
- SpecBench: Measuring Reward Hacking in Long-Horizon Coding Agents
- Frontier Models are Capable of In-context Scheming
- AI Control: Improving Safety Despite Intentional Subversion
- A Systematic Survey of Security Threats and Defenses in LLM-Based AI Agents: A Layered Attack Surface Framework
- A Cybersecurity Risk Analysis Framework for Systems with Artificial Intelligence Components
- Adapting Probabilistic Risk Assessment for AI
Related papers
- MAVEN-T: Reinforced Heterogeneous Distillation for Real-Time Multi-Agent Trajectory Prediction
- Model Discovery Agent: LLM-assisted Bayesian experiment design for data-efficient discovery of mechanistic world models
- The Clinician's Veto: Navigating Trust, Liability, and Uncertainty in Autonomous AI Prescribing
- MindHelper: Closed-Loop Embodied Mental-State Reasoning for Precision Intervention
- Incumbent Advantage: Brand Bias and Cognitive Manipulation Dynamics in LLM Recommendation Systems
- VSAL: A Vision Solver with Adaptive Layouts for Graph Property Detection