AI Finds A Way
cs.AI
Submitted: 2026-08-24
Updated: 2026-08-26
Comments: 26 Anecdotes, 40 Pages (59 with references/appendix), updated affiliation footnote
Code: https://github.com/aadharna/aifw
Project page: https://drubinstein.github.io/pokerl
License: http://creativecommons.org/licenses/by/4.0/
The gist: Artificial Intelligence (AI) algorithms frequently learn creative and unexpected solutions, surprising even expert researchers who develop and study them.
Terminology
Abstract
Artificial Intelligence (AI) algorithms frequently learn creative and unexpected solutions, surprising even expert researchers who develop and study them. They often astonish practitioners by discovering unanticipated behavior, exploiting loopholes in reward signals, or spontaneously uncovering previously unknown scientific phenomena. However, accounts of such unconventional behavior across machine learning are seldom formally documented. This work presents 26 curated firsthand anecdotes from various machine learning subfields representing the work of over 100 researchers. These anecdotes showcase the capability of modern AI systems to circumvent human-imposed design limitations and discover unexpected solutions to the tasks we train them on. Furthermore, these accounts are particularly important for the safety of future AI systems. They illustrate the fundamental challenge of aligning models with human values without diminishing their creativity, so they can make surprising discoveries without producing surprising, potentially harmful outcomes. The paper first details AI achieving superhuman success through reinforcement learning across many challenging domains. However, reward-driven optimization can fail when the model learns to hack an underspecified reward or unarticulated constraint. We then present case studies suggesting that harnessing internet-scale foundation models (FMs) has not resolved these fundamental challenges and, in fact, can supercharge them. Nevertheless, we argue that these same learning dynamics can be harnessed to accelerate scientific discovery. Finally, we hope this work provides a consolidated resource to inform future research and demonstrates that the tendency toward unexpected behaviors is commonplace in modern AI, highlighting the need to anticipate and manage AI's capacity for innovative, yet unpredictable, solutions. (abstract abridged)
Sources
- Do As I Can, Not As I Say: Grounding Language in Robotic Affordances
- Primitive sets and von Mangoldt chains: Erd\H{o}s Problem #1196 and beyond
- An Initial Introduction to Cooperative Multi-Agent Reinforcement Learning
- Concrete Problems in AI Safety
- Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedback
- Monitoring Reasoning Models for Misbehavior and the Risks of Promoting Obfuscation
- Learning in High Dimension Always Amounts to Extrapolation
- Irrationality of rapidly converging series: a problem of Erd\H{o}s and Graham
- Sparks of Artificial General Intelligence: Early experiments with GPT-4
- Weak-to-Strong Generalization: Eliciting Strong Capabilities With Weak Supervision
- Evaluating Large Language Models Trained on Code
- CycleGAN, a Master of Steganography
- Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilities
- TASRA: a Taxonomy and Analysis of Societal-Scale Risks from AI
- DeepSeek-V3 Technical Report
- When Chain of Thought is Necessary, Language Models Struggle to Evade Monitors
- OMNI-EPIC: Open-endedness via Models of human Notions of Interestingness with Environments Programmed in Code
- Semi-Autonomous Mathematics Discovery with Gemini: A Case Study on the Erd\H{o}s Problems
- Noisy Networks for Exploration
- The Llama 3 Herd of Models
Related papers
- MAVEN-T: Reinforced Heterogeneous Distillation for Real-Time Multi-Agent Trajectory Prediction
- Model Discovery Agent: LLM-assisted Bayesian experiment design for data-efficient discovery of mechanistic world models
- The Clinician's Veto: Navigating Trust, Liability, and Uncertainty in Autonomous AI Prescribing
- MindHelper: Closed-Loop Embodied Mental-State Reasoning for Precision Intervention
- Incumbent Advantage: Brand Bias and Cognitive Manipulation Dynamics in LLM Recommendation Systems
- VSAL: A Vision Solver with Adaptive Layouts for Graph Property Detection