Break Step: Recursive Training Resonates with Replayed Sampling Noise
cs.CL, cs.AI, cs.LG
Submitted: 2026-09-10
Updated: 2026-09-29
License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
Terminology
Sources
- Strong Model Collapse
- The Impact of AI-Generated Text on the Internet
- Characterizing Model Behavior Under Synthetic Data Training: An Empirical Study Across Scales and Mixing Ratios
- Is Model Collapse Inevitable? Breaking the Curse of Recursion by Accumulating Real and Synthetic Data
- Epistemic diversity across language models mitigates knowledge collapse
- Benchmarking Recursive-Collapse Warning Claims Under Matched False-Positive Control
- When Sample Selection Bias Precipitates Model Collapse
- How Bad is Training on Synthetic Data? A Statistical Analysis of Language Model Collapse
- Delving into: the quantification of Ai-generated content on the internet (synthetic data)
- What happens when generative AI models train recursively on each others' outputs?
- LLM Web Dynamics: Tracing Model Collapse in a Network of LLMs
- When Does Model Collapse Occur in Structured Interactive Learning?
Related papers
- Exploring Solution Divergence and Its Effect on Large Language Model Problem Solving
- Ishigaki-IDS-Bench: A Benchmark for Generating Information Delivery Specification from BIM Information Requirements
- Subliminal Steering: Stronger Encoding of Hidden Signals
- MedStruct-S: A Benchmark for Key Discovery, Key-Conditioned QA and Semi-Structured Extraction from OCR Clinical Reports
- The End of Transformers? On Challenging Attention and the Rise of Sub-Quadratic Architectures
- Untangling the Mechanisms of Misleading Context in Medical Question Answering