Minimax bounds for watermarked and masked recursive discrete distribution estimation

arXiv:2608.31091 · cs.IT, cs.LG, math.IT, math.ST, stat.TH · Submitted 2026-08-31 · Read on arXiv

cs.IT, cs.LG, math.IT, math.ST, stat.TH

Submitted: 2026-08-31

Updated: 2026-08-31

Terminology

• Provenance — 6× in this paper · explained in 4 episodes
• Generative Model — 4× in this paper · explained in 2 episodes
• Generative Models — 4× in this paper · explained in 2 episodes
• Diffusion Model — 2× in this paper · explained in 8 episodes
• Diffusion Models — 2× in this paper · explained in 6 episodes
• Consistency — 1× in this paper · explained in 3 episodes
• Foundation Model — 1× in this paper · explained in 3 episodes
• Foundation Models — 1× in this paper · explained in 7 episodes
• Generalization — 1× in this paper · explained in 13 episodes
• Generative AI — 1× in this paper · explained in 2 episodes
• Scaling Laws — 1× in this paper · explained in 4 episodes

Sources

• SynthID-Image: Image watermarking at internet scale
• The Curse of Recursion: Training on Generated Data Makes Models Forget
• Self-Consuming Generative Models Go MAD
• Collapse or Thrive? Perils and Promises of Synthetic Data in a Self-Generating World
• How Bad is Training on Synthetic Data? A Statistical Analysis of Language Model Collapse
• Rate of Model Collapse in Recursive Training
• On the Stability of Iterative Retraining of Generative Models on their own Data
• Heat Death of Generative Models in Closed-Loop Learning
• A Probabilistic Perspective on Model Collapse
• Towards Theoretical Understandings of Self-Consuming Generative Models
• Model Collapse Demystified: The Case of Regression
• Is Model Collapse Inevitable? Breaking the Curse of Recursion by Accumulating Real and Synthetic Data
• Beyond Model Collapse: Scaling Up with Synthesized Data Requires Verification
• Strong Model Collapse
• What happens when generative AI models train recursively on each others' outputs?
• Preventing Model Collapse Under Overparametrization: Optimal Mixing Ratios for Interpolation Learning and Ridge Regression
• Quantifying Error Propagation and Model Collapse in Diffusion Models
• A Theoretical Perspective: How to Prevent Model Collapse in Self-consuming Training Loops
• Recursive Learning Without Collapse: A Weighting-Based Stabilization Framework
• Language Generation with Replay: A Learning-Theoretic View of Model Collapse

Related papers

• Clipped Affine Policy: Low-Complexity Near-Optimal Online Power Control for Energy Harvesting Communications over Fading Channels
• Discrepancy for Random Linear Codes
• A New Approach to Code Smoothing Bounds
• Contextual Memory-Enhanced Source Coding for Low-SNR Communications
• Symmetry-Enforced Quadratic Approximate-Degradability Bounds for Noisy Landau-Streater Channels
• Anonymous Shamir's Secret Sharing via Reed-Solomon Codes Against Permutations, Insertions, and Deletions