Understanding Generalization Requires Universal Induction

arXiv:2609.34458 · stat.ML, cs.AI, cs.IT, cs.LG, math.IT, math.ST, stat.TH · Submitted 2026-09-28 · Read on arXiv

stat.ML, cs.AI, cs.IT, cs.LG, math.IT, math.ST, stat.TH

Submitted: 2026-09-28

Updated: 2026-09-28

Terminology

• Generalization — 37× in this paper · explained in 13 episodes
• Regret — 21× in this paper · explained in 3 episodes
• No Free Lunch — 19× in this paper
• Transformer — 8× in this paper · explained in 2 episodes
• Bayesian Inference — 6× in this paper · explained in 5 episodes
• Generalization Bounds — 4× in this paper · explained in 2 episodes
• Meta-learning — 4× in this paper · explained in 2 episodes
• Adam — 3× in this paper · explained in 2 episodes
• Hilbert Space — 2× in this paper · explained in 2 episodes
• In-Context Learning — 2× in this paper · explained in 4 episodes
• Loss Function — 2× in this paper · explained in 2 episodes
• Regularization — 2× in this paper · explained in 3 episodes
• AI Agents — 1× in this paper · explained in 2 episodes
• Ab Initio — 1× in this paper · explained in 1 episode
• Abstraction — 1× in this paper · explained in 1 episode
• Attention Mechanism — 1× in this paper · explained in 3 episodes
• Reinforcement Learning — 1× in this paper · explained in 4 episodes
• Reproducing Kernel Hilbert Space (RKHS) — 1× in this paper · explained in 3 episodes
• Sample Complexity — 1× in this paper · explained in 15 episodes
• Sample Efficiency — 1× in this paper · explained in 4 episodes
• Scaling Laws — 1× in this paper · explained in 4 episodes
• Transformer Architecture — 1× in this paper · explained in 2 episodes
• World Model — 1× in this paper · explained in 7 episodes

Sources

• User-friendly introduction to PAC-Bayes bounds
• The committee machine: Computational to statistical gaps in learning a two-layers neural network
• Determination of the fifth Busy Beaver value
• Absolutely No Free Lunches!
• Random semicomputable reals revisited
• How Uniform Random Weights Induce Non-uniform Bias: Typical Interpolating Neural Networks Generalize with Narrow Teachers
• Why Boltzmann Brains Are Bad
• Pessimism About Unknown Unknowns Inspires Conservatism
• Algorithmic Complexity Bounds on Future Prediction Errors
• Sample Complexity of Agnostic Multiclass Classification: Natarajan Dimension Strikes Back
• Self-Play Pretraining with Zero Data
• A Survey on In-context Learning
• Optimal Learners for Multiclass Problems
• Golden Handcuffs make safer AI agents
• The fate of Earth during the Sun's giant phases: New constraints from ab initio tidal modelling and AGB mass loss
• Toward Universal Laws of Outlier Propagation
• From Entropy to Epiplexity: Rethinking Information for Computationally Bounded Intelligence
• Loss Bounds and Time Complexity for Speed Priors
• The Indefinite Logarithm, Logarithmic Units, and the Nature of Entropy
• Lecture notes on descriptional complexity and randomness

Related papers

• Behavior of prediction performance metrics with rare events
• Optimal Estimation of Generic Dynamics by Path-Dependent Neural Jump ODEs
• A Posterior-Dynamics Framework for Imaging Inverse Problems with Pretrained Diffusion Priors
• One Permutation Is All You Need: Fast, Deterministic Feature Importance and Model Stress-Testing
• Online Conformal Prediction for Non-Exchangeable Panel Data
• Deep Time-Series Forecasting in 10 Years: A Survey