A Farewell to the Bias-Variance Tradeoff? An Overview of the Theory of Overparameterized Machine Learning
stat.ML, cs.LG
Submitted: 2021-09-06
Updated: 2026-09-09
License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
Terminology
Sources
- Support vector machines and linear regression coincide with very high-dimensional features
- Distribution of Classification Margins: Are All Data Equal?
- Failures of model-dependent generalization bounds for least-norm interpolation
- Deep learning: a statistical viewpoint
- Fit without fear: remarkable mathematical phenomena of deep learning through the prism of interpolation
- Risk Bounds for Over-parameterized Maximum Margin Classification on Sub-Gaussian Mixtures
- On the robustness of minimum norm interpolators and regularized empirical risk minimizers
- Double Double Descent: On Generalization Errors in Transfer Learning between Linear Regression Tasks
- The Common Intuition to Transfer Learning Can Win or Lose: Case Studies for Linear Regression
- A Precise Performance Analysis of Learning with Random Features
- Revisiting minimum description length complexity in overparameterized models
- Computing Nonvacuous Generalization Bounds for Deep (Stochastic) Neural Networks with Many More Parameters than Training Data
- Probing transfer learning with a model of synthetic correlated datasets
- Surprises in High-Dimensional Ridgeless Least Squares Interpolation
- Evaluation of Neural Architectures Trained with Square Loss vs Cross-Entropy in Classification Tasks
- NeurIPS 2020 Competition: Predicting Generalization in Deep Learning
- Uniform Convergence of Interpolators: Gaussian Width, Norm Bounds, and Benign Overfitting
- A Quotient Property for Matrices with Heavy-Tailed Entries and its Application to Noise-Blind Compressed Sensing
- Interpolating Classifiers Make Few Mistakes
- A Precise High-Dimensional Asymptotic Theory for Boosting and Minimum-$\ell_1$-Norm Interpolated Classifiers
Related papers
- Behavior of prediction performance metrics with rare events
- Optimal Estimation of Generic Dynamics by Path-Dependent Neural Jump ODEs
- A Posterior-Dynamics Framework for Imaging Inverse Problems with Pretrained Diffusion Priors
- One Permutation Is All You Need: Fast, Deterministic Feature Importance and Model Stress-Testing
- Online Conformal Prediction for Non-Exchangeable Panel Data
- Deep Time-Series Forecasting in 10 Years: A Survey