Mallows-type model averaging: Non-asymptotic analysis and all-subset combination

arXiv:2505.02637 · math.ST, stat.ML, stat.TH · Submitted 2025-05-05 · Read on arXiv

math.ST, stat.ML, stat.TH

Submitted: 2025-05-05

Updated: 2026-09-26

Terminology

• Aggregation — 9× in this paper · explained in 1 episode
• Ensemble Learning — 4× in this paper · explained in 2 episodes
• Accuracy — 2× in this paper · explained in 1 episode
• Alignment — 2× in this paper · explained in 1 episode
• Benchmark — 2× in this paper · explained in 2 episodes
• Generalization — 2× in this paper · explained in 13 episodes
• Loss Function — 2× in this paper · explained in 2 episodes
• Minimax Regret — 2× in this paper · explained in 2 episodes
• Non-asymptotic Analysis — 2× in this paper · explained in 2 episodes
• Regret — 2× in this paper · explained in 3 episodes
• Regularization — 2× in this paper · explained in 3 episodes
• Adaptation — 1× in this paper · explained in 1 episode
• Consistency — 1× in this paper · explained in 3 episodes
• Decoupling — 1× in this paper · explained in 2 episodes
• Signal-to-noise ratio (SNR) — 1× in this paper · explained in 2 episodes
• Sparsity — 1× in this paper · explained in 4 episodes
• Theoretical Guarantees — 1× in this paper · explained in 2 episodes
• Wavelet Transform — 1× in this paper · explained in 2 episodes

Sources

• On High-Dimensional Asymptotic Properties of Model Averaging Estimators
• Error Reduction from Stacked Regressions
• On the design-dependent suboptimality of the Lasso
• Ensemble linear interpolators: The role of ensembling
• From Model Selection to Model Averaging: A Comparison for Nested Linear Models
• On Asymptotic Optimality of Least Squares Model Averaging When True Model Is Included

Related papers

• Conformal Prediction for Dyadic Regression Under Complex Missingness
• Bentkus-type asymptotic e-values
• High-Dimensional Asymptotics of Differentially Private PCA
• KL Convergence Guarantees for Score diffusion models under minimal data assumptions
• Geometric bias in eigenspace perturbation under random heterogeneous noise
• On the Asymptotic Inadmissibility of Double Machine Learning Estimators Under Structure-Agnostic Models