An Active-Bottleneck Mechanism for Weak-to-Strong Generalization
stat.ML, cs.AI, cs.LG
Submitted: 2026-09-27
Updated: 2026-09-27
Terminology
Sources
- Two-Point Deterministic Equivalence for Stochastic Gradient Dynamics in Linear Models
- A Dynamical Model of Neural Scaling Laws
- Weak-to-Strong Generalization: Eliciting Strong Capabilities With Weak Supervision
- Distillation Scaling Laws
- Discrepancies are Virtue: Weak-to-Strong Generalization through Lens of Intrinsic Dimension
- Weak-to-Strong Generalization is Nearly Inevitable (in Linear Models)
- Weak-to-Strong Generalization Even in Random Feature Networks, Provably
- Relating Misfit to Gain in Weak-to-Strong Generalization Beyond the Squared Loss
- Improved Scaling Laws via Weak-to-Strong Generalization in Random Feature Ridge Regression
- On the Emergence of Weak-to-Strong Generalization: A Bias-Variance Perspective
- Representations Shape Weak-to-Strong Generalization: Theoretical Insights and Empirical Predictions
- On the Blessing of Pre-training in Weak-to-Strong Generalization
Related papers
- Behavior of prediction performance metrics with rare events
- Optimal Estimation of Generic Dynamics by Path-Dependent Neural Jump ODEs
- A Posterior-Dynamics Framework for Imaging Inverse Problems with Pretrained Diffusion Priors
- One Permutation Is All You Need: Fast, Deterministic Feature Importance and Model Stress-Testing
- Online Conformal Prediction for Non-Exchangeable Panel Data
- Deep Time-Series Forecasting in 10 Years: A Survey