Clipped or Unclipped? Finite-Sample Trade-offs for Averaged SGD under Heavy-Tailed Noise
cs.LG, math.OC
Submitted: 2026-09-26
Updated: 2026-09-26
Terminology
Sources
- Llama 2: Open Foundation and Fine-Tuned Chat Models
- High Probability Convergence of Clipped-SGD Under Heavy-tailed Noise
- Can SGD Handle Heavy-Tailed Noise?
- Overparameterized Nonlinear Learning: Gradient Descent Takes the Shortest Path?
- Open Bandit Dataset and Pipeline: Towards Realistic and Reproducible Off-Policy Evaluation
- Fuk-Nagaev inequality in smooth Banach spaces: Optimum bounds for distributions of heavy-tailed martingales
- Accelerated stochastic first-order method for convex optimization under heavy-tailed noise
- In-Expectation Convergence of Stochastic Gradient Methods under Heavy-Tailed Noise
Related papers
- Polynomial-Augmented Neural Networks (PANNs) with Weak Orthogonality Constraints for Enhanced Function and PDE Approximation
- AIRL-S: Unifying Reinforcement Learning and Search-Based Test-Time Scaling via Adversarial Inverse Reinforcement Learning
- Transformers as Bayesian In-Context Experimenters: Smoothness-Adaptive Efficient ATE Estimation
- Convergence issues in Relational Concept Analysis based on AOC-posets
- Beliefs Beyond Posteriors: Local-Consistency Optimisation for Bayesian Neural Networks
- Understanding Diffusion Models via Ratio-Based Function Approximation with SignReLU Networks