Stochastic Bilevel Optimization with Heavy-Tailed Noise

arXiv:2509.14952 · cs.LG · Submitted 2025-09-18 · Read on arXiv

cs.LG

Submitted: 2025-09-18

Updated: 2026-09-25

Terminology

• Reinforcement Learning — 6× in this paper · explained in 4 episodes
• Meta-learning — 5× in this paper · explained in 2 episodes
• Accuracy — 4× in this paper · explained in 1 episode
• Regularization — 4× in this paper · explained in 3 episodes
• Model-Agnostic — 3× in this paper · explained in 2 episodes
• Objective Function — 3× in this paper · explained in 2 episodes
• Adaptation — 2× in this paper · explained in 1 episode
• Convergence Guarantees — 2× in this paper · explained in 3 episodes
• Robustness — 2× in this paper · explained in 5 episodes
• Theoretical Guarantees — 2× in this paper · explained in 2 episodes
• Actor — 1× in this paper · explained in 3 episodes
• Adversarial Attacks — 1× in this paper · explained in 1 episode
• Adversarial Robustness — 1× in this paper · explained in 3 episodes
• Deep Reinforcement Learning — 1× in this paper · explained in 2 episodes
• Generative Adversarial Networks (GANs) — 1× in this paper · explained in 3 episodes
• Large Language Model (LLM) — 1× in this paper · explained in 3 episodes
• Transformer — 1× in this paper · explained in 2 episodes

Sources

• On the Condition Number Dependency in Bilevel Optimization
• Approximation Methods for Bilevel Programming
• Bilevel Optimization under Unbounded Smoothness: A New Algorithm and Convergence Analysis
• Complexity of normalized stochastic first-order methods with momentum under heavy-tailed noise
• High Probability and Risk-Averse Guarantees for a Stochastic Accelerated Primal-Dual Method
• A High Probability Analysis of Adaptive SGD with Momentum
• Deep Reinforcement Learning: An Overview
• High Probability Convergence of Clipped-SGD Under Heavy-tailed Noise
• On First-Order Meta-Learning Algorithms
• Revisiting Gradient Normalization and Clipping for Nonconvex SGD under Heavy-Tailed Noise: Necessity, Sufficiency, and Acceleration
• Nonconvex Decentralized Stochastic Bilevel Optimization under Heavy-Tailed Noise
• Neural Architecture Search with Reinforcement Learning

Related papers

• Polynomial-Augmented Neural Networks (PANNs) with Weak Orthogonality Constraints for Enhanced Function and PDE Approximation
• AIRL-S: Unifying Reinforcement Learning and Search-Based Test-Time Scaling via Adversarial Inverse Reinforcement Learning
• Transformers as Bayesian In-Context Experimenters: Smoothness-Adaptive Efficient ATE Estimation
• Convergence issues in Relational Concept Analysis based on AOC-posets
• Beliefs Beyond Posteriors: Local-Consistency Optimisation for Bayesian Neural Networks
• Understanding Diffusion Models via Ratio-Based Function Approximation with SignReLU Networks