Sharp Regret Bounds and a Task-Covariance Correction for Spectral Representation Learning

arXiv:2609.15825 · cs.LG, cs.IT, math.IT, stat.ML · Submitted 2026-09-14 · Read on arXiv

cs.LG, cs.IT, math.IT, stat.ML

Submitted: 2026-09-14

Updated: 2026-09-26

Comments: 27 pages, 4 figures

License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/

Terminology

• Regret — 31× in this paper · explained in 3 episodes
• Alignment — 22× in this paper · explained in 1 episode
• Anisotropy — 22× in this paper · explained in 3 episodes
• Trace — 10× in this paper · explained in 2 episodes
• Representation Learning — 5× in this paper · explained in 2 episodes
• Spectral Gap — 5× in this paper · explained in 5 episodes
• Calibration — 3× in this paper · explained in 7 episodes
• Error Exponents — 3× in this paper · explained in 2 episodes
• Self-Supervised Learning — 3× in this paper · explained in 7 episodes
• Contrastive Learning — 2× in this paper · explained in 5 episodes
• Accuracy — 1× in this paper · explained in 1 episode
• Downstream Tasks — 1× in this paper · explained in 2 episodes
• Failure Modes — 1× in this paper · explained in 3 episodes
• Feature Selection — 1× in this paper · explained in 4 episodes
• Few-Shot Learning — 1× in this paper · explained in 4 episodes
• Generative AI — 1× in this paper · explained in 2 episodes
• Redundancy — 1× in this paper · explained in 2 episodes
• Regularization — 1× in this paper · explained in 3 episodes
• Robustness — 1× in this paper · explained in 5 episodes
• Structural Constraints — 1× in this paper · explained in 2 episodes
• Transfer Learning — 1× in this paper · explained in 10 episodes
• Transferability — 1× in this paper · explained in 4 episodes

Sources

• Revisiting the Platonic Representation Hypothesis: An Aristotelian View
• Back into Plato's Cave: Examining Cross-modal Representational Convergence at Scale
• A Statistical Theory of Contrastive Learning via Approximate Sufficient Statistics
• A Statistical Theory of Contrastive Pre-training and Multimodal Generative AI
• Universal Feature Selection with Noisy Observations and Weak Symmetry Conditions

Related papers

• Polynomial-Augmented Neural Networks (PANNs) with Weak Orthogonality Constraints for Enhanced Function and PDE Approximation
• AIRL-S: Unifying Reinforcement Learning and Search-Based Test-Time Scaling via Adversarial Inverse Reinforcement Learning
• Transformers as Bayesian In-Context Experimenters: Smoothness-Adaptive Efficient ATE Estimation
• Convergence issues in Relational Concept Analysis based on AOC-posets
• Beliefs Beyond Posteriors: Local-Consistency Optimisation for Bayesian Neural Networks
• Understanding Diffusion Models via Ratio-Based Function Approximation with SignReLU Networks