Understanding Deep Learning via Notions of Rank
cs.LG, cs.AI, cs.NE, stat.ML
Submitted: 2024-08-04
Updated: 2026-08-29
Comments: PhD thesis
Code: https://github.com/noamrazin/gnn_interactions
License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
Terminology
Sources
- GPT-4 Technical Report
- Layer Normalization
- On implicit regularization: Morse functions and applications to matrix factorization
- Geometric Deep Learning: Grids, Groups, Graphs, Geodesics, and Gauges
- Gradient Descent for Deep Matrix Factorization: Dynamics and Implicit Bias towards Low Rank
- More is Less: Inducing Sparsity via Overparameterization
- Limitations of Implicit Bias in Matrix Sensing: Initialization Rank Matters
- Fast Graph Representation Learning with PyTorch Geometric
- The expressive power of kth-order invariant graph networks
- The Low-Rank Simplicity Bias in Deep Networks
- Inductive Bias of Multi-Channel Linear Convolutional Networks with Bounded Weight Norm
- Geometry of Linear Convolutional Networks
- DeeperGCN: All You Need to Train Deeper GCNs
- What Happens after SGD Reaches Zero Loss? --A Mathematical Framework
- Weisfeiler and Leman go Machine Learning: The Story so far
- Implicit Regularization in Deep Learning
- Revisiting Graph Neural Networks: All We Have is Low-Pass Filters
- Implicit Bias of Policy Gradient in Linear Quadratic Control: Extrapolation to Unseen Initial States
- Pitfalls of Graph Neural Network Evaluation
- On Margin Maximization in Linear and ReLU Networks
Related papers
- Polynomial-Augmented Neural Networks (PANNs) with Weak Orthogonality Constraints for Enhanced Function and PDE Approximation
- AIRL-S: Unifying Reinforcement Learning and Search-Based Test-Time Scaling via Adversarial Inverse Reinforcement Learning
- Transformers as Bayesian In-Context Experimenters: Smoothness-Adaptive Efficient ATE Estimation
- Convergence issues in Relational Concept Analysis based on AOC-posets
- Beliefs Beyond Posteriors: Local-Consistency Optimisation for Bayesian Neural Networks
- Understanding Diffusion Models via Ratio-Based Function Approximation with SignReLU Networks