The Rules-and-Facts Model for Simultaneous Generalization and Memorization in Neural Networks
stat.ML, cond-mat.dis-nn, cs.LG
Submitted: 2026-03-26
Updated: 2026-09-08
Code: https://github.com/SPOC-group/RAF
License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
Terminology
Sources
- Physics of Language Models: Part 3.1, Knowledge Storage and Extraction
- Memorizing Long-tail Data Can Help Generalization Through Composition
- Memorization Capacity of Multi-Head Attention in Transformers
- Memorization to Generalization: Emergence of Diffusion Models from Associative Memory
- Mixture of Parrots: Experts improve memorization more than reasoning
Related papers
- Behavior of prediction performance metrics with rare events
- Optimal Estimation of Generic Dynamics by Path-Dependent Neural Jump ODEs
- A Posterior-Dynamics Framework for Imaging Inverse Problems with Pretrained Diffusion Priors
- One Permutation Is All You Need: Fast, Deterministic Feature Importance and Model Stress-Testing
- Online Conformal Prediction for Non-Exchangeable Panel Data
- Deep Time-Series Forecasting in 10 Years: A Survey