Diffusion Models and Concept Formation
cs.AI, cs.LG
Submitted: 2026-09-11
Updated: 2026-09-21
Comments: Advances of Cognitive Systems 2026 Oral Presentation
License: http://creativecommons.org/licenses/by/4.0/
The gist: Humans organize knowledge into a taxonomy of concepts with nested levels of abstraction and a basic level at which people recognize and name objects with the least cognitive effort.
Terminology
Abstract
Humans organize knowledge into a taxonomy of concepts with nested levels of abstraction and a basic level at which people recognize and name objects with the least cognitive effort. Cobweb is a classic cognitive account of this ability, an incremental learner that builds a probabilistic concept hierarchy by maximizing category utility. We argue that diffusion models, although designed for image synthesis, implicitly perform the same computation. The noisy marginals of a diffusion model are Gaussian smoothings of the data distribution, and the modes of these marginals form a hierarchy that corresponds to a Cobweb tree of probabilistic prototypes in four respects. Both are hierarchical density models, both are hierarchical-Bayesian models with Gaussian prototypes, both treat categorization as score-following that reduces uncertainty, and in both a basic level emerges. We locate this basic level for a diffusion model at an intermediate noise level, where recent analyses show that the reverse process commits to the class identity of a sample. The two models differ mainly in how they represent and learn the taxonomy. Cobweb learns a discrete tree incrementally, whereas a diffusion model encodes a continuous, interpolable hierarchy in a single learned score field fit to the data distribution. We test the correspondence on MNIST and Fashion-MNIST by recovering the diffusion hierarchy through mode-finding and comparing the basic levels of the two models. This reframes diffusion as a cognitive model of concept formation and offers Cobweb a continuous, scalable instantiation.
Sources
- The statistical thermodynamics of generative diffusion models: Phase transitions, symmetry breaking and critical instability
- Incremental Concept Formation over Visual Images Without Catastrophic Forgetting
- Dynamical Regimes of Diffusion Models
- Compositional Visual Generation and Inference with Energy Based Models
- Denoising Diffusion Probabilistic Models
- Classifier-Free Diffusion Guidance
- Variational Deep Embedding: An Unsupervised and Generative Approach to Clustering
- Elucidating the Design Space of Diffusion-Based Generative Models
- Variational Diffusion Models
- Auto-Encoding Variational Bayes
- Information-Theoretic Diffusion
- Cobweb: An Incremental and Hierarchical Model of Human-Like Category Learning
- Compositional Visual Generation with Composable Diffusion Models
- Tree Variational Autoencoders
- A Phase Transition in Diffusion Models Reveals the Hierarchical Nature of Data
- Deep Unsupervised Learning using Nonequilibrium Thermodynamics
- Generative Modeling by Estimating Gradients of the Data Distribution
- Score-Based Generative Modeling through Stochastic Differential Equations
- Test-Time Compositional Generalization in Diffusion Models via Concept Discovery
- Taxonomic Networks: A Representation for Neuro-Symbolic Pairing
Related papers
- MAVEN-T: Reinforced Heterogeneous Distillation for Real-Time Multi-Agent Trajectory Prediction
- Model Discovery Agent: LLM-assisted Bayesian experiment design for data-efficient discovery of mechanistic world models
- The Clinician's Veto: Navigating Trust, Liability, and Uncertainty in Autonomous AI Prescribing
- MindHelper: Closed-Loop Embodied Mental-State Reasoning for Precision Intervention
- Incumbent Advantage: Brand Bias and Cognitive Manipulation Dynamics in LLM Recommendation Systems
- VSAL: A Vision Solver with Adaptive Layouts for Graph Property Detection