Word-Class and Construction-Like Structure Emerges in Neural Successor Representations Trained on Natural Language

The gist The central result of this work is that SRs, trained as multi-horizon predictive representations over a large naturalistic text corpus, spontaneously organise into a geometry that mirrors the hierarchical linguistic structure of language, namely its constructional organisation, without any explicit supervision.

How it works

1. Successor Representations (SRs) model not the immediate next state but the expected discounted distribution of future states instead of predicting the next token in a sequence The paper explores an alternative predictive principle derived from reinforcement learning, which models what is expected to occur in the future and how often

2. The model is trained to predict the expected discounted distribution of future words across multiple temporal horizons instead of predicting the next token This objective learns representations of the long-range transition structure of language

3. The training involves treating SR targets as probability distributions using KL divergence rather than direct regression, optimizing a deep residual neural network on WikiText-103 This allows linguistic structure to emerge spontaneously without explicit supervision or annotation of linguistic categories

Emergent Linguistic Structure

The model develops a pronounced geometric organization with respect to part-of-speech (POS) categories after training, where nouns, verbs, and adjectives become recoverable through unsupervised clustering The temporal prediction horizon systematically shapes representation geometry, as short horizons preserve local syntactic regularities while longer horizons integrate broader contextual and semantic information

Recovering Basic Syntactic Categories

The unsupervised clustering of SR embeddings at a low discount factor γ = 0.2 recovers the three major categories of nouns, verbs, and adjectives with per-cluster POS purities of 0.89, 0.91, and 0.85 respectively The inter-cluster transition network replicates well-motivated grammatical asymmetries: strong ADJ→NOUN flow (67%) reflects the modifier–head construction, and dominant NOUN→VERB connectivity (52%) encodes the subject–predicate template

Constructional Slots as Lossy Clusters of Memory Traces

Increasing the number of clusters to k = 30 reveals semantically coherent sub-classes, such as ordinal modifiers for adjectives and event-type classes for verbs, which are interpretable as distributional signatures of constructional slots This structure suggests that the SR space encodes not only category membership but also the constructional contexts items inhabit, consistent with usage-based accounts of grammar

Limitations and Future Directions

A practical limitation is the absence of a principled criterion for selecting the appropriate clustering resolution k or temporal horizon γ, as higher values of γ integrate over longer contextual windows and reduce cluster coherence Furthermore, the inability to distinguish between distinct senses of a surface form due to type-level representations necessitates a shift toward contextualised SR estimation conditioned on preceding context

The SR space instantiates precisely such a space, and the clusters that emerge from it can be understood as such compressions: each cluster summarises a family of partially overlapping usage traces whose shared successor profile reflects the combinatorial privileges conferred by repeated participation in similar contexts

This multi-granular alignment provides a topological metric for the theoretical distinction between errors and innovation, quantifying exactly how and where a rule is broken The SR space thus encodes the full slot-and-filler architecture predicted by usage-based accounts

The model's ability to recover these categories so robustly without explicit supervision suggests that structural categories are emergent consequences of predictive sequence learning rather than intrinsic lexical properties This demonstrates that the SR space encodes the full slot-and-filler architecture predicted by usage-based accounts

The paper's findings support a unifying hypothesis that constructional knowledge is not a specialized linguistic module, but an instance of a more general predictive memory system operating over structured sequences The SR may serve as a computational bridge between the constructional organization of language and the neuro-scientific accounts of the memory system in which that organization is grounded

The results confirm that structural categories can be discovered directly from sequential statistics, offering a data-driven tool for crosslinguistic analysis that organically maps a language’s true emergent categories without requiring any prior commitment to traditional taxonomies The SR space encodes the full slot-and-filler architecture predicted by usage-based accounts

The SR transition structure provides a concrete, mathematical mechanism to bridge the gap between abstract template planning and concrete lexical instantiation The model's ability to recover these categories so robustly without explicit supervision suggests that structural categories are emergent consequences of predictive sequence learning rather than intrinsic lexical properties The SR space encodes the full slot-and-filler architecture predicted by usage-based accounts

The SR geometry captures the sequential links that bind elements together into constructions (syntagmatic relations) as well as the equivalence classes of items that can substitute into the same structural slots (paradigmatic relations) This view suggests that category membership is not an intrinsic lexical property but an emergent consequence of the positional and combinatorial privileges a form repeatedly occupies across constructions

The SR space instantiates precisely such a space, and the clusters that emerge from it can be understood as such compressions: each cluster summarises a family of partially overlapping usage traces whose shared successor profile reflects the combinatorial privileges conferred by repeated participation in similar contexts This view suggests that category membership is not an intrinsic lexical property but an emergent consequence of the positional and combinatorial privileges a form repeatedly occupies across constructions

The SR space encodes the full slot-and-filler architecture predicted by usage-based accounts The model's ability to recover these categories so robustly without explicit supervision suggests that structural categories are emergent consequences of predictive sequence learning rather than intrinsic lexical properties

The SR transition structure provides a concrete, mathematical mechanism to bridge the gap between abstract template planning and concrete lexical instantiation The model's ability to recover these categories so robustly without explicit supervision suggests that structural categories are emergent consequences of predictive sequence learning rather than intrinsic lexical properties The SR

1. Successor Representation Training for Linguistic Structure Emergence: Implement a multi-horizon predictive training objective using KL divergence to model the expected discounted distribution of future states across multiple temporal horizons, which allows the system to learn the long-range transition structure of language rather than just immediate next-token prediction.

2. Spontaneous Part-of-Speech Recovery via Unsupervised Clustering: Leverage the learned representation space by applying KMeans and BGM clustering on SR embeddings to recover linguistic categories, as shown in Figure 4, demonstrating that nouns, verbs, and adjectives become recoverable through unsupervised clustering despite the absence of linguistic labels during training.

3. Context-Sensitive Constructional Slot Modeling: Utilize high-resolution clustering (e.g., k=120) to identify constructional slots by analyzing inter-cluster transition weights, enabling the system to model the full slot-and-filler architecture predicted by usage-based accounts, such as distinguishing between bare cardinals and measurement units and calendar years and month names.

4. Scale-Dependent Creativity Modeling: Apply the SR geometry to quantify creativity by observing how it models deviation: a collo-creative utterance—such as an unexpected lexical filler—corresponds to a drop in SR transition weights to near-zero at a fine-grained, item-specific level (such as k=120), which allows the system to distinguish between deliberate, creative deviations from unintelligible errors.

5. Multiscale Planning Substrate: Design an architectural extension where the hierarchical structure of the SR embedding space could serve as a substrate for multiscale planning in language production, allowing the model to commit to a high-level frame (e.g., [MONTH YEAR] frame) while using fine-grained transition probabilities to constrain the selection of a specific month name and year token in sequence."

Neural language models are typically trained on next-token prediction, although linguistic structure spans multiple temporal scales. Successor representations (SRs) make this horizon explicit by encoding discounted distributions over future states. Here, we ask whether such predictive representations can recover not only word classes, but also finer functional and construction-like structure from natural language. A residual network trained on WikiText-103 predicts SR distributions at three horizons without part-of-speech supervision. At the shortest horizon, unsupervised clustering robustly recovers nouns, verbs, and adjectives, while directed inter-cluster transitions reproduce familiar syntactic asymmetries. At finer resolutions and across 13 part-of-speech categories, the same geometry reveals semantic-functional groupings that cross category boundaries and directed relations tracing candidate date, measurement, and title-name constructions. Part-of-speech agreement declines as the predictive horizon lengthens. These results suggest that word classes are coarse regions within a richer predictive geometry in which categorical and construction-like linguistic structure emerge from future-word distributions.