Small Foundation Models of Human Cognition and Behaviour

arXiv:2608.05224 · cs.AI, cs.CY · Submitted 2026-08-11 · Read on arXiv

Listen

Radio episode about this paper

Transcript

Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.

Tom: Next we'll be talking about the paper "Small Foundation Models of Human Cognition and Behaviour".

Jane: The paper was written by Nick Oh and Fernand Gobet from socius labs and London School of Economics and Political Science.

Tom: Stay tuned as we take you through the paper and discuss its implications.

Jane: We also have Lu with us today — senior AI researcher at Tsinghua.

Tom: We also have Meng with us today — lead engineer at a mysterious AI startup.

Jane: We also have Lalam with us today — the in-house Large Language Model.

Tom: Alright, let's get started.

Title: Tom: Welcome back to the show, everyone. Today we’re digging into a paper that’s got a wonderfully ambitious title: “Small Foundation Models of Human Cognition and Behaviour.” And Jane, I have to say, the word “small” in that title is doing a lot of heavy lifting.

Jane: It really is, Tom. When we think of foundation models, we usually picture these massive, hundred-billion-parameter beasts. But this team from Socius Labs and the LSE, Nick Oh and Fernand Gobet, they’re asking a pretty cheeky question: do you actually need all that bulk to simulate how a person thinks?

Tom: And their answer, at least for a lot of tasks, is a resounding no. They took the big Centaur model, which is a seventy-billion-parameter model fine-tuned on human behavioral data, and they tried to shrink it down. They trained fourteen smaller models, some as small as one hundred thirty-five million parameters.

Jane: Right, that’s a factor of over five hundred smaller than the original. And the wild part is, when they tested these models on the same psychological experiments they were trained on, the small ones basically matched the giant one. It’s like discovering you can drive a compact car at the same speed as a Formula one car, as long as you’re just going around the block.

Tom: That’s a great analogy, because the difference shows up when you leave the block. On new, unseen experiments, the bigger models still have a clear advantage. So the size isn’t useless, it’s just not for the job everyone thought it was for.

Jane: Exactly. The paper is essentially saying that scale is a tool for generalization, not for fitting the training data. And that reframes how we should think about building these cognitive proxies in the first place.

Tom: So we’re not just talking about a smaller model, we’re talking about a different philosophy of what these models are for. And that’s a perfect segue into what they actually found when they started poking at what these models learn.

Jane: And that’s where it gets really interesting, because it’s one thing to predict a choice, it’s another thing to use the information the way a human would. Let’s get into that.

Summary: Tom: So, Jane, we’ve established that smaller models can match the big guy on familiar tasks. But the paper’s summary goes deeper. It’s not just about size; it’s about *what* the models are actually paying attention to.

Jane: Right. There’s been a lot of skepticism about these Centaur-style models. Critics said they might just be exploiting statistical shortcuts in the data, like patterns in the choice history, without really understanding the task. So the authors ran a clever set of experiments to test that.

Tom: And this is my favorite part. They systematically stripped away different parts of the prompt. They removed the task instructions, they masked the content of the stimuli, they even removed the feedback. And they found that if you take away the actual content—the shapes, the points, the feedback—the model’s performance collapses.

Jane: It drops below chance, actually. That’s a huge finding. It means the model isn’t just memorizing sequences of button presses. It’s using the information about what the participant saw and what happened as a result. The choice history alone is not enough.

Tom: Exactly. They call it the “content-masked” condition, and it destroys about seventy-five percent of the learned information. That’s the smoking gun that these models are processing task structure, not just statistical regularities.

Jane: And they even tested something called exchangeability. They took a task where the trials are independent, like picking the odd-one-out from a set of pictures, and they shuffled the order of past trials. A model that’s just learning sequences would be confused, but these models didn’t care. They knew the order didn’t matter.

Tom: But then they did the same shuffle on a task where the order *does* matter, like an adaptive test where the next question depends on your last answer. And the models were very sensitive to that shuffle. So they’re not just ignoring order; they’re using it only when the task demands it.

Jane: That’s task-adaptive information use. It’s a really strong piece of evidence that these models are learning something about the *logic* of the experiments, not just surface patterns. It makes them much more credible as scientific tools.

Tom: So we have these small, efficient models that seem to understand the tasks they’re trained on. The obvious next question is, what can we actually *do* with them? What’s the practical improvement here?

Improvements: Tom: So we’ve got these small, verified models. Jane, what’s the big improvement they offer over the current state of the art?

Jane: I think the biggest one is accessibility. The original Centaur model requires a serious amount of computing power to run. These small models, some of them can run on a single GPU, or maybe even on a laptop. That means a single lab, or even a single researcher, can now use a cognitive proxy without needing a supercomputer.

Tom: That’s a huge democratization of the tool. But they also frame it as something more specific: a “noise ceiling estimator.” Can you break that down for our listeners?

Jane: Sure. Think of it as a measuring stick for how predictable human behavior is in a given experiment. If you have a theory about how people make decisions, you can compare its predictions to what this model predicts. The model represents the best possible prediction you can make from the data.

Tom: So if your theory is worse than the model, it means your theory is missing something. There’s a gap.

Jane: Exactly. And that gap is a map. It tells you *where* your theory is failing, which experiments are more structured than your theory accounts for. That’s the “scientific regret minimization” idea from earlier work. This paper provides a cheap, accessible way to generate that map.

Meng: If I can jump in here, from a practical standpoint, the fact that they’ve released all the code and the model weights is huge. We can actually start testing these things in our own pipelines. The training times they report, even for the larger small models, are like a day or two on a single A100. That’s nothing compared to training a 70B model.

Tom: So it’s not just a theoretical improvement, it’s an engineering one. It lowers the barrier to entry so much that it changes what kind of projects are feasible.

Jane: And that’s the point. It moves these models from being a curiosity that only a few big labs can play with, to a standard tool in the cognitive scientist’s kit. But, of course, they’re careful to point out the limits. These models are only as good as the paradigms they’ve seen.

Tom: Right, they’re not a theory of cognition. They don’t explain *how* a choice is made, they just predict it very well. That’s a crucial distinction, and it’s a great note to end on for this segment. Let’s wrap up our thoughts.

Conclusion: Tom: Well, we’ve had a fascinating conversation about “Small Foundation Models of Human Cognition and Behaviour.” Let’s try to tie it all together.

Jane: I think the core message is that we don’t need to build bigger and bigger models to simulate human behavior. We need to build the right-sized model for the job, and then we need to verify that it’s using the information the way a human would. This paper does both.

Tom: It shows that a zero point six-billion-parameter model can match a seventy-billion-parameter model on familiar tasks, and that these models are genuinely processing task structure, not just memorizing sequences. That’s a powerful combination.

Jane: And it gives us a practical tool. A way to estimate the noise ceiling of an experiment and find where our theories fall short. It’s not a replacement for theories of cognition, but it’s a fantastic guide for where to build them.

Tom: It’s a bit like having a really accurate map of the territory before you start exploring. You know which mountains are actually there and which ones are just mirages.

Jane: That’s a nice way to put it. So we’re saying goodbye to this paper, but we’re taking that map with us. It’s a genuinely useful contribution to cognitive science, and we’re excited to see what people do with it.

Tom: Absolutely. A huge thank you to Nick Oh and Fernand Gobet for their work, and to our listeners for joining us. We’ll be back soon to break down the next big idea. Until then, keep thinking.

Nick Oh, Fernand Gobet

socius labs · London School of Economics and Political Science

cs.AI, cs.CY

Submitted: 2026-08-11

Updated: 2026-08-12

Comments: published as a conference paper at COLM'26

Code: https://github.com/socius-org/Centauri

License: http://creativecommons.org/licenses/by/4.0/

Importance score: 69/100

Key concepts

Small Foundation Models
The paper challenges the need for massive AI models to simulate human thought. The authors trained smaller versions of a behavioral model, demonstrating that these efficient, smaller models can match larger ones on familiar tasks.
Content-Masked Condition
This experiment tests if the model is merely memorizing data. By removing all the actual content (shapes, feedback) from a task prompt and running it through the small model, performance collapses significantly, proving it is not just learning sequences.
Noise Ceiling Estimator
This concept describes a tool that measures how predictable human behavior is in an experiment. It allows researchers to compare their theories against the model's predictions, revealing gaps where the theory fails to account for the data.
Task-Adaptive Information Use
The models are not just learning random patterns; they use past trial information only when the task demands it. This is shown by testing tasks where order matters versus those where it does not, proving they understand experimental logic.

Terminology

Summary

Summary

This paper investigates whether large language models fine-tuned on human behavioral data can serve as general-purpose cognitive proxies, focusing on two open questions: the necessity of model scale and the nature of the information these models use. The authors train fourteen models ranging from 135M to 14B parameters across four architecture families (Qwen3, Llama-3, SmolLM2/3, and OLMo2/3) on Psych-101, a dataset of 10.7 million trial-level choices from 160 experiments. They compare these models against a reproduced Centaur-70B baseline.

The first key finding concerns scale. For in-distribution simulations on Psych-101, model scale barely matters. The models fall within a narrow band of performance, as though against a ceiling, and models with 0.6B to 1B parameters suffice to match the 70B baseline on held-out participants. The paper states that whether a model has been fine-tuned matters roughly ten times more than how large it is. Specifically, fine-tuning gains average 0.188 nats across models, while scaling from 0.6B to 14B parameters gains only 0.019 nats. Adapter capacity (LoRA rank) substitutes for parameters; for example, Qwentaur-0.6B at rank 64 matches Qwentaur-14B at rank 4 with twenty-three times fewer parameters. The size needed to match Centaur-70B drops from 8B at rank 4 to 0.6B at rank 32 and above. Training data returns are also diminishing, with gains from each doubling of data declining steadily.

Out-of-distribution, the picture reverses. On Psych-201-RT, a set of 18 held-out experiments, the performance band opens into a markedly steeper scaling gradient, with larger models clearly advantaged in generalization to novel task structure. The eight matched models span 0.244 nats on Psych-201-RT compared to just 0.028 nats on Psych-101. The paper concludes that Model size should therefore be chosen for transfer rather than for in-distribution fit.

The second key finding concerns what information the models use. The authors decompose the prompts into four information channels: task instructions (I), experimental stimuli (S), outcome feedback (F), and choice history (C). They conduct structural ablations across 27 experiments, progressively stripping these channels. The results show that removing instructions costs little (12.5% of learned information), but masking the content of stimuli and feedback destroys 75.7% of learned information and pushes models below chance. Choice history alone does not account for performance, with mean retention under history-only below chance. This refutes both the history sufficiency and format sufficiency forms of the shortcut hypothesis.

The authors also conduct order permutation tests on two experiments with contrasting exchangeability structure. On THINGS odd-one-out, where trials are independent and exchangeable, fine-tuned models produce order-invariant predictions. On intertemporal choice, which uses an adaptive staircase procedure where trial order is determined by prior responses, fine-tuned models are order-sensitive. The paper concludes that the picture that emerges is one of task-adaptive information use.

The paper positions these models not as theories of cognition in the tradition of ACT-R, CHREST, or Soar, since they posit no architecture, commit to no mechanism, and explain nothing about how a choice is generated. Instead, they serve as noise ceiling estimators for psychological experiments, providing an empirical bound on how much behavior is predictable and thus where process-level theories have room to improve. The paper notes that what binds them is the reach of that coverage rather than their capacity, and that their utility is bounded by the paradigms seen in training data.

Improvements for AI systems

Based on the paper, here are specific improvements to AI systems and the resulting capabilities:

Improvement: Implement a dynamic scaling strategy that separates in-distribution fit from out-of-distribution transfer. Instead of defaulting to the largest model (70B), use a small model (0.6B–1B) with high LoRA rank (r=32–64) for tasks within the training distribution, and reserve larger models (8B–14B) only for novel task generalization.

Resulting Capability: An AI system that achieves Centaur-70B-level accuracy on known cognitive tasks (Psych-101) with 20–100x fewer parameters, reducing inference cost and latency, while retaining superior performance on unseen tasks (Psych-201) where scale matters.

Abstract

Large language models fine-tuned on human behavioural data have emerged as general-purpose cognitive proxies, but the scale this requires, and whether these models process task structure or exploit statistical shortcuts, remain open questions. We train fourteen models from 135M to 14B parameters across four architecture families on Psych-101, a dataset of 10.7 million trial-level choices from 160 experiments. For in-distribution simulations, scale barely matters. The models fall within a narrow band, as though against a ceiling, and 0.6B to 1B parameters suffice to match a 70B baseline on held-out participants. Out-of-distribution, that band opens into a markedly steeper scaling gradient, with larger models clearly advantaged in generalisation to novel task structure. To determine what information these models use, we run two diagnostics. We progressively strip four prompt channels -- task instructions, experimental stimuli, outcome feedback, and choice history -- across 27 experiments, and permute trial order. Masking the content of stimuli and feedback destroys 75.7% of learned information and pushes models below chance, demonstrating that choice history alone does not account for performance. Permutation reveals invariance on tasks with independent trials but sensitivity where trial order is determined by prior responses. Small cognitively fine-tuned models therefore show promise as noise ceiling estimators for psychological experiments, though their scope remains bounded by the paradigms seen in training.

Sources

Related papers