Poor Man's Agentic Modeling: Simulating Large LLM-Agent Societies on a Laptop

Summary

The paper introduces a method for simulating societies of many large language model (LLM) agents at low cost by replacing each expensive LLM agent with a low-parameter surrogate model fitted from a small number of cheap queries. The central claim is that whether this works is decided before the simulation runs, chiefly by what each agent perceives. The authors introduce an [interaction order × memory] taxonomy that maps perception and memory to an effective theory and a predicted N-trend of the surrogate error.

The method is validated on "a faithful reimplementation of the LLM macroeconomy EconAgent and seven further named LLM simulations, with agent decisions cloned from genuine LLM elicitations (primarily DeepSeek) for a few dollars; the predicted error trends hold cell by cell, and the two refuted predictions, both on a strongly saturating response and traced to its curvature, are themselves matched quantitatively by the theory with no free parameters."

The paper's contribution is a criterion for when the statistical-physics stance applies to LLM societies. The deciding property is perception: "An agent that reacts to one population-wide aggregate (an inflation rate, a global trending feed) sits in a mean-field regime, and a scalar surrogate reproduces the macroscopic observable with an error that vanishes as N −1/2. An agent that reacts to a signal shared only within its community, or only to its graph neighbours, sits in a regime where the same scalar surrogate carries an error that does not vanish, and may even grow with N. The taxonomy maps the perception and memory design of a simulation to a cell, an effective theory, and a predicted N-trend of the surrogate error. Because the perception design of a social simulation is in practice set by its recommender, the recommender is the switch between the regimes in which cheap modelling succeeds and those in which it fails."

Validation occurs in three ways: "First, on a faithful, code-authoritative reimplementation of the LLM macroeconomy EconAgent, we show that the surrogate reproduces the target’s macroscopic signatures and that doing so exposes what those signatures do and do not measure. Second, we falsify the taxonomy directly: its cell assignments are pre-registered and then tested blind, both on held-out contact graphs and on the measured response function of a real LLM. Third, we classify and reproduce eight named LLM simulations spanning all three perception cells: EconAgent, AgentTorch, OASIS, AgentSociety, De Marzo et al.’s consensus game, Williams et al.’s generative epidemic, LLMTraveler’s congestion game, and Generative Agents’ Smallville (with TwinMarket’s financial stylised facts as a documented boundary case), together with a cross-domain check against a differentiable agent-based model, using agent responses cloned throughout from genuine LLM decisions."

Two findings of independent interest emerge. First, "EconAgent’s frequently cited reproduction of Okun’s law turns out to be an accounting identity that a behaviour-free policy already satisfies, whereas its Phillips curve is a genuine behavioural signature carried by a single labour-cyclicality coefficient; we estimate that coefficient from cloned decisions and recover the macroscopic value as an out-of-sample prediction. Second, when the pipeline is driven by genuine LLM decisions, a 2 × 2 ablation isolates the reasoning step—not the wording of the prompt—as the cause of the emergent Phillips curve, so the cheap surrogate becomes an instrument: what makes the macro law appear is itself measurable."

The paper connects several literatures: "Expensive LLM simulations such as Generative Agents, EconAgent, OASIS, and AgentSociety establish that LLM societies reproduce human-like macroscopic phenomena and define the observables a surrogate must hit, but none is paired with a low-parameter model whose scaling is analysed. Sociophysics supplies off-the-shelf few-parameter rules, and The closest prior work is MF-LLM, which couples a population-level mean field to per-agent LLM decisions; it keeps the LLM in the loop and analyses no scaling, whereas we replace the agent and study the macroscopic observable as N varies."

The taxonomy is formalized: "Let Φt denote the microscopic update of the LLM society and P the projection onto the macroscopic observable of interest. A cheap surrogate replaces Φt by a low-parameter map Φ̂t acting on the projected variables. The surrogate reproduces the observable exactly when coarse-graining commutes with the dynamics, P Φt = Φ̂t P; in general it does not, and the size of the commutation defect ∥P Φt − Φ̂t P ∥ is what the taxonomy predicts. The surrogate error is the resulting gap in the macroscopic observable."

The taxonomy has two axes: interaction order and memory. "A global aggregate feed is order zero (every agent sees the same population statistic) and yields a mean-field theory in which the scalar surrogate error is set by sampling noise and vanishes as N −1/2. A community feed, shared within each of a fixed number of blocks, is a heterogeneous mean field: the error no longer vanishes but falls only with the number of blocks, leaving an O(1) floor in N. A local feed, restricted to graph neighbours, is genuinely k-body, and the error is controlled by the degree structure rather than by N. Memory determines whether an agent’s decision depends only on current inputs or on an accumulated internal state. Long memory makes the dynamics non-Markovian and requires a memory kernel in the closure."

Two further axes refine the picture: "A shared driver that itself fluctuates makes the mean field random, so its fluctuations fall more slowly than the naive rate. And a strongly curved per-agent response makes coarse-graining fail via Jensen’s inequality even under a private feed. The commutation defect is bounded, heuristically, by a sum of an interaction-order term, a memory term, and a response-curvature term."

Three propositions are proved. Proposition 1 (Community floor): If A is affine, then Floor(B) = EN (m1, v1 /B); that is, the scalar mean-field surrogate error floor equals exactly the mean of a folded normal distribution. Consequence 1: "If the response is symmetric about the operating point, then D is odd in the mean-zero perturbation δ, so m1 = ED = 0 and Floor(B) = p 2v1 /(πB) decays as B −1/2. If the response is curved, then m1 ̸= 0, the decay stalls at Floor(∞) = m1. Proposition 2 (The knee N ∗): Under a private feed the scalar surrogate error equals EN (m1, v1 /N). It decreases as N −1/2 up to the knee N ∗ = v1 /m21 ≈ 4 A′ (g ∗)2 /(A′′ (g ∗)2 σ 2), and plateaus beyond it at the curvature floor m1 ≈ 21 A′′ (g ∗)σ 2. Proposition 3 (Exact commutation at the mean-field cell): For any response f, E[gt+1 gt] = a gt + f (gt) =: Φ̂(gt). The commutation defect ∥P Φt − Φ̂t P ∥ therefore vanishes as N → ∞."

A propagation lemma (Lemma 1) bounds the trajectory error: "Suppose the surrogate map Φ̂t is L-Lipschitz for every t and the one-step defect satisfies εt ≤ ε for every t < T. Then the trajectory error obeys eT ≤ ε (L T − 1)/(L − 1)."

The method is a single procedure (Algorithm 1): "classify the simulation’s perception cell, which predicts the N-trend of the surrogate error before any fitting; screen the observable; elicit a few hundred to a few thousand genuine LLM decisions on the target’s own prompts; clone a low-parameter surrogate; read off its error floor and knee from the fitted response; and run the surrogate society to large N on a laptop, validating the macroscopic observable and checking the error trend against the cell’s prediction."

The primary target is EconAgent. EconAgent’s published targets are a Phillips correlation of −0.619 and an Okun correlation of −0.918. The a-priori test separates them: "In EconAgent real GDP is, by construction, an affine-invertible function of the number of working agents (an R2 = 0.996 fit), so Okun’s law relates a quantity to an affine image of itself; a behaviour-free Bernoulli(0.5) work policy already yields an Okun correlation of −0.998. Reproducing Okun therefore validates nothing. The Phillips curve is genuine: wage inflation is driven by goods-market imbalance with no direct employment-to-wage channel, so a negative unemployment–inflation relation is a genuine behavioural signature. Its mechanism is a single procyclical-labour coupling: work propensity rising with the price signal. Fitting the twelve-parameter student to a procyclical teacher recovers that coupling, and the macroscopic Phillips correlation, which never enters the fit, emerges as an out-of-sample prediction at −0.569 ± 0.138, within noise of both the teacher and the published target (∆ = 0.05)."

Identifiability is a frontier: Only labour keyed to the price signal reaches the published value, and near that frontier the channel is nearly pinned. The paper maps the reachable Phillips frontier of three candidate micro-channels (Table 1).

Driving the pipeline with genuine LLM decisions: The runs cost 0.44 and 0.67 for three and six thousand decisions. The headline: asking the model to reason is what produces the Phillips curve. A 2 × 2 ablation crosses reasoning with wording: with no reasoning the cloned Phillips is weak and even flips sign with wording (−0.43 plain, +0.04 amplified), whereas with reasoning it is strongly negative under both wordings (−0.73 and −0.66). The reproduction is model- and reasoning-conditional: Under a reasoning prompt DeepSeek-chat’s cloned Phillips is −0.665 ± 0.12, the nearest cell landing ∆ ≈ 0.05 from the published −0.619; but sibling models under the same prompt scatter to −0.78 and −0.84.

Closure-machinery checks on known ground truth: "On epidemic dynamics over contact graphs, a scalar mean field is accurate on a complete graph, degrades on a k-regular graph, and fails near threshold on a scale-free graph and on a real Facebook network, exactly as the interaction-order axis predicts. On the Minority Game, a finite-size-scaling data collapse recovers the critical control parameter to within 13%."

A blind test of the taxonomy on held-out graphs: All three predictions held. The mean-field cell’s error shrank with N, the community cell’s settled to an O(1) floor, and the local cell’s tracked the degree structure.

A blind test on the LLM perception layer: "On the near-linear consumption head all five pre-registered predictions held: the global and private errors fall as N −1/2, the community error is flat in N at an O(1) floor, that floor falls as B −1/2 in the number of communities, and a block-aware surrogate repairs it. The strongly saturating work head refutes two of its five predictions... its private-feed error does not shrink but sits at a floor of 0.018. The cause is response curvature: the infinite-population Jensen bias Eε f (g ∗ + ε) − f (g ∗) equals 0.018, matching the floor. This confirms Proposition 2: the work head’s fitted response gives a knee N ∗ = v1 /m21 ≈ 29, so its private-feed error should already be flat at the floor by N = 100, as observed across N = 100 to 3200; the consumption head gives N ∗ ≈ 2.5 × 104, so it keeps improving throughout."

A recommender dial: "Holding the total misperception variance fixed and letting a knob λ set the fraction that is community-shared rather than private sweeps a real DeepSeek society across the mean-field boundary... the near-linear head sweeps cleanly from a vanishing error at λ = 0 to an O(1) floor at λ = 1, the saturating head floors at the Jensen level for all λ."

Cross-model robustness: The three predictions that define coarse-graining—the global feed averages out, the correlated community feed leaves an O(1) floor, and a block-aware closure repairs it—held for twelve of the thirteen models tested. The single exception is gpt-4o-mini, the smallest model in the panel.

De Marzo et al.: "a published universal result: an LLM shown its peers’ opinions adopts the majority with probability P (m) = 12 [tanh(βm) + 1], governed by one majority-force β, with consensus... requiring β > 1 and a critical group size Nc where β(N) = 1. The paper proves (Proposition 4): The iterated response m 7→ tanh(βm) is odd with Jacobian β at m = 0. Its disordered fixed point m = 0 is therefore stable iff β < 1 and loses stability at βc = 1 through a supercritical pitchfork... Hence consensus exists iff β > 1, independently of N. Consequence 5: A finite critical group size therefore exists if and only if the measured slope βeff (N) decays through 1. The slope βeff (N) is the resolution with which an agent reads a weak majority in a list of N opinions, so Nc is a perception threshold, not a thermodynamic one. Models split: the reasoning models Opus-4.8 and GLM-5.2 count perfectly (βeff pegged, Nc = ∞), DeepSeek holds a flat βeff ≈ 1.9 > 1 (Nc = ∞), while GPT-4o, Llama and GPT-4-Turbo show a decaying βeff and hence a finite Nc. An intervention confirms: Handing GPT-4o the explicit tally (nk, nz) in the prompt... flattens its βeff (N) from a decay to a pegged constant, sending Nc → ∞."

A measured memory kernel: "Eliciting DeepSeek attitude updates given a controlled history of past interactions, we recover a discrete Mori–Zwanzig kernel by regression... All three pre-registered predictions held. The current-interaction weight K(0) = 0.27 is in the range of the independently fitted µ = 0.415 and, tellingly, below it... The past-interaction tail is real, at 47% of K(0), so the update is genuinely non-Markovian. And the tail is concentrated in the first few lags and vanishes beyond."

The external validation suite (Appendix E) covers further targets. AgentTorch: "the isolation behaviour is a global feed, so the archetype error shrinks with the number of archetypes, whereas contagion runs on a contact graph, where a well-mixed surrogate over-predicts the peak and the break is driven by clustering rather than by the degree tail. OASIS: Its Reddit hot-score feed is a single global leaderboard: the cloned herd experiment converges to its mean-field value with an error that effectively vanishes. Its interest feed is a per-community echo chamber: the group-polarisation error is O(1) and falls as B −1/2 in the number of communities. AgentSociety: sits in the local, long-memory cell. A scalar mean field cannot represent its between-block polarisation at all... A pair-plus-memory closure repairs it, and an ablation of the conviction memory collapses most of the polarisation. A cross-domain check against GradABM: we lift a heterogeneous mean field... and calibrate two parameters by a derivative-free simplex. It recovers the transmission rate to within 13% and the fatality rate to within 6% at a mortality-curve RMSE of 22, in 0.34 seconds on a laptop. A distillation scaling law: The error falls with budget, reaching tolerance by a couple of thousand decisions; it falls with capacity and plateaus at four features... below four features the error plateaus above tolerance for any budget. The classification is automatable: an LLM emits the structured specification that classify cell consumes; the resulting cell matched the hand assignment in all eight cases we tested. Williams et al.: From 462 real DeepSeek decisions on their verbatim prompt we reproduce all three levels... The fitted logistic has the published sign structure, including the negative squared term (−1.77). LLMTraveler: We elicit real DeepSeek route choices and fit a two-parameter rule P (switch) = σ(β ∆ − γ)... Run as a day-to-day dynamic on the 16-traveler two-route network, it converges to the DUE at +4.7%, inside the published ±10% band. TwinMarket is a documented boundary case: the returns stay near-Gaussian (excess kurtosis ≈ 0), because the elicited herding leaves the market subcritical... Raising that coupling makes the same weak trader supercritical (excess kurtosis rises past 7), so the stylised facts are under-determined by the elicited response alone. Smallville: at a plausible acquaintance degree (∼ 6) the cascade reaches 11.4 of 25 (published 13) and 4.5 attend (published 5)... a well-mixed control that lets every informed agent talk to everyone reaches all 25, overshooting, whereas the local cascade does not."

The paper concludes: The recurring lesson is that the surrogate’s fit is a measurement: the microscopic property that a macroscopic observable depends on is exposed, not hidden, by replacing the expensive agent with a cheap one.

Based on the paper, here are specific improvements I can implement in AI systems:

• Improvement: Add a pre-flight classification layer that maps any multi-agent LLM simulation's perception design (global/community/local feed × memory length) to a predicted error-scaling regime before running expensive simulations.

• What the improved system can do: Automatically decide whether a cheap low-parameter surrogate (2–12 parameters) can replace each LLM agent for macroscopic questions, or whether a block/pair/memory closure is required. This cuts simulation costs from tens of dollars to cents and enables laptop-scale studies at any N.

• Improvement: Extend the surrogate fitting pipeline to compute the response curvature (A″) and the resulting knee N* = 4A′2/(A″2σ2) from fitted responses, flagging when the surrogate error will plateau regardless of N.

• What the improved system can do: Predict, before simulation, whether a private-feed society will keep improving with population size (N* large) or saturate at an O(1) floor (N* small), preventing false confidence in scaling results.

• Improvement: Implement the 2×2 ablation (reasoning on/off × wording plain/amplified) as a diagnostic tool in any LLM-agent pipeline, with byte-identical numeric inputs across cells.

• What the improved system can do: Determine whether an emergent macro-phenomenon (e.g., Phillips curve, epidemic flattening) is caused by the agent's reasoning step or merely by prompt phrasing—turning the surrogate into an instrument for mechanism discovery, not just cost reduction.

• Improvement: Replace finite-size-scaling analysis with a perception-threshold analysis: fit β eff(N) from LLM decisions and detect when it crosses β c=1, rather than assuming a thermodynamic critical point.

• What the improved system can do: Correctly identify whether a finite critical group size in opinion-dynamics is a perception limit (list-to-count resolution decay) or a true phase transition, avoiding spurious data-collapse claims and guiding interventions (e.g., providing explicit tallies to restore consensus).

• Improvement: Add a regression-based Mori–Zwanzig kernel estimator that fits K(τ) from controlled history-elicitation traces, then uses it in a pair-plus-memory closure.

• What the improved system can do: Reproduce polarisation dynamics in local, long-memory societies (e.g., AgentSociety) with a bounded O(1) error that grows only with densification, not with N—enabling accurate large-N studies without running the full LLM society.

• Improvement: Expose a single knob λ (community-shared fraction of the feed, at fixed misperception variance) that sweeps a society across the mean-field boundary.

• What the improved system can do: Let a system designer tune a recommender to either (a) keep macroscopic observables predictable (λ→0, error vanishes as N-1/2) or (b) deliberately induce O(1) floors for diversity/polarisation—with the exact error trend known in advance.

• Improvement: Implement the scaling law (error ≈ rank-one separable in budget B and capacity p, with a capacity floor) to recommend minimal elicitation budgets.

• What the improved system can do: Automatically purchase the cheapest configuration (e.g., 4 features, 1,000–2,000 decisions, <1) that meets a target macro-observable tolerance, avoiding wasted API spend on capacity-limited or data-limited regimes.

• Improvement: Add a pre-registered panel test across 13 traces from 5 labs (DeepSeek, OpenAI, Anthropic, Google, Meta) that verifies whether the coarse-graining predictions (global feed averages out, community feed floors, block closure repairs) hold for a given model.

• What the improved system can do: Flag models (like gpt-4o-mini) where block-aware closures fail by the factor-of-two criterion, so users know when a surrogate will not scale—before committing to a simulation campaign.

• Improvement: Use an LLM to read a neutral description of any simulation's perception/memory design and emit a structured spec that maps to the taxonomy cell, without leaking taxonomy terms.

• What the improved system can do: Automatically pre-flight-check any new LLM-agent simulation for surrogate feasibility in seconds, with 8/8 agreement on named published systems.

• Improvement: Add a diagnostic that detects when a macroscopic stylised fact (e.g., fat-tailed returns) is under-determined by the elicited agent response alone and requires an un-elicited market coupling parameter.

• What the improved system can do: Prevent false claims that a surrogate reproduces a phenomenon when it actually depends on unmeasured mechanism strength—saving researchers from publishing spurious reproductions.

Bottom line: These improvements turn the paper's taxonomy and measurement techniques into a reusable toolkit that makes LLM-agent societies cheap, predictable, and mechanistically interpretable—at any scale, on a laptop.