Do LLMs Choose Like Humans? Using Cognitive Theory to Evaluate LLM Decision-Making

arXiv:2609.22225 · cs.CL, cs.LG · Submitted 2026-09-02 · Read on arXiv

cs.CL, cs.LG

Submitted: 2026-09-02

Updated: 2026-09-02

Comments: Accepted to EMNLP Findings 2026

License: http://creativecommons.org/licenses/by/4.0/

The gist: Large language models (LLMs) exhibit a range of human-like decision-making behaviors, but whether these reflect similar underlying mechanisms or surface-level mimicry remains unclear.

Terminology

Abstract

Large language models (LLMs) exhibit a range of human-like decision-making behaviors, but whether these reflect similar underlying mechanisms or surface-level mimicry remains unclear. We evaluate whether LLM context sensitivity aligns with a cognitive economic theory that explains human behavior through problem categorization and attention allocation. Across 12 open-source and commercial LLMs on a novel 140,000-trial product choice benchmark, context induces human-like shifts in choice and problem categorization, but does not reliably reweight attention between features like price and quality. Neither scale nor chain-of-thought reasoning reliably attenuates context sensitivity or generates human-like behavior. These results suggest that LLM decision mechanisms are distinct from human ones.

Related papers