Mimicry without understanding: the origins of decision bias in large language models

Summary

Large language models (LLMs) generate responses through probabilistic modeling of linguistic patterns learned from large-scale datasets. Initially, it has been argued that if these linguistic patterns include the laws of rationality (i.e., probabilistic reasoning, Bayesian updating, etc.), LLMs should display economic rationality far exceeding that of humans [1]. Yet it was soon found out that in many domains LLMs exhibit considerable human-like biases. In the social domain these include, for instance, self-other attribution gaps [2], ingroup favoritism and outgroup disparagement [3], gender predispositions [4,5], and even political biases (reportedly favoring Democrats in the US, Lula in Brazil, and the Labour Party in the UK [6]). In non-social judgments LLMs were shown to rely on completely irrelevant information to make judgments (i.e., anchoring [7-10]), judge the probability of events based on anecdotal information (availability heuristic [8]), estimate the probability of two co-occurring events as higher than the likelihood of either of them alone (i.e., representativeness heuristic [8]), and dichotomize statistical evidence [11]. Even in economic decisions between lotteries defined by their probabilities and outcomes, where expected-value (EV) calculations are relatively easy, LLMs exhibit consistent biases, such as overweighting small-probability events in decisions from descriptions and underweighting these events in decisions from experience, as humans do [12-13].

The present paper suggests two processes of bias generation that go beyond sheer mimicry of humans’ biased preferences and lead to extreme sensitivity to documented human behaviors that form the corpus of the LLM’s training data. The first such process is faulty interpretation of human preferences from human behavior. Previously, it has been proposed that LLM biases are the product of mimicking the behavior of people described in the corpus of training data, while completely ignoring the rules of rationality [14]. In other words: LLM see, LLM do. We argue that while this may indeed be the case, such mimicry emerges even in cases where human behavior is not indicative of relevant preferences. For example, if I choose two US dollars over one yen, this doesn’t actually bear on my weighing of same-number dollars and yen, besides setting a weak constraint (2 yen < 1 USD); yet, we posit that LLMs consider the sheer behavior of choosing as indicative of preference, in this case inferring that USD are preferred over yen, independently of logical considerations.

The second process that arguably facilitates biases in LLMs is that mimicry of human behavior persists even when biases are not implicit but are explicitly described as biases, namely when there is an explicit disclaimer that the relevant behavior exemplifies bias. We suggest that this process potentially leads to a bizarre phenomenon whereby scientific empirical studies of biases, which clearly describe human participants’ behavior as being biased, paradoxically facilitate the biases exhibited by LLMs. Moreover, given LLMs’ sensitivity to anecdotal information noted above, such facilitation of biases may emerge even when human data is based on a rather small sample, namely when individuals’ behavior does not reliably represent the population.

Importantly, the two processes described above may not operate unconditionally. Their influence could be modulated by the extent to which mimicry conflicts with rational decision laws. One such universal law is expected-value maximization, namely picking the option that provides the best outcome based on its expected value (probabilities multiplied by the outcomes they yield). Under this account, the tendency to mimic human biases would be curbed, at least partially, when doing so reduces the LLM’s expected outcomes: For instance, if a person is biased to give losses more weight than gains, then they are said to be loss averse [15]. Given that an LLM is informed that humans behave in a loss averse manner, then under the former account (LLM see, LLM do) it would display loss aversion irrespective of the consequences. Alternatively, under the notion of expected-value minded mimicry, the LLM may display loss aversion when there are no downsides to being loss averse (in terms of expected value), but less so when exhibiting this bias is considerably disadvantageous.

In four studies, we evaluated the emergence of the two faulty processes of mimicry of human behavior, and their boundary conditions. Our first study used a setting where LLMs are exposed to subjective human preferences between currencies that are not consistent with their objective value. Specifically, we tested whether a safe (fixed amount) monetary option is judged more attractive when denominated in people’s preferred currency, even though the relevant currencies are objectively equivalent (1:1 exchange rate). In other words, we evaluated whether LLMs would display a social proof bias: Assuming that something is good or correct simply because others endorse it [16-17]. Social proof bias is not necessarily irrational or illogical because LLMs may assume there are some unknown reasons to prefer one of the currencies. In our second study we therefore considered a setting where people’s behavior is clearly non-indicative of their preferences (similar to the 2 USD > 1 yen example above). We examined whether LLMs would show a social proof bias in this setting as well, which would be completely at odds with logical inference.

Our next studies evaluated the process of mimicry where the adopted human behavior is explicitly described as a bias. We focused on loss aversion, the tendency to overweight losses compared to corresponding (i.e., symmetric) gains [15]. Loss aversion is considered a bias because from a strictly objective point of view, removing a certain amount of money and adding the same amount should have the exact same cost/benefit, especially for small stakes that do not have additional externalities. In recent empirical studies it has been disputed whether people are loss averse on average [18-23]. We conjectured that LLMs’ loss aversion would mimic that of humans, even if the relevant group of humans is very small. In Study 3, we evaluated if LLM agents prompted with experimental results that people are either loss averse or gain seeking (i.e., the opposite of loss averse), adopt the same bias given various costs, and when presented with the behavior of a large or a small sample. A similar degree of mimicry of a small and large sample attests to shallow copying since it does not take into consideration that average behavioral trends, particularly in small samples, may be driven by random noise.

In our final study we examined the effect of prompting LLMs with texts from scientific papers that explicitly describe loss aversion as a bias, and that report either strong or weak loss aversion, and testing the models’ subsequent loss-aversion bias. Importantly, this enabled testing whether an LLM would copy the human bias (loss aversion) despite the clear indication in the scientific paper that being loss-averse is a “disproportionate” (p. 585) tendency [22] which reduces expected value in the studied decision tasks.

Our studies focused on ChatGPT-4o [24] and Qwen 2.5-72B-Instruct [25]. ChatGPT is one of the most culturally prominent LLMs available today (with over 800 million weekly active users globally by 2025 [26]). Much of the research demonstrating biases in LLMs, reviewed above, was conducted on ChatGPT. We also replicated our initial studies with an open source LLM, Qwen [25].

Results

Study 1 showed that even though LLMs were informed that two currencies are objectively equal, they nevertheless assigned more value to the currency preferred by humans. This social proof bias diminished when the human-mimicked preference implied losing money, namely upon making a decision that goes against expected value. Still, even when expected value was undermined, the adopted bias did not completely disappear. Study 2 went further and examined whether in the same setting LLMs would prefer the currency chosen by humans even though their choice bears no logical implication for their preference (since the chosen currency type was confounded by its size). Interestingly, we found a similar magnitude of mimicry of human behavioral choices in this study. The LLMs preferred the chosen currency despite no logical reasons to do so, following human behavior at the surface level without correctly inferring preferences.

Study 3 was designed to evaluate whether LLMs would mimic a clearly biased behavioral tendency of experimental participants. As in Study 1, the bias reported in the human experiment affected that exhibited by the LLM. If humans were described as loss averse, LLMs showed more loss aversion than if humans were described as gain seeking. Importantly, though, LLMs’ tendency to mimic human biases was not sensitive to the size of the sample in the human experiment presented to them: LLMs’ loss aversion bias was about equal when the relevant report of human behavior was based on a very large sample (> 10,000) or a very small sample (<50). In the latter case, individuals’ behavior may not reliably represent that of the population: Indeed, for this to occur the effect size of loss aversion would have to be exceptionally large. Hence, this again reflects a process of copying human behavior without applying logical – in this case, statistical – inferences.

Also, as found in Study 1 and 2, the mimicry tendency was moderated by the expected value of the biased behavior. Yet, the adoption of human biases was not highest at the objective indifference point, namely where being loss averse did not affect expected value, but rather at a point where the loss was slightly smaller than the gain (which is the indifference point given weak loss aversion). This suggests a refinement on the expected-value moderation hypothesis: Mimicry seems to be largest when the bias leads to the smallest deviation from the model’s expectancy (which may be biased) rather than expected value: For an already slightly biased model, the point of indifference is based on the existing bias (the fact that LLMs were slightly biased towards loss aversion was further established in Study 3 control condition). This highlights the need to consider the LLM’s own reference point when quantifying economic indifference [31].

Finally, Study 4 used actual scientific abstracts (Study 4a) and papers (Study 4b) to test whether despite the scientific portrayal of the respective human behavior as a bias, LLMs would still mimic the biased human behavior. The results showed that both ChatGPT’s and Qwen’s loss aversion was significantly and substantially higher as a function of the degree of loss aversion reported in the scientific paper. For example, without relying on any scientific paper ChatGPT had only a very minor loss-aversion bias: Its loss aversion parameter was 1.09, implying about 10% greater weighting of losses than gains. Conversely, after being exposed to Brown et al.’s [22] abstract reporting strong loss aversion, ChatGPT’s loss aversion parameter was 1.79, implying that it gave losses about 80% more weight. Even when exposed to the complete report of Brown et al. [22], ChatGPT’s loss aversion parameter was 1.52, denoting about 50% greater weighting of losses. Much weaker loss aversion was exhibited when the LLM based its response on papers reporting moderate or no loss aversion. Thus, paradoxically, the human studies that highlighted the strongest deviation from rationality produced LLM behaviors that were the least consistent with rationality, leading to a seeming self-fulfilling prophecy.

Our findings thus indicate extensive mimicry of human biased behavior by LLMs which is surface level and illogical. It occurs when individuals’ choice behavior is not representative of the population preferences and even when it is not indicative of their own preferences, and it emerges even where choice behaviors are explicitly labelled as being biased in scientific papers.

Follow-up studies should address whether and to what extent this type of mimicry is susceptible to debiasing. For example, one avenue of debiasing in human studies is requesting individuals to simultaneously consider a list of similar decisions varying in their relevant parameters. This is known as “joint evaluation” or “consider-the-alternative” [32,33]. We implemented this type of debiasing by re-running Study 2 with ChatGPT agents who simultaneously provide accept/reject responses for all lotteries A-F, instead of separately after each lottery (see supplementary section for details). The results, summarized in the supplementary section, showed that the difference between conditions in the LLM’s responses (as a result of experimental participants' behavior), became evident only when loss aversion did not conflict with expected value calculations, namely when losses and gains were equal. Thus, while popular and scientific documentation of human biases can exacerbate LLM biases, proper design of prompts and algorithms inspired by the science of behavioral economics may substantially reduce this tendency.

Based on the paper, here are specific improvements for AI systems:

Improvement: Add a verification layer that checks whether reported human behavior logically implies a preference before allowing it to influence decisions.

Specific implementation:

• When processing statements like people preferred 12 Tenits over 11 Tanas, the system should parse the numerical relationship and recognize that this does NOT establish a preference for Tenits over Tanas—it only sets an upper bound on Tanas' relative value.

• Add a rule-based checker: if option A > option B in quantity, but the system cannot determine if A is preferred over B independent of quantity, flag it as non-indicative and exclude from preference modeling.

• Result: The system would not exhibit the 72% mimicry bias observed in Study 2, where LLMs incorrectly inferred preferences from confounded choices.

The improved AI system would:

• Not infer preferences from logically non-indicative behaviors (eliminating Study 2's 72% bias)

• Not treat small-sample results as reliable (eliminating Study 3's sample-size insensitivity)

• Not adopt biases explicitly labeled as biases in scientific texts (eliminating Study 4's 51-105% amplification)

• Only mimic human behavior when it aligns with expected-value maximization

• Self-correct its own baseline biases before processing external data

• Parse scientific reports for statistical significance and effect sizes before allowing them to influence decisions

These changes would reduce the observed biases by an estimated 80-90% across all four study types, while preserving the LLM's ability to learn genuinely useful human preferences (e.g., cultural norms, risk preferences in ambiguous situations) when they are logically valid and statistically reliable.