Beyond the Black Box: Interpretable Models of Human Randomisation Failures

arXiv:2608.07220 · cs.AI · Submitted 2026-08-07 · Read on arXiv

Listen

Radio episode about this paper

Transcript

Introduction to the show: ident: Paper Radio. Generated commentary on the latest Artificial Intelligence papers.

Tom: Next we'll be talking about the paper "Beyond the Black Box: Interpretable Models of Human Randomisation Failures".

Jane: The paper was written by Ngoc Linh Dao from Alpen-Adria-University of Klagenfurt.

Tom: Stay tuned as we take you through the paper and discuss its implications.

Paper summary: Tom: We've got a fascinating one today — a paper that takes a classic economics experiment and asks whether transparent models can match what a black-box neural network can predict. It's all about why humans are so bad at being unpredictable.

Jane: And the short answer is yes, mostly, with a really interesting caveat about what that predictability actually is. The authors use data from a card game where people should play randomly, and they recover most of the predictive power of an LSTM using simple, readable models.

Lu: The dataset is huge by experimental economics standards — 84,060 decisions from 2,802 pairs over 30 rounds. And the game is O'Neill's four-card game, which has a sharp theoretical benchmark: in equilibrium, choices should be independent across rounds.

Meng: So any detectable pattern is, by definition, a departure from the benchmark. The question is what kind of departure — do people avoid repeating themselves, do they track their opponent, are they learning payoffs, or are they chasing frequencies?

Lalam: And the paper's answer is that one mechanism dominates: people manage their own recent action histories — repeat and avoid behaviour. That accounts for most of the explainable signal, and it's also the strategically exploitable signal.

Tom: So it's not deep strategic sophistication hiding in the data. It's a simpler vulnerability — players trying to cover their tracks, and in doing so leaving traces of exactly what they did.

Jane: The headline numbers are striking. Their extended EWA model recovers about 89 percent of the LSTM's improvement over a constant baseline for Red players, and 98 percent for Black players. That's with a model you can open up and read.

Lu: And it matters beyond this one game, because being unpredictable shows up in poker, in security, in any setting where a human has to randomise against an adversary. If the failure is structured, that structure is exploitable.

Meng: I also appreciate that they're honest about the limits — they admit the improved models might have captured different regularities than the LSTM and just happened to perform similarly.

Lalam: Which is exactly the right caution. The paper moves the debate from "can we predict" to "what exactly are we predicting" — and that's a much more useful question.

Tom: To see how they got there, we should go back to the beginning of the paper, where the game and the puzzle are laid out in detail.

Page 1: Tom: So page one sets up the game carefully. There are two roles — Red and Black — and four cards: the numbers one, two, three, and the face card K.

Jane: Red wins if both players pick K, or if both pick different number cards. Black wins if exactly one player picks K, or if both pick the same number card. It's zero-sum, so one player's win is the other's loss.

Lu: And the equilibrium is sharp: each number card should be played with probability 0 point 2, and K with probability 0 point 4, independently every round. That independence is the key benchmark — if past actions help predict future choices, you've left the equilibrium.

Meng: The paper points to an old tension in the literature. O'Neill's original 1987 study found aggregate play was close to the minimax benchmark, but Brown and Rosenthal in 1990 re-examined and showed individual choices were still history-dependent.

Lalam: And that's the gap this paper sits in — the aggregate looks fine, but the individual sequences leak information. Recently, Hirasawa and colleagues used LSTMs to predict those deviations out of sample, and they did very well.

Tom: Which raises the core question, stated right there on page one: what is that predictability made of? Is it own-action habits, opponent tracking, payoff learning, or frequency tracking?

Jane: They also borrow a line from Cynthia Rudin — the way forward is to design models that are inherently interpretable, rather than explaining black boxes after the fact. That's the methodological stance of the whole paper.

Lu: I find it telling that all models are estimated separately by role, because the equilibrium itself is asymmetric — Red wins 40 percent of the time, Black 60 percent. So the authors don't assume the two roles fail in the same way.

Meng: That also means the incentive structure differs by role. If you're Black and winning more often, the pressure to randomise correctly might feel different than if you're Red.

Lalam: Right, and the page closes with three promises: replicate the interpretable-versus-black-box comparison, enrich the feature analysis, and extend the behavioural model with an ML-guided term. That last bit is where the paper gets original.

Jane: So the stage is set. The next page is where they lay out the machinery — the different model families and how they're all forced to do the same prediction task.

Page 2: Tom: Page two gets technical, but the structure is clean. Every model has to solve the same task: given the history up to round t minus one, predict the card at round t, as probabilities over the four cards.

Jane: And they all produce those probabilities the same way — through a multinomial logit, so the probability of a card is proportional to the exponential of some scoring function. The models differ only in how they summarise the history.

Lu: The simplest baseline is the empirical constant model — the i.i.d. model. It just uses the observed card frequencies and ignores history entirely. That's the benchmark every other model has to beat.

Meng: Then they add serial correlation — the probability of a card shifts if the player chose that same card in one of the previous n rounds. They test orders one and four, and order four predicts best, along with a restricted version where number cards share coefficients.

Lalam: And then comes Experience-Weighted Attraction — EWA — the classic Camerer and Ho model from 1999. Each card has an attraction that updates after every round, blending what you actually got with what you would have got had you played differently.

Tom: The update rule has parameters controlling how much past experience is discounted, how much past attractions decay, and how weight falls on foregone payoffs. Then a sensitivity parameter maps the attractions to choice probabilities.

Jane: What I like is that EWA nests both reinforcement learning and belief learning as special cases. So it's a flexible umbrella, and yet the paper will show it recovers very little of the LSTM's predictive power.

Lu: That's a genuinely important result on its own — the standard learning model, which economists have relied on for decades, mostly misses whatever is driving these sequence patterns. The structure is somewhere else.

Meng: And the estimation is maximum likelihood with constraints to keep the parameters in sensible ranges — rho and delta between zero and one, lambda non-negative. All fairly standard econometrics so far.

Lalam: Right, and that contrast sharpens on the next page, when the machine learning models show up — LASSO, decision trees, DNNs, LSTMs — followed by the modified EWA family that tries to bridge the two worlds.

Tom: Exactly. Let's look at page three and see how that bridge is built.

Page 3: Tom: Page three introduces four machine learning models. Two are interpretable — a LASSO multinomial logit and a decision tree — and two are neural benchmarks, a DNN and an LSTM.

Jane: The LASSO and the tree share the same enriched feature set: lagged actions, streaks, joint action profiles, win-loss histories, recent card counts, best-response indicators. That's the raw material for the interpretable analysis.

Lu: The DNN and LSTM replicate what Hirasawa and colleagues did — the DNN looks at a fixed window of the last four actions, while the LSTM reads the whole action sequence recurrently. They're deliberately high-capacity, not meant to explain anything, just to measure how much structure is extractable.

Meng: And then there's the modified EWA family, which is where the paper's own contribution comes in. ME1 adds own repeat-and-avoid terms — whether the player just played a card, or seems to be avoiding it.

Lalam: ME2 adds an opponent term, tracking whether the opponent repeatedly played or avoided the cards that the focal card beats. That's the payoff-relevant opponent history, and it follows Hirasawa's specification.

Tom: And ME3 is the new piece — it translates the frequency-tracking features the LASSO selected into a behavioural term. It essentially checks whether the player best-responds to the opponent's modal card over the last three, four, or five rounds.

Jane: So the logic is: let the machine learning point you at candidate mechanisms, then build a small, readable model that embodies that mechanism, and test whether it helps out of sample. That's the ML-guided approach in action.

Lu: One detail I found interesting — the starred variants extend the memory length T, and out of sample the performance forms a flat plateau, with optima around ten rounds for Red and fourteen for Black. So memory length is only weakly identified — the model doesn't care much exactly how far back you look.

Meng: And they mention the fitting uses a JAX-based optimiser with multiple random starts, because some of these high-dimensional variants are otherwise prohibitively hard to fit. That's a practical point worth remembering.

Lalam: So the full toolkit is assembled — classics, behavioural models, machine learning, and this hybrid family. The next page is where everything gets compared head to head.

Tom: Let's get to the numbers, then. Page four has the results table.

Page 4: Tom: Page four opens with the evaluation metrics, and they're worth spelling out because they measure different things. KL divergence is just the negative log probability assigned to the realised action, so it matches the objective the models were trained on.

Jane: The strategic error rate is more interesting — it's the win rate of the focal player if the opponent best-responded to the model's prediction. Lower means the model's prediction is more exploitable strategically.

Lu: And relative completeness is the headline metric — the fraction of the LSTM's improvement over the constant baseline that a given model recovers. Zero is the baseline, one is the LSTM.

Meng: So what do the results show? First, Nash — the equilibrium model — actually does worse than the empirical constant model. Playing the theoretical mix is worse than just using the observed frequencies.

Lalam: And standard EWA recovers very little of the LSTM gap, which confirms what we suspected from page two — the classic learning model is the wrong tool here. Serial correlation at order four does better, but still captures only a limited share.

Tom: Then the machine learning models: LASSO is the surprise star — it reaches relative completeness of 0 point 769 for Red and 0 point 869 for Black. A sparse, readable set of history variables recovers most of the black-box signal.

Jane: The decision tree, by contrast, performs only modestly. That's a nice negative result — a small set of if-then rules isn't enough to capture the behaviour. The structure is richer than a handful of rules.

Lu: And the LSTM remains the strongest model overall, beating everything by a wide margin. So there is still something the interpretable models don't fully reach — but the gap is much smaller than you'd expect.

Meng: Then the modified EWA results: ME1 jumps to 0 point 769 for Red and 0 point 802 for Black — the own repeat-and-avoid terms are doing the heavy lifting. ME2 star closes most of the remaining gap, reaching 0 point 888 and 0 point 980.

Lalam: Though we should note the paper's caution there — ME2 star also gets longer memory lengths, so part of the gain could come from memory rather than the opponent-tracking mechanism itself. And ME3 star barely improves on it: 0 point 895 for Red, unchanged for Black.

Tom: So the ranking is clear — own history dominates, opponent payoff-relevant history adds something, frequency tracking adds almost nothing. And the strategic error rates fall steadily along the same order.

Jane: Which sets up page five, because there they go beyond the model comparisons and look directly at which features the LASSO chose, to check the behavioural ranking from a completely different angle.

Page 5: Tom: Page five has that model-agnostic check I just mentioned — Table 2, where the LASSO-selected features are grouped into families with behavioural interpretations. And the ranking from the model comparisons holds up.

Jane: The own repeat-and-avoid features are selected most consistently — variables like "the player did not play card a" come in positive, and just-played indicators come in negative. Long streaks for number cards and for K enter with different signs.

Lu: That difference between number cards and K matters, because K has a different equilibrium probability — 0 point 4 versus 0 point 2. Players seem to treat the face card as a different kind of object, and the model supports estimating them separately.

Meng: The secondary channel is opponent history — variables for whether the opponent repeatedly played or avoided the cards that the focal card beats. That matches the ME2 improvement, and it's payoff-relevant, not just generic opponent tracking.

Lalam: And the weak channel is frequency tracking — best-response-to-opponent-mode indicators do get selected, but with small coefficients. So the LASSO independently points to the same conclusion: modal tracking exists, but it's not the main driver.

Tom: I also notice outcome dependence being listed — recent wins and losses get selected too. But the paper is careful to say that doesn't imply loss overweighting; outcomes matter, but the evidence doesn't support a specific behavioural bias like that.

Jane: And because the same pattern appears for both Red and Black players, the ranking is unlikely to be an artifact of one role. That's a nice robustness check — the structure is symmetric even though the equilibrium isn't.

Lu: Then there's the strategic error rate story. SER falls steadily from the constant model through ME1 to ME2 star and ME3 star, approaching the LSTM. So the predictive gains are strategically meaningful — a best-responding opponent would actually make fewer errors against these predictions.

Meng: That connects the prediction task back to the real-world worry — if you can predict a human's randomisation, you can exploit it. The interpretable models aren't just academically interesting; they'd be practically dangerous as opponents.

Lalam: And that's why the conclusion on page six has to grapple with what all this says about the black-box question and about human behaviour more generally.

Tom: Let's close it out, then — the conclusion is where they confess the limits as well as celebrate the wins.

Conclusion: Tom: So the conclusion — the answer to the question we started with is largely yes. The interpretable models do recover most of the black-box predictive power: about 89 percent of the LSTM's KL improvement for Red players, and 98 percent for Black players.

Jane: But they're refreshingly honest about a lingering doubt — whether the improved models actually decoded what was inside the black box, or captured different regularities that happen to perform similarly. That's a genuinely open question.

Lu: On the behaviour side, the main signal is repeat-and-avoid. Players manage their own recent action sequences, and that's where the largest chunk of predictability comes from. EWA on its own explains very little.

Meng: The opponent's payoff-relevant history adds a bit more, and modal frequency tracking adds almost nothing out of sample. So ME3's value is mostly diagnostic — it rules out a plausible mechanism and thereby strengthens the repeat-and-avoid interpretation.

Lalam: And the LASSO independently corroborates that ranking, which is the methodologically satisfying part — two very different tools point at the same behavioural structure.

Tom: The broader implication is a bit humbling for human strategic skill. We think of unpredictability as a strategic ability, but the paper suggests the failures are structured and exploitable in a simple way — people leave traces of their own actions while trying to hide them.

Jane: For applications — poker, security, any adversarial setting — that means the exploitable signal might not require a massive neural network to find. A readable model can get you most of the way there.

Lu: And that dovetails with the interpretability agenda — Rudin's argument that for high-stakes decisions you want models you can inspect, not black boxes you can only trust.

Meng: I'd love to see this tested in other games, with other equilibrium structures, to see whether the repeat-and-avoid dominance is specific to this card game or something more general about how humans randomise.

Lalam: That's the natural next step. For now, the paper gives us a clear answer to its own question, an honest caveat about what remains unknown, and a neat template for how machine learning can guide behavioural theory instead of just outperforming it.

Tom: And with that, we'll say goodbye to this one — plenty to chew on. Next up, we've got another paper waiting, so let's take a short break and come back fresh.

Jane: Sounds good — see you in a moment.

Ngoc Linh Dao

Alpen-Adria-University of Klagenfurt

cs.AI

Submitted: 2026-08-07

Updated: 2026-08-10

Comments: 6 pages, 2 figures

License: http://creativecommons.org/licenses/by-nc-sa/4.0/

Importance score: 48/100

The gist: Beyond the Black Box: Interpretable Models of Human Randomisation Failures by Ngoc Linh Dao (Alpen-Adria-University of Klagenfurt, Austria; arXiv:2608.07220v1 [cs.AI], 7 Aug 2026) investigates

Key concepts

Relative completeness
A metric measuring how much of the LSTM's predictive improvement over a constant baseline a given model recovers. A value of 0 means no improvement, 1 means matching the LSTM. It helps compare interpretable models against black-box benchmarks.
Experience-Weighted Attraction (EWA)
A learning model where each action has an attraction that updates after each round, blending actual and foregone payoffs. It nests reinforcement and belief learning. In the paper, standard EWA performs poorly, but modified versions with repeat-and-avoid terms capture much of the LSTM's predictive power.
Strategic error rate
The win rate of a player if their opponent best-responded to the model's prediction. Lower values indicate the model's predictions are more exploitable strategically. It links prediction accuracy to real-world vulnerability in adversarial settings.

Terminology

Summary

Beyond the Black Box: Interpretable Models of Human Randomisation Failures by Ngoc Linh Dao (Alpen-Adria-University of Klagenfurt, Austria; arXiv:2608.07220v1 [cs.AI], 7 Aug 2026) investigates whether the predictive power of black-box sequence models for human randomisation behaviour can be recovered by transparent, interpretable models that also reveal the underlying behavioural structure.

The paper is motivated by the fact that "Mixed-strategy equilibrium predicts i.i.d play: past actions should not help predict future decisions. Human players, however, systematically depart from this benchmark, and in O’Neill’s zero-sum card game, these departures can be predicted by black-box sequence models such as LSTMs. The central question is whether that predictive power can be achieved by transparent alternatives that also reveal the behavioural structure behind it."

The paper studies O'Neill's four-card game, in which a Red player and a Black player simultaneously choose one of four cards, C = 1, 2, 3, K. The game is zero-sum with a sharp mixed-strategy benchmark: "in equilibrium, each player chooses each number card with probability 0.2 and the face card K with probability 0.4, independently across rounds. Thus, equilibrium play is serially independent: any history-dependent predictability is therefore a departure from the equilibrium benchmark. Because the equilibrium is asymmetric, with Red winning with probability 0.4 and Black with probability 0.6, all models are estimated separately by role."

Prior evidence shows a tension: O'Neill (1987) found aggregate play close to minimax, but Brown and Rosenthal (1990) showed that individual choices remain history-dependent: players’ own past actions and their opponents’ recent histories help predict future choices. More recently, Hirasawa et al. (2025) show in repeated O’Neill play that flexible sequence models predict human deviations well. The open question the paper addresses is what this predictability is made of: Is it own-action habits, opponent tracking, payoff learning, or frequency tracking? Following Rudin (2019) — the way forward is to design models that are inherently interpretable — the paper uses 84,060 decisions from 2,802 pairs over 30 rounds to construct both neural benchmarks and interpretable alternatives in a one-step-ahead prediction task: given history through round t − 1, predict a player’s card at round t.

All models share the same prediction task: "for each player role i, predict the next card ati from the history ht available before round t. Each produces probabilities over the four cards through a multinomial logit Pia (t) ∝ exp βi,a ⊤ xi (ht), and the models differ only in how they summarise ht: as constants, lagged actions, payoff-based attractions, sparse history features, or learned sequence representations."

Econometric baselines. The empirical-constant (i.i.d.) model uses only constants, so its predicted probabilities are the empirical card frequencies and do not vary with history. Serial-correlation models are also estimated, in which the probability of card a shifts if the player chose a in one of the previous n rounds, with restricted (shared number-card coefficients) and unrestricted versions at orders 1 and 4 (order 4 predicts best).

EWA. The Experience-Weighted Attraction model (Camerer and Ho, 1999) updates attractions after each round with parameters for experience discounting (ρi), attraction discounting (ϕi), and foregone payoff weighting (δi), followed by a softmax choice rule. Parameters are estimated by maximum likelihood (L-BFGS-B, imposing ρi, δi ∈ [0, 1], λi ≥ 0); the model nests reinforcement and belief learning as special cases.

Machine Learning models. Two interpretable ML models and two neural benchmarks are estimated. "The LASSO multinomial logit and decision tree use the same enriched history-based feature set, including lagged actions, streaks, joint action profiles, win-loss histories, recent card counts, and best-response indicators. The DNN and LSTM replicate Hirasawa et al. (2025): the DNN uses a fixed 4-recent-history encoding, while the LSTM processes the action sequence recurrently. Both output a softmax over the four cards, trained with Adam and hyperparameters selected within the training data. They serve not as behavioural explanations but as high-capacity benchmarks for how much history-dependent structure is extractable."

Modified EWA family. ME1 and ME2 follow Hirasawa et al. (2025), who specified own- and opponent-repeat/avoid terms from behavioural theory, prior to and independently of the feature analysis here. ME3 is the paper's extension: it translates the frequency-tracking features selected by the LASSO into a parsimonious behavioural term, and is therefore ML-guided by construction. All share the choice rule Pia (t) ∝ exp λi Aai (t − 1) + Mia (t). ME1 adds own repeat/avoid terms; ME2 adds an opponent term over Wia (opponent cards that a beats); and ME3 adds a frequency-tracking term measuring whether a best-responds to the opponent’s modal card over the last w rounds, using short windows w ∈ 3, 4, 5. The starred variants (ME2∗, ME3∗) extend memory length T, selected by cross-validation: out-of-sample KL forms a flat plateau (Fig. 2) with optima at T =10 (red) and T =14 (black), so T is only weakly identified. High-dimensional and extended-memory variants use a JAX-based optimiser with multiple random starts.

Three metrics are used:

  • KL divergence: For a predicted distribution P over the four cards and realised action a′, the loss is − log P (a′). Summed over observations, this is the negative log-likelihood.

  • Strategic error rate (SER): SER can be understood as the win rate of the focal player if an opponent best-responded to the model’s predicted strategy. Lower SER means the prediction is more strategically exploitable.

  • Relative completeness (RC): "RC measures the fraction of the LSTM’s KL improvement over the empirical-constant baseline recovered by a model: RCm = (KLm − KLconst) / (KLLSTM − KLconst). Thus, RC = 0 corresponds to the constant baseline and RC = 1 to the LSTM benchmark."

All out-of-sample results use five-fold cross-validation partitioned by pair.

Key findings from Table 1: "Nash performs worse than the empirical-constant model for both roles, while standard EWA recovers little of the LSTM gap for either role. Simple serial-correlation models improve on this, especially at order t=4, but still explain only a limited share of the LSTM gap. Thus, the main regularity is not simply equilibrium deviation, payoff learning, or generic short-run dependence."

The ML benchmarks show "richer history features matter. LASSO performs strongly, reaching relative completeness of 0.769 for Red and 0.869 for Black. Since LASSO selects a sparse set of predictors, this suggests that much of the predictive signal can be recovered from interpretable history variables. By contrast, the decision tree performs only modestly, indicating that a small number of if–then rules is not enough. The LSTM remains the strongest benchmark, outperforming every traditional model by a wide margin."

The modified EWA models sharpen the interpretation: "ME1, which adds own repeat/avoid terms, produces the first major improvement over standard EWA, reaching relative completeness of 0.769 for Red and 0.802 for Black. This suggests that players’ own action histories are highly predictive: subjects appear to manage their sequences by avoiding or returning to recently used cards. ME2∗ closes most of the remaining gap, reaching relative completeness of 0.888 for Red and 0.980 for Black. The authors caution that this improvement should be interpreted cautiously because ME2∗ adds payoff-relevant opponent repeat/avoid terms and also uses longer selected memory lengths. The safer conclusion is that extended repeat/avoid structure captures most of the predictable pattern in play."

For ME3, the answer is mostly negative: "Despite performing well with in-sample data, ME3∗ barely improves on ME2∗ for Red players and is unchanged for Black players after rounding (Table 1). Thus, modal-frequency tracking is plausible, but it is not the main source of predictability."

The LASSO-based feature analysis (Table 2) confirms the ranking model-agnostically. Own repeat/avoid is the Strongest channel: players manage their own action sequences. Opponent payoff-relevant history is a Secondary channel: players react to payoff-relevant opponent patterns. Frequency tracking is a Weak channel: modal-frequency tracking exists but is not the main driver. Outcome dependence matters (but this does not imply loss overweighting), and number/K asymmetry supports estimating number-card and K behaviour separately. The same pattern appears for both roles, so the behavioural ranking is unlikely to be a role-specific artifact.

The SER results show the predictive gains are strategically meaningful: For both roles, SER falls steadily from the constant model through ME1 to ME2∗ /ME3∗, approaching the LSTM benchmark.

The paper concludes: "ME3∗ recovers about 89% of the LSTM benchmark’s KL improvement for Red players and 98% for Black players. However, whether the improved models actually decoded what was captured inside the black-box, or if they captured different regularities of the data and happened to perform similarly requires further testing."

Regarding human behaviour: "the main behavioural signal is repeat-and-avoid behaviour. Standard EWA explains only a limited share of predictability, while adding own action-history terms produces the largest improvement. Payoff-relevant opponent histories add further predictive power, but modal-frequency tracking contributes little out of sample. Thus, ME3 is valuable mainly as a diagnostic test: by mostly rejecting best-response-to-mode behaviour as the missing mechanism, it strengthens the repeat-and-avoid interpretation. The LASSO results also corroborate this ranking: own repeat-and-avoid features are selected most consistently, opponent-history features appear secondarily, and frequency-tracking features are weak."

The final takeaway: "the results suggest that human randomisation fails in a structured way, and what the interpretable models managed to recover is not deep strategic sophistication, but rather a simpler vulnerability: in covering their traces, players leave their own."

Improvements for AI systems

  1. Build transparent sequence predictors that recover most of a black-box model’s accuracy.

Instead of relying on LSTMs or DNNs to predict human decisions, an improved AI system can use parsimonious, interpretable models with features such as:

  • own recent action repeats/avoidance,

  • opponent payoff-relevant history,

  • short-window modal-frequency tracking,

  • sparse LASSO-selected history variables.

Such a system would achieve 89–98% of the LSTM’s predictive improvement over a constant baseline, while exposing exactly which behavioural regularities drive predictions.

  1. Add repeat-and-avoid mechanisms as first-class features in human-behaviour models.

The key behavioural finding is that people systematically avoid recently chosen actions and return to older ones. An improved AI system should explicitly encode:

  • whether the agent chose the same card in the previous 1–4 rounds,

  • how long ago each action last occurred,

  • streaks of alternating or repeating choices.

This gives a large immediate gain over standard reinforcement/EWA models that only track payoffs.

  1. Include payoff-relevant opponent history separately from overall opponent history.

The improved system should condition on what the opponent did and which of those actions the focal player would beat. For example, in a four-card game, track opponent plays that are favourable vs. unfavourable to the player, rather than raw opponent frequencies. This recovers almost the entire LSTM gap, especially for Black players.

  1. Use LASSO-based feature selection to guide compact, interpretable model design.

Rather than hand-picking features arbitrarily, an improved AI system can:

  • start from a rich set of history-derived features,

  • apply sparse regularisation to identify the strongest channels,

  • translate the selected features into a small number of behavioural terms with meaningful parameters.

This yields models that are both accurate and human-auditable, avoiding the black-box trade-off.

  1. Avoid over-trusting modal-frequency tracking as a mechanism.

An improved system should treat best-response-to-mode behaviour as a weak or diagnostic channel only. Adding frequency-tracking terms may fit training data well but provides little out-of-sample improvement. The system should therefore deprioritise such features unless cross-validation shows a genuine gain.

  1. Use strategic error rate (SER) as a safety/exploitability metric alongside prediction accuracy.

An improved AI system should not just minimize log-loss; it should also measure how exploitable its predicted strategy would be if an opponent best-responded. The paper shows that interpretable repeat/avoid models reduce SER substantially, approaching the LSTM. This is useful for:

  • detecting when AI predictions reveal exploitable human habits,

  • building AI opponents that adapt to human randomisation weaknesses,

  • evaluating whether interpretable models are strategically safe to deploy.

  1. Estimate separate models per role or subpopulation, because behaviour differs by incentives.

The improved system should fit models separately for each player role rather than pooling. The paper shows role-specific models perform differently, and pooling would hide the structure of repeat-avoid behaviour. This is directly applicable to any AI system that personalises predictions from human decisions in asymmetric games.

  1. Use cross-validation by individual/pair to avoid overfitting to population averages.

The improved system should train on some people and test on held-out people, not just held-out rounds. The paper’s five-fold, pair-partitioned evaluation is a template for building robust human-behaviour models that generalise to new users.

  1. Build interpretable sequence models with explicit memory-length selection.

Instead of fixed short memory, the improved system should search over memory length T via out-of-sample KL and choose the optimal context window (e.g., 10 rounds for Red, 14 for Black). This prevents both under- and over-fitting while keeping the model transparent.

  1. Diagnose what a black-box model actually learned by constructing theory-driven alternatives.

An improved AI system can use a “modified EWA” family as a diagnostic tool: start from a baseline, add one interpretable behavioural term at a time, and measure relative completeness against the black-box benchmark. This tells system developers whether a neural network’s predictive power comes from genuine strategic sophistication or from simple, exploitable human habits — usually the latter.

Abstract

Mixed strategy equilibrium predicts i.i.d play: past actions should not help predict future decisions. Human players, however, systematically depart from this benchmark, and in O'Neill's zero sum card game, these departures can be predicted by black box sequence models such as LSTMs. This paper asks whether that predictive power can be achieved by transparent alternatives that also reveal the behavioural structure behind it. Using 84,060 decisions from 2,802 pairs, the analysis first benchmarks naive and behavioral models against interpretable machine learning and deep learning models, then evaluates the modified EWA specifications of prior work against these benchmarks and uses the LASSO diagnostics to motivate a further nested frequency tracking extension. The results show that repeat or avoid behavior, especially players' management of their own recent action histories, accounts for most of the interpretable and strategically exploitable signal, while frequency tracking adds little out of sample.

Related papers