What If They Took the Shot? A Hierarchical Bayesian Framework for Counterfactual Expected Goals

summary

The gist

The gist: This study develops a hierarchical Bayesian framework that integrates expert domain knowledge to quantify player-specific effects in expected goals (xG) estimation, addressing a core

In short

The study developed a hierarchical Bayesian framework to compare how different players would perform given identical shooting opportunities. It uses Football Manager ratings as expert knowledge to estimate player-specific conversion probabilities, allowing for counterfactual reasoning about who would be more likely to score under specific conditions.

Key concepts

Hierarchical Bayesian Framework: This statistical method allows researchers to model player performance by treating individual skill differences as part of a larger group structure. It combines observed data with expert knowledge (priors) to produce probabilistic estimates, handling uncertainty when comparing different players on the same shot.
Counterfactual Expected Goals (xG): This concept addresses 'what if' scenarios in football analytics. Instead of just measuring what happened, it calculates the expected goals if a different player had taken the exact same shot. This helps determine which player has a higher probability of scoring in that specific context.
Context-Conditioned Counterfactual Transport (C3T): C3T is a tool used to measure 'tactical fit.' It quantifies how changing a player's finishing ability affects their expected output when they are placed within a new team's specific tactical system. It treats the player's skill distribution as being transformed by the new context.
FM-Informed Priors: These are expert opinions, specifically ratings from Football Manager, used to guide the statistical model. When data is scarce for a player, these ratings provide informed starting points (priors) for the model's calculations. This bridges formal statistics with qualitative scouting expertise.

Terminology used across episodes

Reinforcement Learning (RL) — explained in 25 episodes
Sample Complexity — explained in 19 episodes
Federated Learning — explained in 17 episodes
Retrieval-Augmented Generation (RAG) — explained in 17 episodes
Quantum Fisher Information (QFI) — explained in 16 episodes
Interpretability — explained in 15 episodes

This episode discusses

The paper

What If They Took the Shot? A Hierarchical Bayesian Framework for Counterfactual Expected Goals · Read on arXiv

Mikayil Mahmudlua, Oktay Karakus, Hasan Arkadas

Cardiff University · Dead Ball Analytics Limited

How would the probability of scoring change if a different player took the same shot? Although expected goals models describe the difficulty of scoring opportunities, comparing players on identical chances requires accounting for uncertainty in their estimated conversion probabilities. We address this question through a hierarchical Bayesian framework that evaluates alternative shooters under the same observed circumstances. Using 35,700 eligible shots from four leagues during the 2015/16 season, we compare a contextual benchmark with a primary player-intercept model and a provisional extension that allows player differences to vary with shooting conditions. Paired posterior draws preserve shared uncertainty when individual probability differences are aggregated across opportunities. Chronological validation demonstrates improved predictive performance relative to a constant-rate baseline, but introducing player effects provides negligible additional gains. Controlled single-seed experiments show that greater information per player improves estimation accuracy and narrows uncertainty intervals, although distinguishing small differences remains difficult and confident conclusions about near-null effects can become more frequent. Applications involving Giroud--Vardy and Berardi--Sansone similarly yield estimated differences that are small relative to their posterior uncertainty. These findings highlight an important distinction between predicting the outcome of a shot and establishing whether one player would be more likely to convert it. The proposed framework makes this distinction explicit by evaluating alternative shooters on fixed recorded opportunities while accounting for their contextual exposure. The resulting comparisons concern conditional conversion rather than the causal consequences of replacing a player in a match.

Transcript

Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.

Tom: Today's paper: "What If They Took the Shot? A Hierarchical Bayesian Framework for Counterfactual Expected Goals".

Jane: The gist: This study develops a hierarchical Bayesian framework that integrates expert domain knowledge to quantify player-specific effects in expected goals (xG) estimation,

Tom: First, who's behind it and why it matters.

Paper summary: Tom: So, what we're looking at here is this paper called "What If They Took the Shot? A Hierarchical Bayesian Framework for Counterfactual Expected Goals". Basically, they're taking the idea that we have to account for how different players are at scoring when they take the same shot.

Jane: Right. It addresses a problem where traditional expected goals models treat all players as if they're identical finishers, and this framework tries to fix that by using expert knowledge.

Lu: The core idea is a hierarchical Bayesian framework that integrates expert domain knowledge, specifically from Football Manager ratings, to stabilize those player-level estimates in expected goals estimation (xG) <ref:2511.23072#pg1>.

Meng: So, they're using those FM ratings as priors to guide the parameter estimation in Bayesian logistic regression? That sounds like a way to handle data sparsity for players who haven't taken many shots.

Tom: Exactly. They use nine thousand nine hundred seventy shots from StatsBomb’s two thousand fifteen–sixteen dataset and those expert ratings from Football Manager two thousand seventeen to combine Bayesian logistic regression with these FM-informed priors <ref:2511.23072#pg1>.

Jane: And what they claim is that this approach substantially reduces posterior uncertainty relative to weakly informative priors and shows strong external validity, meaning the hierarchical and baseline predictions correlate at R two ≈ zero point seven five <ref:2511.23072#pg1>.

Lalam: From my view, the main thing they contribute is that this framework allows for counterfactual reasoning. They introduce a rigorous counterfactual evaluation framework for transfer assessment, which lets clubs quantify how prospective signings would perform when substituted into their own shot contexts <ref:2511.23072#pg3>.

Tom: Yeah, so the paper claims they can quantify how much expected goals player B would generate if they took player A’s shots, compared to what actually happened with player A <ref:2511.23072#pg3>.

Jane: That's the big lift for scouting. They also validate this framework against real-world decisions, showing strong coherence between those counterfactual predictions and subsequent transfer outcomes and on-field performance <ref:2511.23072#pg3>.

Conclusion: Tom: So, looking at the title, "What If They Took the Shot? A Hierarchical Bayesian Framework for Counterfactual Expected Goals", it’s really about moving beyond just predicting an outcome to understanding player differences in conversion probability <ref:2511.23072#pg1>.

Jane: It’s about taking that uncertainty we talked about and turning it into a measurable difference between players, rather than just a fuzzy idea <ref:2511.23072#pg3>.

Lu: The authors are showing that FMinformed priors, when properly normalized and aligned with relevant shot attributes, substantially reduce posterior uncertainty and yield more stable player-specific estimates <ref:2511.23072#pg3>.

Meng: For deployment at scale, the paper suggests this is a way to make those player-specific estimates more robust even when data is sparse <ref:2511.23072#pg1>.

Tom: The implication for me as an engineer is that if you have limited shot history, you can borrow strength from expert knowledge in a structured way instead of starting from a very weak prior <ref:2511.23072#pg1>.

Jane: And it changes how we think about transfer valuation because they show how counterfactual improvements align closely with observed market behavior, like in the Vardy versus Giroud study <ref:2511.23072#pg3>.

Lalam: What this really means for us is that tactical context isn't just noise anymore; it’s an explicit conditioning variable, transforming "fit" into a posterior probability distribution instead of just a feeling <ref:2511.23072#pg3>.

More episodes

← Home