Causal Falsification of Digital Twins

arXiv:2301.07210 · stat.ME, cs.CE, cs.LG, stat.AP · Submitted 2026-08-08 · Read on arXiv

Listen

Radio episode about this paper

Transcript

Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.

Tom: Next we'll be talking about the paper "Causal Falsification of Digital Twins".

Jane: The paper was written by Rob Cornish, Muhammad Faaiz Taufiq, Arnaud Doucet and Chris Holmes from Nanyang Technological University and University of Oxford.

Tom: Stay tuned as we take you through the paper and discuss its implications.

Title and Authors: Tom: Welcome back, everyone! Tom here, and I’ve got Jane with me, and we are absolutely buzzing about today’s paper. It’s called “Causal Falsification of Digital Twins,” and it comes out of Oxford and NTU Singapore.

Jane: And Tom, I have to say, just the title alone got me excited. “Falsification” is such a bold word. It’s not trying to prove a digital twin is right — it’s trying to catch it being wrong.

Tom: Exactly! And the authors are Rob Cornish, Muhammad Faaiz Taufiq, Arnaud Doucet, and Chris Holmes. These are heavy hitters in statistics and machine learning. Doucet and Holmes are legends in the field.

Jane: For sure. And the idea here is really important. A digital twin is basically a simulation of a real-world process — like a patient in an ICU, or a car on the road. You want to use it to predict what happens if you intervene, like giving a drug or slamming on the brakes.

Tom: Right, and the problem is, how do you know the twin is actually good? You can’t just compare it to past data, because past data was collected under one set of conditions, and the twin is supposed to predict under different conditions. That’s the causal gap.

Jane: And that’s where the word “falsification” comes in. Instead of trying to prove the twin is correct — which they show is basically impossible with observational data alone — they flip it. They try to find specific situations where the twin is definitely wrong.

Tom: It’s like a detective trying to prove someone is guilty rather than proving innocence. You gather evidence of failure. And if you can’t find any, that doesn’t mean the twin is perfect, but it means you haven’t caught it lying yet.

Jane: And that’s a much more honest and robust approach. The paper even shows that certifying a twin is interventionally correct requires assumptions like no unmeasured confounding, which almost never holds in real-world data. So they say, fine, let’s not assume that. Let’s just find the failures.

Tom: And the beauty is, their method works even with confounded data. That’s the big deal here. They don’t need to model the real-world process or know the twin’s internals. They just need an i.i.d. dataset of observational trajectories.

Jane: Which makes it incredibly general. You could apply this to medical simulators, engineering models, even climate models. Anywhere you have a digital twin and some real-world data.

Tom: And they actually tested it on a real sepsis model using the Pulse Physiology Engine and the MIMIC-III dataset. That’s real ICU patient data. So this isn’t just theory — they put it to work.

Jane: I love that. And the results were striking. They found the twin consistently underestimated things like chloride and skin temperature, and overestimated sodium and glucose. That kind of granular feedback is gold for developers.

Tom: So the title really captures it. “Causal Falsification” — you’re using causal reasoning to falsify, not to certify. And that shift in mindset is what makes this paper so powerful.

Jane: And it sets up the whole rest of the paper. Next we’re going to dig into the core problem they’re solving — why certification is fundamentally unsound without strong assumptions.

Tom: Stay with us, because this gets even better.

Summary and Core Problem: Jane: Welcome back. We’re still on “Causal Falsification of Digital Twins,” and Tom, I want to get into the meat of the problem they’re tackling.

Tom: Absolutely. So the paper starts with a pretty sobering result. They show that if you only have observational data — meaning you didn’t run a randomized trial — you cannot uniquely identify what would happen under a specific intervention. That’s the fundamental problem of causal inference.

Jane: And they prove it in the paper. If the actions you observed weren’t the exact actions you care about, then the distribution of the potential outcomes is not identified. So you literally cannot know the true effect from the data alone.

Tom: And that kills any hope of certification. If you can’t identify the true interventional distribution, you can’t check whether the twin matches it. So any procedure that claims to certify a twin from observational data is either making hidden assumptions or it’s just wrong.

Jane: And the common assumption people make is “no unmeasured confounding.” That means the actions taken were based only on what you recorded, not on some hidden factor. But the paper points out that this rarely holds in practice. In medicine, for example, doctors make decisions based on things they see and feel that aren’t in the dataset.

Tom: Right. And they even give a toy example in the appendix — cars with old brake pads. The drivers know the pads are old, so they avoid aggressive braking. The data looks like aggressive braking works fine, but that’s only because it was never tried on cars with old pads. A twin trained on that data would be dangerously wrong.

Jane: So instead of trying to certify, they propose falsification. You look for hypotheses that must be true if the twin is correct. If you can show one of those is false, then the twin is definitely wrong. And crucially, you can do that without the unconfoundedness assumption.

Tom: And that’s the key move. They construct these hypotheses using causal bounds. The classical result is from Manski — he showed you can bound the average treatment effect even with confounding, using worst-case values for the unobserved outcomes.

Jane: But Manski’s bounds have a limitation. They only work when you condition on the initial state, not on intermediate observations. And for a digital twin, you really want to check it at different points along the trajectory, not just at the start.

Tom: Exactly. And that’s where this paper makes a real contribution. They derive new bounds that work with intermediate conditioning. They call them “identifiable conditional bounds.” And they show these bounds are tight — you can’t improve them without extra assumptions.

Jane: And the practical payoff is huge. With these bounds, you can test hypotheses like “the twin’s average output is at least this value” or “at most that value.” If the twin violates those, you’ve falsified it.

Tom: And they prove their testing procedure has exact type I error control using Hoeffding’s inequality. No asymptotic approximations, no hand-waving. You get a valid p-value with finite samples.

Jane: That’s really rigorous. And it means the method is reliable even with small datasets, which is often the case in medicine and engineering.

Tom: So the summary is: certification is impossible in general, falsification is possible, and they give you the tools to do it properly.

Jane: And next we’re going to talk about the actual improvements they make over the classical bounds — that’s where the technical innovation really shines.

Tom: Let’s get into it.

Improvements and Methodology: Jane: Back with “Causal Falsification of Digital Twins.” Tom, I want to dig into the technical improvements, because that’s where the paper really flexes.

Tom: Definitely. So the classical Manski bounds — they’re elegant, but they’re also pretty loose. They give you a range for the average outcome, but that range can be huge, especially for longer action sequences. The reason is they only use the initial state to condition on.

Jane: And the paper’s improvement is to allow conditioning on intermediate observations. That’s a big deal because it lets you zoom in on specific subpopulations or specific points in time. And they show that this can make the bounds much tighter.

Tom: Right. The tightness is governed by something called the propensity score — the probability that the observed actions match the intervention you care about, given the conditioning event. If that probability is high, the bounds are narrow. And conditioning on intermediate states can boost that probability a lot.

Jane: And they prove this formally. Proposition seven in the paper shows that their bounds are strictly tighter than Manski’s whenever the conditional propensity score is higher. That’s a clean, testable condition.

Tom: And they also prove sharpness. Proposition eight shows you can’t do better without additional assumptions. There’s always a set of potential outcomes consistent with the data that hits the worst-case bound. So their bounds are optimal given the information available.

Jane: That’s really important for credibility. If someone comes along and says “I have a tighter bound,” the paper says “show me your assumptions, because without them, it’s impossible.”

Tom: And there’s a really interesting negative result too. They show that if you try to condition on the exact value of a continuous intermediate variable, you get trivial bounds. You can’t get any information. That’s Theorem nine and it’s a bit surprising.

Jane: It is surprising. You’d think more granular conditioning would help, but it actually breaks down. The paper explains why — with continuous variables, the probability of observing the exact value is zero, so the propensity score collapses and the bounds become useless.

Tom: So they wisely stick to conditioning on events, like “the value falls in this range,” rather than exact values. And that’s enough to get useful, tight bounds in practice.

Jane: And then they build a full testing procedure on top of these bounds. They define hypotheses about the twin’s expected output, and they test them using one-sided confidence intervals. They use Hoeffding’s inequality for exact finite-sample guarantees.

Tom: And they also offer a bootstrap alternative for tighter intervals, though that’s approximate. The Hoeffding approach is exact, which is what they use in the main results.

Jane: And the whole thing is designed to be practical. They use sample splitting to choose hypothesis parameters, so you can search for failure modes without biasing your tests.

Tom: That’s a really thoughtful design. It’s not just theory — it’s a toolkit you can actually deploy.

Jane: And next we’re going to see it deployed. The case study with the Pulse Physiology Engine and MIMIC-III is where the rubber meets the road.

Tom: Can’t wait to talk about that.

First Page and Case Study: Jane: So we’re still on “Causal Falsification of Digital Twins,” and now we get to the case study. Tom, this is where they actually run the method on a real digital twin.

Tom: And it’s a serious one. They use the Pulse Physiology Engine, which is an open-source human physiology simulator. It models things like heart rate, blood pressure, and lab values for patients with conditions like sepsis.

Jane: And they test it against MIMIC-III, which is a massive dataset of real ICU patients. They focused on sepsis patients, following the sepsis-three criteria, and ended up with over eleven thousand trajectories.

Tom: That’s a lot of data. And they used a held-out five percent to choose the hypotheses, then tested on the remaining ninety-five percent. That’s the sample splitting approach we talked about.

Jane: And they defined a bunch of hypotheses — one thousand four hundred forty-two in total — covering fourteen different physiological quantities. Each hypothesis was about whether the twin’s average output was above or below a certain bound.

Tom: And the results were pretty damning for Pulse. They rejected hypotheses for ten different quantities. Chloride, sodium, potassium, skin temperature, calcium, glucose, and more. The twin was consistently off.

Jane: And the direction of the errors was consistent too. Pulse underestimated chloride and skin temperature, and overestimated sodium and glucose. That’s really actionable information for the developers.

Tom: And they compared their bounds to Manski’s original bounds. With Manski’s, they got almost no rejections — just a handful. With their new bounds, they got dozens. So the improvement isn’t just theoretical; it’s the difference between finding failures and missing them entirely.

Jane: That’s a really compelling demonstration. And they also showed why naive assessment — just comparing the twin’s output to the observed data — can be misleading. In one example, the naive comparison looked fine, but the causal bounds showed the twin was actually falsified.

Tom: Right. Because the observed data is confounded. The twin might match the observed distribution by accident, but that doesn’t mean it’s right for the intervention you care about. The causal bounds account for that.

Jane: And they even show histograms where the raw data looks almost identical between the twin and the real patients, but the causal test still rejects. That’s a powerful visual proof that you need causal reasoning, not just eyeballing distributions.

Tom: And the whole thing runs on real hardware — they generated over twenty-six thousand simulated trajectories from Pulse. So this is a practical, scalable method.

Jane: And the implications are huge. For safety-critical applications like medicine, having a rigorous way to catch twin failures before deployment could save lives.

Tom: Absolutely. And next we’re going to wrap up with the big picture — what this means for the field and where it goes from here.

Jane: Let’s do it.

Conclusion: Tom: And we’re back for the final stretch on “Causal Falsification of Digital Twins.” Jane, let’s pull it all together.

Jane: So the big takeaway is that you can’t certify a digital twin from observational data alone. The paper proves that. But you can falsify it, and they give you a rigorous, general-purpose way to do that.

Tom: And the method is sound even with unmeasured confounding. That’s the key. It doesn’t require heroic assumptions. It just needs an i.i.d. dataset and the ability to run the twin.

Jane: And the bounds they derive are both tight and practical. They improve on Manski’s classical result by allowing intermediate conditioning, and they show that improvement matters in practice — it’s the difference between catching failures and missing them.

Tom: And the case study with Pulse and MIMIC-III is a real-world proof of concept. They found consistent, actionable failures in a physiology simulator used in medical training and research. That’s not a toy example.

Jane: And the implications go beyond medicine. Anywhere you have a digital twin — aviation, manufacturing, civil engineering, agriculture — this method gives you a way to check it without pretending the data is cleaner than it is.

Tom: And there are limitations, of course. The method assumes no distribution shift between testing and deployment. And the choice of hypothesis parameters is somewhat ad hoc. But those are avenues for future work, not fatal flaws.

Jane: And the paper even suggests extensions — using sensitivity analysis to get tighter bounds if you’re willing to make assumptions, or using machine learning to automate the search for failure modes.

Tom: So overall, this is a paper that shifts the conversation. Instead of asking “is this twin correct?” it asks “can I prove it’s wrong?” And that’s a much more honest and useful question.

Jane: And it gives you the tools to answer it. That’s what makes this paper so impactful.

Tom: Alright, that’s a wrap on “Causal Falsification of Digital Twins.” Great paper, great authors, great implications. Thanks for joining us, and we’ll see you next time.

Jane: Bye, everyone!

Rob Cornish, Muhammad Faaiz Taufiq, Arnaud Doucet, Chris Holmes

Nanyang Technological University · University of Oxford

stat.ME, cs.CE, cs.LG, stat.AP

Submitted: 2026-08-08

Comments: Accepted for publication in the Journal of Machine Learning Research (JMLR)

Code: https://github.com/faaizT/CausalTwinAssessment

License: http://creativecommons.org/licenses/by/4.0/

Importance score: 85/100

Key concepts

Digital Twin
A simulation of a real-world process, such as an ICU patient or a car on the road. It is used to predict outcomes when interventions are applied, like giving a drug or braking.
Causal Falsification
Instead of trying to prove a digital twin is correct, this approach tries to find specific situations where it is definitely wrong. This method uses causal reasoning and works even with observational data without needing assumptions about unmeasured confounding.
Manski's Bounds
These are classical bounds used in causal inference to estimate the average treatment effect. The paper improves upon these bounds by allowing conditioning on intermediate observations, resulting in tighter bounds for digital twin testing.
Propensity Score
This is the probability that observed actions match the intervention being studied, given a specific conditioning event. A higher propensity score indicates that intermediate observations provide more information, leading to tighter causal bounds.

Terminology

Summary

Summary

This paper addresses the problem of assessing the accuracy of digital twins—simulation-based models designed to predict how a real-world process will evolve in response to interventions. The authors formulate this assessment as a causal inference problem, leading to a precise definition of what it means for a twin to be “correct.” They define a twin as interventionally correct if, for almost all initial observations x 0 and all action sequences a 1:T, the distribution of the twin's output 1:T(x 0, a 1:T) equals the conditional distribution of the real-world potential outcomes X 1:T(a 1:T) given X 0 = x 0. They show this is equivalent to an unconditional formulation: the distribution of (X 0, 1:T(X 0, a 1:T)) equals the distribution of X 0:T(a 1:T) for all action sequences.

The paper establishes a fundamental limitation: using observational data alone, it is impossible to certify that a twin is interventionally correct. This follows from the fundamental problem of causal inference (Theorem 3), which states that if the probability of observing a particular action sequence a 1:T is not 1, then the distribution of the potential outcomes X 0:T(a 1:T) is not uniquely identified by the observational data. This holds even with an infinitely large dataset, and the authors provide a self-contained proof. They note that the common assumption of no unmeasured confounding (sequential randomization) would allow identification, but this assumption will rarely hold for stochastic phenomena and typical datasets, and they illustrate the pitfalls with a toy example involving car braking and brake pad age.

To avoid these assumptions, the authors propose a falsification approach. Instead of trying to verify correctness, they seek hypotheses H with the property: If the twin is interventionally correct, then H is true. If they can show H is false using data, they have identified a concrete failure mode of the twin. This approach is sound even under arbitrary unmeasured confounding.

The core theoretical contribution is a novel set of longitudinal causal bounds (Theorem 6). The classical result of Manski (1990) bounds the conditional expectation E[Y(a 1:t) X 0:t(a 1:t) in B 0:t] using worst-case values, but these bounds are not identifiable when conditioning on intermediate observations. The authors introduce a random variable N:= 0 s t A 1:s = a 1:s, which is the largest prefix of the action sequence that matches the observed actions. They prove that the unidentifiable Manski bounds can themselves be bounded by identifiable quantities:

[

E[Y lo X 0:N(A 1:N) in B 0:N] E[Y(a 1:t) X 0:t(a 1:t) in B 0:t] E[Y up X 0:N(A 1:N) in B 0:N],

]

where Y lo and Y up are the observed outcome when the action sequence matches and the worst-case bounds otherwise. This result allows conditioning on intermediate observations, which yields tighter and more informative bounds than Manski's original result. The tightness is quantified by 1 - P(A 1:t = a 1:t X 0:N(A 1:N) in B 0:N), which is related to the propensity score. The authors prove their bounds are sharp (Proposition 8) and show that continuous conditional bounds must be trivial (Theorem 9).

Based on these bounds, the authors develop a general-purpose statistical testing procedure. For each choice of parameters (t, f, a 1:t, B 0:t), they define two hypotheses:

  • H lo: Q lo

  • H up: Q up

where is the twin's conditional expectation of the outcome f, and Q lo, Q up are the identifiable bounds. The testing procedure uses i.i.d. observational data and i.i.d. twin trajectories. It constructs one-sided confidence intervals for Q lo and using either Hoeffding's inequality (exact, finite-sample) or bootstrapping (approximate). The test rejects H lo when the upper confidence interval for is below the lower confidence interval for Q lo. The procedure controls type I error exactly when using Hoeffding's inequality, and multiple testing is handled via the Holm-Bonferroni method.

The methodology is applied to a large-scale, real-world case study: assessing the Pulse Physiology Engine (an open-source human physiology simulator) using the MIMIC-III dataset of ICU patients with sepsis. The authors extract 11,677 sepsis patient trajectories, use 5% for selecting hypothesis parameters, and the rest for testing. They consider 14 physiological quantities (e.g., heart rate, blood concentrations) and 25 discrete actions (combinations of intravenous fluids and vasopressors). They generate 26,115 twin trajectories from Pulse. Using Hoeffding's inequality with Holm-Bonferroni correction, they obtain rejections for 10 physiological quantities, including Chloride, Sodium, Potassium, Skin Temperature, and Glucose. The p-value plots show that for each rejected quantity, the twin consistently either underestimates or overestimates the outcome. In contrast, using Manski's unconditional bounds yields far fewer rejections (e.g., only 1 rejection for Chloride vs. 24 with the new bounds). The authors also demonstrate that naive assessment (directly comparing twin outputs with observational data) can be misleading, showing cases where the twin appears accurate but is actually falsified by the causal bounds, and vice versa.

The paper concludes by discussing limitations, including the assumption of no distribution shift between testing and deployment, and the ad hoc nature of choosing hypothesis parameters B 0:t. It suggests future work on automated parameter selection and sensitivity analysis.

Improvements for AI systems

Based on the paper, here are the specific improvements I can make to AI systems and what the improved systems can do:

  • Implementation: Add a falsification layer that tests AI-generated predictions against causal bounds derived from real-world observational data, rather than merely checking distributional similarity.

  • What the improved AI can do: When an AI system (e.g., a digital twin, a reinforcement learning simulator, or a generative model of physical processes) produces predictions, the system can now determine specific conditions under which those predictions are provably wrong—even in the presence of unmeasured confounding—without requiring assumptions like unconfoundedness.

  • Implementation: Integrate the novel bounds from Theorem 6 into AI systems that make sequential decisions (e.g., treatment policies in healthcare, autonomous driving, robotics control).

  • What the improved AI can do: For any given action sequence, the AI can now compute identifiable lower and upper bounds on the expected outcome, conditional on intermediate observations. This allows the AI to flag when its own predicted outcome falls outside these bounds, indicating a falsification of its model—even when the underlying data is confounded.

  • Implementation: Use the exact testing procedure from Section 5.2 (based on Hoeffding’s inequality) to replace approximate or asymptotic validation methods in AI systems.

  • What the improved AI can do: The AI can now provide finite-sample guarantees (no asymptotic approximations) that its rejection of a hypothesis (e.g., the twin is correct for this patient subgroup) is valid at a specified significance level, even with small datasets and arbitrary confounding.

  • Implementation: Adopt the sample-splitting approach (Cox, 1975) used in the case study to choose hypothesis parameters (e.g., which subgroups, actions, and outcomes to test) without inflating type I error.

  • What the improved AI can do: The AI can automatically select which aspects of its model to validate (e.g., which patient subgroups, which treatment sequences, which physiological metrics) using a held-out dataset, then test those on the main dataset, ensuring that the validation results are not overfit to the data.

  • Implementation: Replace bootstrap or normal-approximation confidence intervals with exact Hoeffding-based intervals when outcomes are bounded (as in the case study).

  • What the improved AI can do: The AI can produce confidence intervals for its predictions that are guaranteed to have the stated coverage probability, without relying on large-sample assumptions. This is particularly useful in safety-critical applications where conservative but correct intervals are preferred.

  • Implementation: Implement a workflow where the AI first attempts to falsify its own model using the causal bounds, and only then proceeds to use the model for prediction or decision-making.

  • What the improved AI can do: The AI can now actively seek out its own failure modes (e.g., my model underestimates blood chloride levels for elderly sepsis patients with high heart rate after receiving vasopressors) and report these to the user, rather than passively providing predictions that may be misleading.

  1. For a Digital Twin of a Human Patient:

  • Given a sequence of treatments (e.g., IV fluids and vasopressors), the AI can now output a range of plausible outcomes (e.g., heart rate) that are consistent with the observational data, even if that data is confounded by unmeasured factors (e.g., disease severity not recorded in the dataset).

  • The AI can then compare its own simulated outcome to this range. If its output falls outside the range, the AI flags this as a falsification and reports the specific patient subgroup and treatment sequence where it fails.

  1. For a Reinforcement Learning Agent in a Medical or Robotics Setting:
  • The agent can now evaluate the safety of a proposed policy by checking whether the expected outcome under that policy lies within the causal bounds. If it does not, the agent can reject the policy as unsafe, even without knowing the true confounding structure.
  1. For a Generative Model of Physical Processes (e.g., climate, engineering):
  • The model can now be tested for causal correctness (i.e., does it correctly predict outcomes under interventions?) rather than just predictive accuracy (i.e., does it match observed data?). This is crucial for using such models in planning and decision-making.
  1. For an AI System that Learns from Electronic Health Records:
  • The system can now provide robust treatment recommendations that are valid even if the observational data contains unmeasured confounders (e.g., lifestyle factors, genetic predispositions). It does this by only recommending treatments whose expected outcomes are within the causal bounds, and by explicitly stating the uncertainty due to confounding.

Aspect Before After

Validation Compare model output to observed data Compare model output to causal bounds that are robust to confounding

Assumptions Often requires unconfoundedness Only requires i.i.d. observational data

Statistical Guarantees Asymptotic or approximate Exact finite-sample control of type I error

Granularity Global accuracy Conditional accuracy on specific subgroups and action sequences

Actionability The model is wrong The model is wrong for elderly patients with high heart rate after receiving vasopressors, because it underestimates chloride levels

These improvements make AI systems significantly more trustworthy and reliable in safety-critical applications, where the cost of an incorrect prediction or decision can be very high.

Abstract

Digital twins are simulation-based models designed to predict how a real-world process will evolve in response to interventions. This modelling paradigm holds substantial promise in many applications, but rigorous procedures for assessing their accuracy are essential for safety-critical settings. We consider how to assess the accuracy of a digital twin using real-world data. We formulate this as a causal inference problem, which leads to a precise definition of what it means for a twin to be "correct". Unfortunately, fundamental results from causal inference mean observational data cannot be used to certify a twin in this sense unless potentially tenuous assumptions are made, such as that the data are unconfounded. To avoid these assumptions, we propose instead to find situations in which the twin is not correct, and present a general-purpose statistical procedure for doing so. Our approach yields reliable and actionable information about the twin under only the assumption of an i.i.d. dataset of observational trajectories, and remains sound even if the data are confounded. We apply our methodology to a large-scale, real-world case study involving sepsis modelling within the Pulse Physiology Engine, which we assess using the MIMIC-III dataset of ICU patients.

Related papers