Doctor Rashomon and the UNIVERSE of Madness: Variable Importance with Unobserved Confounding and the Rashomon Effect

arXiv:2510.12734 · cs.LG · Submitted 2026-08-20 · Read on arXiv

Listen

Transcript

Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.

Tom: Next we'll be talking about the paper "Doctor Rashomon and the UNIVERSE of Madness: Variable Importance with Unobserved Confounding and the Rashomon Effect".

Jane: The paper was written by Jon Donnelly, Srikar Katta, Emanuele Borgonovo and Cynthia Rudin from.

Tom: Stay tuned as we take you through the paper and discuss its implications.

Paper discussion segment 2: Tom: In our last segment, Jane and I focused on the dramatic implications of the paper's title. Today, we’re going to dig into what the authors actually summarize about the methodology itself in "Doctor Rashomon and the UNIVERSE of Madness: Variable Importance with Unobserved Confounding and the Rashomon Effect."

Jane: To put it simply, previous methods for determining variable importance were almost entirely based on correlation—showing how tightly two things move together. But that only tells you about association, not causation. The authors argue that this is a dangerous oversimplification.

Lu: What they introduce is a much more sophisticated way to quantify how much predictive power we might be losing because of variables we simply couldn't collect data on, like economic sentiment or cultural trends. It moves the focus from simple linear relationships to deeper structural causal inference.

Meng: I found it fascinating that they don't just correct the model; they fundamentally change the output structure. Instead of a single 'importance score' for each variable, they provide a spectrum of potential importance, based on how much that variable might be influenced by unobserved factors.

Lalam: This is incredibly valuable because it forces us to distinguish between variables that are merely *correlated* with an outcome and variables that are actually *mechanistically linked* to it, even if the link is obscured by unseen systemic noise.

Tom: So, if I understand this correctly, the summary section of "Doctor Rashomon and the UNIVERSE of Madness: Variable Importance with Unobserved Confounding and the Rashomon Effect" is teaching us to be much more skeptical of any single 'A causes B' conclusion. It’s about building a network of plausible influences rather than drawing a single, confident line.

Jane: Exactly. It moves us from the fallacy of determinism—the idea that everything can be perfectly explained by our current data—to an understanding of probabilistic possibility guided by structural constraints.

Lu: This methodology provides a mathematical rigor that allows us to model not just the expected outcome, but the *range* of possible outcomes given our current informational constraints.

Meng: Which naturally leads us to consider: if the model is already admitting it might be wrong because of missing data, how do we use those insights in real-world scenarios like lending or medicine?

Paper discussion segment 3: Tom: To recap, "Doctor Rashomon and the UNIVERSE of Madness: Variable Importance with Unobserved Confounding and the Rashomon Effect" fundamentally changes how we quantify variable importance by formally integrating the risk posed by unobserved systemic factors into our core calculations. Let's dig into what that means for practical application.

Jane: Exactly. The breakthrough isn't just that we *can* account for missing variables—it’s that it forces us to become deeply honest about what we *don't* know when building models, which is a major mental shift for the industry.

Lu: Previously, if a data set was clean and looked perfect, the model would treat it as gospel truth, leading to those dangerously overconfident conclusions. The paper gives us the mathematical tools to dismantle that illusion of certainty.

Tom: To illustrate this practically: if we build an AI system meant to predict loan defaults, and we don't collect data on the borrower's neighborhood stability—a major unobserved confounder—the old model would give us a single risk score. It would be wrong, but it would *feel* correct.

Jane: But now, the output isn't just a 'result'; it’s an accompanying ‘certainty map.’ Think of it this way: instead of just saying, "This borrower has an eighty percent chance of default," the new framework says, "Based on the variables we have, the risk is high; however, because we cannot account for neighborhood instability (a major unobserved factor), our confidence interval is significantly wider."

Meng: That quantifiable admission of ignorance is key. It fundamentally shifts the goal from merely prediction to something much more valuable: verifiable trustworthiness. The model isn't just giving an answer; it’s providing a risk assessment of its own reliability.

Lalam: And this has massive implications for ethics and equity because if we know that a model's conclusions about medical diagnosis are highly dependent on unobserved socioeconomic factors, we can preemptively flag those variables as needing further study or policy intervention, rather than just accepting the biased output.

Tom: We are essentially moving from optimizing for the highest possible accuracy score in a controlled test environment to optimizing

Paper discussion segment 3: Tom: To synthesize everything we’ve discussed, this methodology fundamentally upgrades our ability to conduct sophisticated causal inference by making us mathematically accountable for the unknowns in our data collection.

Lu: This moves us beyond simply treating data gaps as unfortunate footnotes; it forces us to design entire research protocols around anticipating and modeling those gaps. We are effectively building guardrails into the beginning of the process, rather than just adding a disclaimer at the end.

Meng: From an infrastructure perspective, this means that data governance teams can no longer treat raw data ingestion as a simple plumbing job. They have to build systems that actively model potential systemic confounders alongside the variables we are measuring—it requires an entirely new layer of auditing capability for any AI tool being deployed.

Lalam: And the implication for policy is staggering. Previously, regulatory bodies might accept a high-accuracy prediction from a single source, blind to its limitations. Now, if a model flags that its conclusions about resource allocation are highly dependent on unobserved regional economic stability—a factor outside the model's scope—that immediately triggers a governance review and demands alternative policy approaches.

Jane: It elevates the requirement for evidence from mere statistical significance to genuine, verifiable trustworthiness. We’re moving toward a standard where the *quality of our skepticism* is measured alongside the *accuracy of our prediction*.

Tom: This shift has profound implications across fields—from public health, where we might find that local water treatment standards are an unobserved confounder in disease modeling, to climate science, where subtle geopolitical shifts affect data collection integrity.

Lu: The focus must become holistic. It's not just about fixing the math; it's about institutionalizing a deep, continuous suspicion of our own completeness. Every model build now carries the weight of acknowledging its inherent limitations.

Meng: For industry adoption, this means that risk assessment is no longer an afterthought handled by legal or compliance teams; it becomes a core mathematical output of the system itself. The tools must be designed to fail gracefully and transparently when data integrity is questionable.

Lalam: This level of rigorous self-assessment also has the power to democratize sophisticated analysis. By making the limits of knowledge visible, we prevent high-powered, complex models from becoming black boxes used to enforce existing biases or status quos.

Tom: Ultimately, this framework demands that data scientists become not just model builders, but expert philosophers of science—people who are equally skilled at defining the boundaries of what is knowable as they are at crunching the numbers.

Jane: Understanding this sophisticated requirement for systemic weakness sets us up perfectly to discuss another critical area: how do we ensure that the integrity and privacy of the data we *do* manage to collect—the very data needed to mitigate these unobserved confounders—remain protected in an increasingly interconnected world?

Conclusion: Tom: Ultimately, what we’ve discussed today regarding "Doctor Rashomon and the UNIVERSE of Madness: Variable Importance with Unobserved Confounding and the Rashomon Effect" is a massive methodological upgrade that forces us to be far more intellectually honest about the gaps in our own data collection.

Jane: It really changes the fundamental contract between the model builder and the end-user, doesn't it? It moves us away from blind faith toward verifiable skepticism.

Lu: From a purely theoretical standpoint, this methodology provides necessary rigor by elevating causality inference beyond mere correlation. We are now equipped to confront the inherent messiness of complex systems without giving ourselves false confidence in our knowledge gaps.

Meng: And for those of us actually building these tools in the real world, Lu’s point is everything: it provides a verifiable measure of uncertainty. That quantifiable level of caution is what allows for genuinely responsible deployment, which we simply can't afford to ignore anymore.

Lalam: I think the implications for equity are massive. Being able to flag that our conclusions about risk or opportunity are dependent on unobserved socioeconomic factors means we can build systems that are proactively designed to be less biased from the outset.

Tom: It truly gives us a mathematical tool not just for science, but for accountability itself. We have to show not only *what* the model predicts, but *why* it might be wrong under certain conditions.

Jane: The main takeaway from "Doctor Rashomon and the UNIVERSE of Madness: Variable Importance with Unobserved Confounding and the Rashomon Effect" is that acknowledging what we don't know becomes just as important as knowing something.

Tom: It’s a profound recalibration of scientific ambition—a recognition that true wisdom often begins with admitting ignorance.

Jane: So, while we close the book on this incredible paper, I think it leaves us perfectly positioned to pivot toward another kind of systemic failure next week—one rooted in data governance and privacy law.

Jon Donnelly, Srikar Katta, Emanuele Borgonovo, Cynthia Rudin

cs.LG

Submitted: 2026-08-20

Updated: 2026-08-24

Importance score: 84/100

The gist: The paper addresses "Variable Importance with Unobserved Confounding and the Rashomon Effect," providing theoretical bounds and empirical verification for estimating variable importance under

Key concepts

Variable Importance: The process of determining how much a specific input factor contributes to an outcome. The paper upgrades this by accounting for variables that were not collected (unobserved confounders), moving beyond simple correlation.
Unobserved Confounding: The risk posed by systemic factors or variables that are relevant to the outcome but could not be measured or included in the dataset (e.g., cultural trends). The methodology forces models to account for this missing information.
Structural Causal Inference: A sophisticated method of determining causation, moving beyond mere association (correlation). It focuses on modeling deeper causal relationships and structural constraints rather than just linear statistical links.
Certainty Map: A new output structure that accompanies a model's prediction. Instead of a single score, it provides an assessment of the model's own reliability and confidence interval, based on known data limitations.

Terminology

Regularization — 27× in this paper · explained in 3 episodes
Accuracy — 9× in this paper · explained in 1 episode
Distribution Shift — 7× in this paper · explained in 6 episodes
Core — 6× in this paper · explained in 2 episodes
Loss Function — 5× in this paper · explained in 2 episodes
Causal Inference — 4× in this paper · explained in 4 episodes
Robustness — 4× in this paper · explained in 5 episodes
alpha — 4× in this paper · explained in 1 episode
Verification — 3× in this paper · explained in 2 episodes
External Risk Estimate — 2× in this paper
Feature Selection — 2× in this paper · explained in 4 episodes
Shapley values — 2× in this paper · explained in 2 episodes
Uncertainty Quantification — 2× in this paper · explained in 8 episodes
Accountability — 1× in this paper · explained in 2 episodes
Generalization — 1× in this paper · explained in 13 episodes
Model-Agnostic — 1× in this paper · explained in 2 episodes
Structural Constraints — 1× in this paper · explained in 2 episodes

Summary

The paper addresses Variable Importance with Unobserved Confounding and the Rashomon Effect, providing theoretical bounds and empirical verification for estimating variable importance under challenging conditions.

The paper presents several mathematical bounds related to variable importance (j). These include:

f ′ in (n) (epsilon n + epsilon unobs + lambda sup)

And the core inequalities used for bounding are:

j (f ′, D(n)) − alpha at most* j (fu, P XY)

j (f ′, D(n)) − alpha − tau j at most* j (fu, P XY) − tau j

j (g ∗, PU)

These bounds rely on Lemma 1, which states that A symmetric argument applies for the upper bound, yielding that, with probability at least 1 − (delta + gamma), j (g ∗, PU) in as required.

The paper also defines specific forms for the infimum and supremum bounds:

f in (n) (epsilon n + epsilon unobs + lambda sup) j (f, D(n)) − tau j − alpha,

f in (n) (epsilon n + epsilon unobs + lambda sup) j (f, D(n)) + tau j + alpha

All experiments were conducted using specific computational resources and procedures.

Software/Hardware: All experiments for this work were performed on an academic institution’s cluster computer. We used up to 10 machines in parallel, each with a Dell R730’s with 2 Intel Xeon E5-2640 Processors (40 cores).
Model Computation: All experiments were run using TreeFarms Xin et al. (2022) to compute the Rashomon set of decision trees, with a depth bound of 3 and a regularization value of 0.001.
Data Preprocessing: For efficiency in semi-synthetic experiments, "we dropped all but the first 8 variables (before binarization) from each dataset... during binarization, each variable was processed into 3 binary variables as evenly spaced quantiles over the distribution of the original input variable."
Data Split: We used 80% of all data in each setting to compute the Rashomon set, and the remaining 20% to estimate variable importance over this set.

The core empirical evaluation involves verifying three major theoretical claims (Theorems 1, 2, and 3) regarding variable importance bounds. The analysis consistently compares two strategies for setting upper bounds on the size of S*: using the estimated Rashomon set size or using the model class size.

Theorem 1 (Verifying Finite Sample Biases):

Figure 7 verifies Theorem 1.
The results demonstrate that omitting finite sample adjustments yields Rashomon sets that leave out necessary models.
In contrast, the proposed adjustment yields the target coverage rate, verifying the theorem holds.

Theorem 2 (Verifying Variable Importance Estimation Uncertainty):

Figure 8 verifies Theorem 2.
The analysis shows that achieving a specified coverage rate of 0.8 only when adjusting for (i) model uncertainty via Theorem 1 and (ii) variable importance estimation uncertainty (in gold).
The paper notes that Adjusting for model uncertainty alone (in blue) is not sufficient.

Theorem 3 (Verifying VI Drift):

Figure 9 verifies Theorem 3.
The key finding is that We consistently achieve the specified coverage rate of 0.8 only when we account for (i) model uncertainty, (ii) variable importance uncertainty, and (iii) VI drift.
The figure details that all three adjustments are necessary: All three adjustments are necessary to achieve the target coverage rate of 0.8, with delta = gamma = 0.1.

A critical finding across the verification process is the comparison between two bounding methods:

Model Class Size: Using this method is described as overly conservative, leading to bounds that contain the true VI for all models in S* always.
Estimated Rashomon Set Size: Using this estimate leads to bounds that are probabilistically more valid and are "slightly less conservative; for example, the Wine Quality dataset at 10,000 and 50,000 samples creates bounds that have coverage < 100%."

The final evaluation assesses the cost associated with these necessary adjustments.

Figure 10 displays the average interval width.
The conclusion is that As the sample size increases, our intervals become much tighter–regardless of whether we estimate the Rashomon set size or use the model class size as our upper bound on the size of S*.
Crucially, "each additional adjustment increases interval widths very slightly but contribute to coverage at the specified rate. Our adjustments are therefore a small cost to pay for improved Type-1 error control, especially at larger sample sizes."

Improvements for AI systems

The core improvements center on moving AI systems beyond simple predictive accuracy (Accuracy to Causality) by integrating rigorous uncertainty quantification (UQ) into feature selection and variable importance estimation. The current state-of-the-art often fails when facing unobserved confounding, which the paper explicitly addresses.

Improvement: Implement a dedicated module that estimates the true variable importance (j(f*, P XY)) by explicitly accounting for three sources of uncertainty simultaneously:

Model Structure Uncertainty: Recognizing that the underlying model (f) is not perfectly known (analogous to Theorem 1 adjustment).
Variable Importance Estimation Uncertainty: Quantifying the variance inherent in the estimate of how much a variable contributes (VI-estimation uncertainty, analogous to Theorem 2 adjustment).
Distribution Shift/Omitted Variable Drift: Correcting for changes in data distribution or unmeasured confounders that bias importance estimates (analogous to VI drift, analogous to Theorem 3 adjustment).

What the Improved AI System Can Do:

Causal Feature Ranking: Instead of providing a single score (e.g., SHAP value) for feature importance, the system outputs a confidence interval for each feature's true causal impact. This allows downstream decision-makers to know how much they can trust the ranking.
Confounder Flagging: If the calculated confidence intervals are excessively wide or if the adjustment terms (tau j or lambda sup) indicate significant drift, the system flags potential unobserved confounding variables, alerting the user that prediction accuracy might be high but causally misleading.
Intervention Planning: Enables simulation of counterfactuals with higher fidelity. It doesn't just predict P(YX); it estimates E[Ydo(X=x)] with quantifiable uncertainty bounds, which is critical for policy-making and resource allocation.

Sources

Floodgate: inference for model-free variable importance