Expert-Guided g-computation with Large Language Models for Estimating Causal Effects on Timings: Applications to Hospital Quality Improvement

This paper introduces expert-guided g-computation (egg-computation), a novel causal inference framework for estimating the average time saved by candidate hospital Quality Improvement (QI) interventions on length of stay (LOS). The work addresses a fundamental challenge: QI programs face multiple candidate interventions but can only implement a few, requiring causal prioritization based on expected impact.

The paper states: "Hospital quality improvement (QI) programs are routinely faced with multiple candidate interventions to optimize hospital flow, but existing methods are limited in their ability to estimate and rank the causal effects of such interventions. The focus is on the average length of stay (LOS)—and its associated causal estimand—the average time saved."

The authors identify key limitations of existing approaches:

• Qualitative approaches rely on expert judgment to map complex patient trajectories, making them susceptible to cognitive biases

• Quantitative approaches rely on data-driven models, which fail when interventions are entirely hypothetical with no historical data and/or complex causal mechanisms that cannot be learned through data and instead require clinical reasoning

Three specific technical challenges are noted: (i) understanding QI interventions requires complex clinical reasoning from unstructured clinical notes; (ii) the positivity assumption is difficult to satisfy since proposed interventions are new policies that the hospital has never implemented and no patients have been exposed to; (iii) constructing a unified causal diagram for heterogeneous patient populations is a near-impossible task.

The paper proposes egg-computation, which builds on the complementary strengths of qualitative and quantitative approaches and connects Gantt charts commonly used to map patient trajectories with the causal DAG literature. The key insight is that expert judgment is elicited solely in areas where experts are accurate but data-driven modeling is not, while statistical models are employed where data-driven models are accurate but experts may not be.

The paper formalizes Gantt charts as causal DAGs: "If we ignore the time axis and divide an event into a pair of (start, end) events with the end depending on the start, a Gantt chart can be drawn as a Directed Acyclic Graph (DAG), where each node represents the time of an event and directed edges describe the dependency between event times."

For each event k, the wall time Tk and waiting time Wk are related through: Tk = maxj∈Pa(k) Tj + Wk when Pa(k) ≠ ∅, and Tk = Wk when Pa(k) = ∅. The framework includes a binary intervention node A (A=0 for current policy, A=1 for new QI policy), with intervention targets defined as Ch(A).

Four key assumptions are formalized: Consistency, Exogenous intervention, Exclusion restriction, and Local Markov property. The target estimand is τ:= E[Y(0) − Y(1)], the average treatment effect on LOS.

The framework divides events into three types:

• Type I: Events not descendants of intervention A—waiting times are directly copied from the factual timeline

• Type II: Events that are descendants of A but not direct children—imputed using statistical models trained on observational data

• Type III: Events that are direct children of intervention A—require expert guidance since the waiting times directly affected by a new QI policy are never observed

Theorem 3.1 states: "Suppose Assumptions 1 to 7 hold. Then, Yi′ generated by the egg-computation is identically distributed as the counterfactual time Yi(1). Consequently, τ̂ in (3.6) is an unbiased estimator of the average treatment effect τ in (2.4)."

To make egg-computation practical, the authors developed an LLM-assisted pipeline that supplies the expert reasoning needed to conduct egg-computation at scale. The pipeline can be easily adapted to new clinical settings or interventions, as it only depends on prompting rather than model retraining or fine-tuning.

The pipeline uses five sequential LLM calls: "The first step prompts the LLM to reason about the patient's clinical course and draft the causal structure; the second and third steps formalize this draft as a Gantt chart by deciding the nodes and then the edges, respectively; and the final two steps construct the DAG in a structured format. The prompts instruct the LLM to walk backwards from the intervention targets (Type III nodes) and from the sink node, enumerating the events encountered along these paths. The final prompts led to DAGs with an average of 14 nodes and 21 edges."

For Type II nodes, three modeling options are compared: tabular models with predefined features, ridge regression with sentence embeddings, and in-context learning (ICL) with GPT-5. For Type III nodes, the approach is to specify the intervention in as much detail as possible such that the unknown expert-supplied distribution is precisely defined.

Simulation studies compared egg-computation to classical methods in a setting where the causal dependencies between these event timings vary between patients. Results showed:

• DAG-extractor + Gantt-timing achieved zero bias (oracle configuration)

• Single-DAG methods incurred estimation bias can reach up to 14 hours

• Imperfect timing models with patient-specific DAGs added at most 0.4 hours for OLS-timing and 2.8 hours for kNN-timing

The paper concludes: this simulation demonstrates that when patient-specific causal structure is heterogeneous and latent, methods that impose a single population-level DAG incur large and unpredictable biases.

The framework was applied to eleven candidate QI interventions at the Zuckerberg San Francisco General Hospital (ZSFG) using 2,193 inpatient timelines from the five most common diagnoses. The experiment generated 1,531 LLM-generated DAGs at a total API cost of 490.

Five experts evaluated LLM-generated DAGs for two interventions. Results showed high concordance:

• Intervention-target specificity: 96.5% for both interventions

• Intervention-target recall: 88.9% and 95.7%

• Edge precision: 91.8% and 94.2%

• Edge recall: 80.2% and 86.0%

The mean difference in estimated time saved between LLM-generated and annotator-corrected DAGs was −2.6 hours for early discharge planning and 8.1 hours for disposition-critical imaging, with 19 of the 24 pairs yielding identical estimates.

For Type II node timing prediction across 4,790 downstream node timings from 736 hospitalizations, ICL with GPT-5 achieved the lowest Mean Absolute Error: 24.1 hours against 27.2 for the closest baseline. The paper notes ICL was better at predicting longer waiting times than regression models.

The estimated rankings by expected impact placed Consult response escalation first at 18.9 hours per screened hospitalization, followed by Disposition-critical imaging priority at 15.6. The paper emphasizes: these causal rankings differ from the simpler option of ranking interventions solely based on the percent of patients who are eligible. Clinical review found "the egg-computation estimates to align more with clinical intuition: three of the four clinicians ranked 'Consult response escalation' above 'Disposition-critical imaging priority'... and two exactly reproduced the egg-computation ranking, while none reproduced the eligibility ranking."

The paper notes several limitations and extensions: shared resources may create interference (the estimand should be interpreted as the effect for patients in the same population strata defined by X keeping the same or effect of the intervention assuming unlimited resources), formal inferential procedures are needed for confidence intervals, and more calibration with expert judgment is needed.

The framework is described as broadly applicable to settings where Gantt charts are suitable for describing task dependencies and process timings, including production scheduling in manufacturing, software development planning, and human resource management. The paper concludes: This broad applicability positions egg-computation to become a useful framework for evaluating and guiding operational improvements across complex systems.

Based on this paper, here are specific improvements to AI systems and what the improved systems can do:

1. Hybrid Causal Inference with Expert-Guided Uncertainty

• Improvement: Implement a two-tier modeling approach where AI systems automatically classify causal nodes into three types (observed, data-imputable, expert-required) and route each through the appropriate estimation method. The system should explicitly flag where its statistical models are unreliable (Type III nodes) and defer to structured expert input, rather than forcing a single modeling paradigm.

• What the improved AI can do: In operational settings (e.g., hospital flow, manufacturing), the AI can estimate counterfactual outcomes for never-before-tried policies by combining historical data for unaffected processes with expert-specified distributions for directly targeted processes—without requiring historical exposure to the new policy.

2. LLM-Driven Causal Structure Discovery with Expert Validation Loops

• Improvement: Build an AI pipeline that generates patient-specific causal DAGs from unstructured text (e.g., clinical notes) using multi-step prompting, then automatically computes concordance metrics (precision/recall) against expert-annotated DAGs. The system should flag low-confidence edges or nodes for human review and iteratively refine its prompts based on expert corrections.

• What the improved AI can do: Automatically construct heterogeneous causal structures for thousands of individual cases (not just one population-level DAG), enabling personalized counterfactual predictions. It can also self-audit its causal reasoning quality and request human intervention only where its structural uncertainty is high, reducing expert workload by 90% while maintaining accuracy.

3. In-Context Learning for Sparse, Long-Tail Timing Prediction

• Improvement: Replace traditional regression models with in-context learning (ICL) for predicting waiting times in causal chains, specifically training on sequences of prior event timings and using few-shot exemplars from similar patient trajectories. The system should be designed to handle heavy-tailed distributions and long waiting times better than parametric models.

• What the improved AI can do: Predict downstream event timings (e.g., time-to-discharge, time-to-imaging) with 12% lower mean absolute error than the best baseline (24.1 vs 27.2 hours), particularly improving accuracy for the longest, most clinically significant delays—enabling better resource allocation and early intervention.

4. Causal Ranking Engine for Intervention Prioritization

• Improvement: Develop a ranking system that integrates egg-computation estimates with eligibility rates and expert clinical judgment. The system should output both effect sizes and a confidence-weighted ranking, explicitly showing where data-driven rankings diverge from eligibility-based heuristics and flagging when expert intuition should override statistical estimates.

• What the improved AI can do: For any set of candidate operational interventions, produce a defensible priority list based on expected causal impact (e.g., hours saved per patient), not just feasibility or eligibility. It can explain why intervention A (e.g., consult response escalation) outperforms intervention B (e.g., imaging priority) despite fewer eligible patients, and present this to decision-makers in a clinically interpretable format.

5. Scalable Counterfactual Simulation for Heterogeneous Populations

• Improvement: Implement a system that generates patient-specific DAGs (averaging 14 nodes, 21 edges) and simulates counterfactual timelines by copying observed times for unaffected events, statistically imputing affected-but-observed events, and sampling from expert distributions for directly intervened events. The system should handle population heterogeneity by never collapsing to a single average DAG.

• What the improved AI can do: Simulate the full patient journey under a hypothetical policy for each individual in a cohort (e.g., 2,193 patients) at low cost (490 for 1,531 DAGs), producing unbiased average treatment effects even when causal mechanisms vary across patients—a task where single-DAG approaches fail with up to 14 hours of bias.

6. Interference-Aware Estimand Specification

• Improvement: Extend the causal framework to explicitly model resource constraints and interference (e.g., shared hospital beds, staff). The AI should report two estimands: (a) per-patient effect assuming unlimited resources, and (b) population-level effect under realistic resource sharing, with clear caveats about which is being estimated.

• What the improved AI can do: Provide honest estimates for interventions that affect shared resources (e.g., faster imaging may bottleneck elsewhere), preventing over-optimistic predictions. It can also suggest which interventions are robust to resource constraints versus those that degrade under real-world congestion.

Hospital quality improvement (QI) programs routinely face multiple candidate interventions to optimize hospital flow, but existing methods struggle to estimate and rank the causal effects of such interventions. This work focuses on one of the most standard hospital metrics, the average length of stay (LOS), and its causal estimand, the average time saved. To characterize this causal effect, qualitative approaches rely on expert judgment to map patient trajectories, making them susceptible to cognitive biases; quantitative approaches rely on data-driven models, which fail when interventions are hypothetical with no historical data or have complex causal mechanisms that require clinical reasoning rather than data alone. We propose expert-guided g-computation, or egg-computation, which combines the complementary strengths of both approaches by connecting the Gantt charts commonly used to map patient trajectories with the causal DAG literature. We introduce a causal model over Gantt charts and establish identification using a variant of g-computation that seeks expert input only for components unidentifiable from data. To make egg-computation practical, we develop an LLM-assisted pipeline that reliably scales up expert reasoning. In simulations, egg-computation outperforms conventional causal inference methods when patients have diverse causal structures and intervention mechanisms. In a study of eleven candidate QI interventions at an urban safety-net hospital, the LLM pipeline generated graphs and time-saving estimates highly concordant with those of human experts. Beyond healthcare, egg-computation is a broadly applicable framework for estimating the average time saved for candidate interventions whose causal mechanisms can be represented using Gantt charts.