DACRI: Decision-Aware Causal Intervention Ranking for Critical Supply Chains

DACRI: Decision-Aware Causal Intervention Ranking for Critical Supply Chains

Abstract: Detecting or attributing a supply-chain disruption is not the same as selecting the intervention that maximizes recoverable net value. We present CriticalSCM-Bench v1, a controlled synthetic benchmark with causal ground truth, paired factual/counterfactual rollouts, and an explicit net-value objective. Relative to a full-information train-selected static benchmark, LambdaMART improves median normalized net value by 5.7–16.2%, with paired statistical support on the semiconductor and critical-material archetypes but not on digital infrastructure. On digital infrastructure, a domain-informed constant-buffer policy remains stronger, showing that greater model complexity is not uniformly justified. Across partial and delayed settings, LambdaMART retains 33–75% of full-clamp value. Stress tests further show that intervention fidelity, timing, cost, and held-out disruptions can alter policy ordering. Critical materials show the weakest out-of-distribution retention. Separately, a guarded explanation study over 540 generations preserves every fixed intervention decision after deterministic validation and template fallback, although exact wording remains unstable. Within this controlled setting, the results identify regimes in which adaptive ranking adds value and those in which simpler structural policies remain preferable.

Introduction: The technical gap is not detection alone. It is deciding where an operator should act now to recover the most value. DACRI addresses this gap by framing intervention selection as a learning-to-rank problem. The objective combines counterfactual loss avoided with the cost of acting.

Contributions:

1. CriticalSCM-Bench v1: a controlled benchmark with intervention costs and paired counterfactual labels for three critical-supply-chain archetypes.

2. Decision-aware evaluation: a net-value objective and a baseline hierarchy spanning information-light heuristics, attribution-as-action, a full-information static benchmark, an oracle-like structural rule, learned ranking, and a perfect-value upper bound.

3. Assumption stress tests: partial/delayed interventions, four feasibility specifications, cost and detection timing, learning curves, held-out disruptions, and an oracle ceiling.

A secondary guarded-explanation study evaluates whether a fixed DACRI decision can be rendered into operator-facing text without allowing the language model to own or alter the recommendation.

Problem Setting: Three archetypes support full evaluation: digital infrastructure hardware lifecycle (DI), calibrated using drive-reliability and electronics lifecycle data; semiconductors (SE) (UCI SECOM); and critical materials (CM) (USGS). Directionality is normalized so larger values indicate greater risk. Public datasets provide statistical calibration. Causal graphs and counterfactual labels come exclusively from the controlled benchmark.

For anomaly e = (k, t, sk,t), where k is the flagged KPI, t its detection time, and sk,t its score, candidates are upstream ancestors of k in the candidate graph Gc. In v1 the detection target is not an arbitrary KPI: an episode enters the ranking study only when the flagged KPI is the archetype’s service-risk node, so k always denotes that node.

The ranking label Yi,e truncates negative raw values to create a zero-valued no-action reference. The LambdaMART score orders candidates; it is not a calibrated estimate of Yi,e. The inverse-distance term is a stylized proxy for intervention type rather than coordination distance. Candidates closer to the service-risk node typically represent downstream buffering, emergency capacity, or allocation actions requiring sustained resource commitment, whereas upstream process corrections are modeled as lower-cost one-time actions. The pre-specified scale is ca = 0.3 for DI/SE and ca = 0.1 for CM.

From detection time onward, do(stabilize i) hard-clamps KPI i to baseline mean µi. This is the primary benchmark label specification. Factual and counterfactual rollouts share a seed, so Yi,e is paired. Because this idealization is strong, we treat partial, delayed, and stochastic responses as assumption stress tests.

DACRI Framework: The ranker scores each candidate using: anomaly severity, causal-strength statistic and lag, temporal precedence, lag-alignment residual, downstream criticality, lifecycle-recovery feasibility, and inventory-buffer depletion. Relevance labels come from paired simulations. The headline ranking experiment uses benchmark G to isolate ranking; a separate graph-swap regime retrains under recovered Ĝ and no graph.

LambdaMART (LightGBM) is the episode-adaptive learned policy. We compare it against a conceptual hierarchy. Severity-only is an intentionally information-light monitoring baseline. True-root is an attribution-as-action baseline that acts on the injected source. Train-selected static chooses the single training-set node with highest average net value and reuses it for every test episode. Constant-buffer always intervenes on the archetype-specific loss-driving buffer. A perfect-value oracle selects the highest-value option in C(e) ∪ no action, with no-action value zero, and provides an upper bound using realized counterfactual value.

CriticalSCM-Bench v1: CriticalSCM-Bench combines a hand-designed temporal DAG, autoregressive trajectories with seasonality/noise/missingness, sampled disruptions, and paired counterfactual labels. Calibration magnitudes draw from Backblaze drive data, electronics lifecycle data, SECOM, and USGS; graph and intervention ground truth remain synthetic. Calibration fixes the marginals of input nodes only. Downstream service-risk dynamics follow from graph propagation and are not empirically calibrated: realized dispersion at those sinks exceeds the calibrated standard deviation by roughly 590× (DI), 302× (SE), and 6× (CM). Benchmark magnitudes at the loss-bearing nodes should therefore be read as graph-determined, not data-derived. Train/test base seeds are disjoint (200 episodes each).

Digital Infrastructure (DI): 12 nodes, two-stage topology. Drive reliability is calibrated from Backblaze; lifecycle and bill-of-material context comes from electronics data. Disruptions: recycler yield drop (15–40%), fault-rate spike (1.5–3×), lead-time extension (+5–21d). Lags: 7–14d (yield→buffer), 1–3d (buffer→downtime). For the 30-day horizon, Le = Στ wq(τ)d+(τ), where d+ = max downtime risk − µd, 0 and wq(τ) = max q(τ) − µq + 1, 1 weights asset priority q.

Semiconductors (SE): 10 nodes spanning fab and downstream. UCI SECOM calibration. Disruptions: quality spike (2–5×), rework-yield drop (10–30%), lead-time extension (+7–28d). Lags: 2–5d (quality→inventory), 3–7d (inventory→delay). Over 21 days, Le = λs Στ wp(τ)S+(τ) + λd Στ wp(τ)D+(τ), with λs = λd = 1; S+(τ) and D+(τ) are inventory below and delay above baseline; wp(τ) = max p(τ) − µp + 1, 1 weights product priority p.

Critical Materials (CM): 9 nodes. USGS calibration. Multi-week propagation (4–8 week purity-to-availability lag). Disruptions: purity decline (3–10 percentage points), export restriction (20–50%), stockpile draw (30–60%). Over 26 weeks, Le = Στ wc(τ)S+(τ), where S+(τ) is allocation shortfall above baseline and wc(τ) = max c(τ) − µc + 1, 1 weights end-use criticality c.

Experiments: The main ranking study uses 5 seeds with 200 disjoint train and test episodes per archetype. Unless stated otherwise, seed-level point estimates are medians. Reported seed-level intervals are interquartile ranges (IQRs), whereas paired episode-level analyses use bootstrap confidence intervals (CIs). Because the ranking question is separate from detector selection, the main study holds detection timing at the controlled proxy tdet = tinj + 5. Ranking quality uses normalized discounted cumulative gain at 5 (NDCG@5), while the separate detector study uses a FAR budget of 1 event/month.

Decision-Aware Ranking: Relative to the full-information train-selected static benchmark, LambdaMART’s median gain is 16.2%/5.7%/11.1% (DI/SE/CM). It does not dominate stronger references: constant-buffer is 5.1% better on DI and the equal-weight adaptive heuristic 3.5% better, both relative to LambdaMART, which leads only on SE and CM and by modest margins. Table II therefore supports a bounded conclusion: episode adaptation adds value in some regimes, but greater model complexity is not uniformly beneficial.

Paired bootstrap confidence intervals and sign-flip tests use 1,000 test episodes with disjoint episode seeds across the five runs. They support LambdaMART over static on SE (mean +.0015, 95% CI [.0003,.0027], p =.015) and CM (+.0062, [.0013,.0104], p =.008), but not DI (+.0061, [-.0063,.0182], p =.323). Against constant-buffer, DI is significantly worse (-.0055, [-.0086,-.0028], p <.001), while paired tests favor LambdaMART on SE and CM.

The most consistent result is the distinction between attribution and action. Severity-only obtains 7.2%/6.1%/32.9% of LambdaMART’s value, while true-root is near zero on DI/SE and below LambdaMART on CM. The true-root result is partly a property of the feasibility model rather than of root-cause action as such: a just-disrupted node is frequently driven to the feasibility floor, contributing to near-zero DI value. LambdaMART reaches 57%/55%/88% of the perfect-value oracle. The buffer is value-optimal in only 19.5%/18.0%/9.5% of episodes, so a fixed buffer is strong on average without being episode-optimal.

Detection and Causal-Structure Results: On DI, LSTM-AE provides greater event coverage and substantially shorter censored delay than EWMA under the same FAR budget. Detector selection should therefore consider coverage and lead time jointly. CM EWMA detects fewer than half the events in every seed. Using a PCMCI significance threshold αPCMCI = 0.01, PCMCI recovers 0.75 / 0.44 / 0.67 of true edges on DI / SE / CM (medians over five seeds). Granger recovers 0.58 / 0.56 / 0.78 on the same seeds, so Granger has higher recall on SE and CM and lower recall on DI. In a controlled graph-swap (3 seeds × 150; ranker retrained per regime), recovered Ĝ versus benchmark G gives net value 0.060/0.060 (DI), 0.059/0.048 (SE), and 0.038/0.038 (CM); no graph gives 0.036/0.035/0.024.

Learning Curve: With 25/50/100/200 training episodes (3 seeds, 200 test), LambdaMART median net value is.028/.058/.067/.068 (DI),.039/.041/.046/.052 (SE), and.016/.022/.023/.053 (CM). Medians improve with sample count, but the path is not smooth. Seed-level trajectories remain noisy, and CM changes sharply only at 200. The curves show that the policy is sample-sensitive; they do not show convergence.

Feature Ablation: Leave-one-out (3 seeds × 150 episodes): Lifecycle feasibility carries the clearest signal. Removing lifecycle feasibility causes the largest median drop (-.050/-.012/-.020 for DI/SE/CM); removing edge lag gives -.028/-.015/-.013. Other removals are smaller or can improve value, which suggests redundancy rather than a feature set in which every variable is uniformly helpful.

Cost-Scale Sensitivity: We sweep ca ∈.1,.2,.3,.5 (3 seeds × 150). LambdaMART exceeds constant-buffer only at.1 on DI, is equal or slightly higher on SE, and leads CM at.1–.2 but loses at.3–.5. Buffer-optimal frequency falls with cost. The cost model is an explicit benchmark factor, not an empirical monetary estimate; the sweep evaluates policy dependence on this factor.

Feasibility and Intervention Sensitivity: The feasibility scalar is a synthetic intervention-effect proxy, not an empirical recovery probability. We rerun the injected-root versus loss-optimal agreement audit under four paired specifications: no attenuation (ϕ = 1), mild attenuation (γ = 0.375, ϕmin = 0.1), the current setting (γ = 0.75, ϕmin = 0.1), and a higher floor (γ = 0.75, ϕmin = 0.3). Root/optimal disagreement spans 89.7–99.7% (DI), 97.3–100% (SE), and 57.0–74.3% (CM), remaining high across all four specifications with greater variation on CM.

Since ϕ uses calibrated rather than larger realized dispersion, 51.2%/47.8%/51.9% of DI/SE/CM candidate-episodes are floored, rising to 87.8% (DI) and 75.0% (CM) for disrupted candidates. This contributes to near-zero DI true-root value and limits interpretation of magnitude. The audit supports persistence of root/action disagreement, not empirical calibration of its size.

Partial effects alter value and ordering: constant-buffer remains stronger on DI and becomes competitive on delayed SE. Policy ordering is therefore not invariant to intervention fidelity. A further stochastic-response stress test introduces efficacy variation, outright failure, delay, ramping, and decay (5 regimes, 5 seeds). LambdaMART retention relative to the full-clamp benchmark falls to 0.51/0.29/0.24 under the moderate regime and 0.13/0.00/0.06 under the weak temporary regime for DI/SE/CM. No candidate has positive net value in 45–99% of episodes, and CM’s optimal node changes in up to 63%. Scenario-matched retraining on noisy labels does not consistently outperform the frozen full-clamp ranker.

Forced action has mean value −0.13/ − 0.12/ + 0.00 under full clamp (DI/SE/CM), and roughly half of those episodes lack a positive-net candidate; under stochastic response the share reaches 99%. Separately for each archetype, seed, and response regime, a training-selected rule chooses the top-score cutoff that maximizes summed normalized raw net on training episodes, then freezes it for test. On full-clamp episodes, this rule retains 30–52% of the corresponding clipped-label headline value. Across stochastic regimes, it abstains in 71–100% of episodes and remains approximately break-even (minimum mean test value −0.004).

Circular-Evaluation Control: To test dependence on counterfactual labels, a ranker trained once on hard clamps is frozen and evaluated under unseen intervention specifications against scenario-matched retraining (5 seeds × 100 train/test). Across the four mismatched idealizations, frozen/scenario-matched median value ratios range from 1.02 to 1.13 on DI and from 0.93 to 1.21 on SE; the CM mean across specifications is 0.83. On CM, the frozen ranker loses about half its value relative to matched retraining under five-step delay (ratio 0.49). Dependence on counterfactual labels is therefore mild on DI and SE but material on CM, and the circular-evaluation concern is reduced rather than removed: label mismatch, OOD, and transfer tests all remain inside the same autoregressive simulator family.

Out-of-Distribution Generalization: In a paired leave-one-disruption-template-out OOD test, the same fitted model retains 95%/92%/62% in-domain net value on DI/SE/CM (0.069→0.065, 0.052→0.048, 0.047→0.029). NDCG retention is 96%/95%/78%. Constant-buffer OOD net value is 0.072/0.046/0.049, exceeding LambdaMART on DI and CM. Thus CM has a substantial held-out-disruption weakness.

Schema Transfer: Frozen DI→SE and DI→CM rankers retain 75% (0.036 vs 0.048) and 65% (0.025 vs 0.038) of in-domain value. Transfer is a secondary stress test. The larger CM drop is consistent with several schema and time-scale differences. Because those differences move together, this experiment does not isolate lag scale as the cause.

Controlled Lag-Scale Test: To probe the lag explanation directly, we ran a controlled test that holds topology, weights, calibration, cost, disruptions, and the feature schema fixed and varies only the edge-lag scale (factor f ∈ 0.5, 0.75, 1.5, 2.0 about the trained regime), evaluating a frozen ranker against one retrained at each f (5 seeds). Across the four factors, no monotone degradation pattern is observed. CM degrades only on the stretch side—frozen-to-retrained value falls to 0.79/0.60 at f = 1.5/2.0, the day-to-week direction—while lag compression does not hurt and DI/SE show no consistent effect. Thus lag scale alone does not explain the cross-archetype transfer drop in this controlled test.

Real-Data Detection Sanity Check: We use Backblaze Drive Stats Q1 2024 (25.2M drive-days, 978 failures). Isolation Forest detects pre-failure anomalies in 39% of failed drives at FAR 0.057 per 1k drive-days, median 21 days before failure; EWMA detects 4%. The Backblaze result supports the plausibility of early detection and the detection-to-decision motivation; detector ordering differs from simulation and therefore remains deployment-specific.

Detection-Timing Sensitivity: At detection offsets 3/5/10/14 time steps—days for DI/SE and weeks for CM—(3 seeds × 150), LambdaMART value is.059/.060/.011/.008 (DI),.059/.048/.027/.010 (SE), and.079/.038/.013/.001 (CM). Because intervention begins at detection, late flags leave less recoverable loss. Here the mechanism is direct: detection timing changes the value that remains available to the policy. The effect is severe, especially on CM.

Illustrative Intervention Walkthrough: Consider DI episode 2000001. A fault-rate spike is injected at drive failure rate (drive failure rate) at t = 49 and detected at t = 54; no-action loss is 540.1. Severity-only selects recovery throughput (rework throughput) and true-root selects drive failure rate; both have zero net value. LambdaMART selects spare-buffer level (spare buffer level), obtaining 24.9 (0.046 normalized), while spare availability (spare availability) obtains 16.2. This episode is illustrative rather than general evidence, but it shows that the largest anomaly, injected source, and best intervention can differ.

Guarded Operator-Explanation Layer: This secondary study asks whether a language model can verbalize a fixed DACRI decision, whether evidence constraints improve first-pass acceptance, and whether validation provides fallback. Packets omit counterfactual labels, and the schema has no recommendation field. The validator checks decision preservation, packet entities, edges, lags and numbers, and rejects verified root-cause or promised-outcome claims; rejected drafts use a packet template.

We build 90 packets (30 per archetype, 11 carrying a no-action decision) and generate three times per packet under a generic and an evidence-constrained prompt: 540 calls using the recorded API identifier deepseek-v4-flash. For this explanation study only, no-action is locked when the top ranker score falls below the pre-specified 10th percentile of an out-of-fold training top-score distribution. This rule uses no test labels.

No output redirected the fixed decision, and none of the 66 no-action outputs recommended intervention. Evidence-constrained prompting improves first-pass acceptance from 74.1% to 90.4%, with 59 discordant pairs favoring it against 15 favoring generic (exact McNemar p = 2.55 × 10−7). Evidence-constrained drafts achieve 100% structured grounding under the specified checks, compared with 99.3% for generic prompting; guarded outputs reach 100% after deterministic validation and fallback. All 26 rejections in the evidence-constrained condition arise from one banned claim: describing the intervention target as the verified root cause, an attribution–intervention conflation.

Generation stability is the weakest outcome: neither prompt yields three identical generations for any packet; fallback raises exact stability only to 2.2%. Prose varies across generations, so deployment-oriented reporting should disclose this variability. Checks cover structured claims and numeric tokens, not usefulness, style, or complete semantic faithfulness. All packets are simulator-derived, and no human audit was completed; this remains a constrained secondary component.

Discussion: Attribution is not equivalent to intervention selection: severity-only and true-root miss value, although feasibility-floor saturation partly drives the true-root result. Adaptation is statistically supported over the full-information static benchmark on SE and CM, but not DI, where constant-buffer wins. Operational complexity was not evaluated. The oracle gap leaves further policy headroom.

The stress tests map the benchmark’s validity envelope. Intervention fidelity, cost, timing, and graph specification alter value and ordering, while CM is the weakest held-out-disruption regime. These are explicit, replaceable benchmark factors rather than hidden deployment estimates: the results locate where adaptive ranking helps within the controlled environment and where structural policies remain preferable.

CriticalSCM-Bench supports controlled comparative evaluation, not deployment certification. Independent simulators and retrospective operational data must test transportability, monetary costs, and organizational feasibility. The guarded explanation study evaluates a constrained renderer’s safety and grounding, not human factors.

Conclusion: DACRI starts from a simple distinction: finding the source of a disruption is not the same as choosing where to act. CriticalSCM-Bench makes that distinction measurable through a paired, cost-aware objective. Within the benchmark, severity and true-root often miss recoverable value. LambdaMART improves a full-information static benchmark on SE and CM, but loses to a strong structural rule on DI. Partial, delayed, cost, and OOD tests identify regimes with statistically supported benchmark gains from episode-specific ranking and those favoring structural rules. The contribution is a falsifiable benchmark for deciding where to act, not universal learned-policy superiority. Independently implemented simulators and retrospective operational data provide the next tests of transportability.

Improvements to AI Systems Based on This Paper:

1. Decision-Aware Ranking Over Attribution: AI systems should be built to select interventions that maximize recoverable net value (counterfactual loss avoided minus action cost), rather than merely identifying root causes. The improved system can rank candidate actions by expected value, using paired factual/counterfactual rollouts to learn optimal policy, and can distinguish between where is the problem and where should I act.

2. Adaptive Policy Selection with Domain-Aware Fallbacks: AI systems should not default to complex learned models. The improved system can dynamically choose between episode-adaptive ranking (LambdaMART-style) and simpler structural rules (e.g., constant-buffer) based on archetype characteristics, cost scales, and detection timing. It can identify regimes where complexity adds value (semiconductors, critical materials) and where it does not (digital infrastructure), preventing overfitting to model complexity.

3. Robustness to Intervention Fidelity and Timing: AI systems should be stress-tested under partial, delayed, and stochastic intervention responses. The improved system can retain 33–75% of full-clamp value under degraded conditions, abstain from action when no candidate has positive expected net value (up to 99% of episodes under stochastic response), and use scenario-matched retraining when intervention dynamics shift, rather than relying on frozen policies.

4. Out-of-Distribution and Transfer-Aware Generalization: AI systems should explicitly measure and report performance on held-out disruption types and cross-schema transfers. The improved system can flag weak generalization regimes (e.g., critical materials retaining only 62% OOD value) and trigger retraining or fallback to structural rules when OOD confidence is low, rather than assuming uniform transferability.

5. Cost-Sensitive and Feasibility-Aware Decision Making: AI systems should incorporate explicit cost scales and intervention feasibility floors into their optimization. The improved system can adjust policy ordering as costs vary (e.g., LambdaMART wins only at low cost on DI), and can account for feasibility saturation (where many candidates are floored) to avoid over-interpreting magnitude differences.

6. Guarded Explanation with Deterministic Validation: AI systems that generate operator-facing explanations should separate decision-making from language generation. The improved system can lock the AI's recommendation, use evidence-constrained prompting to raise first-pass acceptance from 74% to 90%, and apply deterministic validation with template fallback to guarantee decision preservation—while explicitly disclosing prose instability rather than hiding it.

7. Detection-to-Decision Integration: AI systems should jointly optimize detection timing and intervention value, not just detection accuracy. The improved system can quantify how detection delay erodes recoverable value (e.g., value drops from.059 to.008 when detection offset goes from 3 to 14 steps on DI) and can prioritize detectors with shorter censored delay and higher coverage (e.g., LSTM-AE over EWMA) under a fixed false-alarm budget.

8. Sample-Efficient Learning with Explicit Convergence Monitoring: AI systems should report learning curves and avoid claiming convergence without evidence. The improved system can track performance across training set sizes (25–200 episodes), identify noisy or non-smooth trajectories, and flag when more data is needed (e.g., CM only improves sharply at 200 episodes) rather than assuming monotonic gains.

9. Feature Importance with Redundancy Awareness: AI systems should use leave-one-out ablations to identify critical features (e.g., lifecycle feasibility, edge lag) and recognize redundancy. The improved system can prune non-contributing features without value loss, and can prioritize features that carry the clearest causal signal for the decision task.

10. Circular-Evaluation Control and Label-Mismatch Resilience: AI systems should test dependence on counterfactual labels by freezing a policy and evaluating under mismatched intervention specifications. The improved system can quantify label-mismatch sensitivity (mild on DI/SE, material on CM) and trigger retraining when label assumptions shift, reducing overconfidence in simulated benchmarks.