acceptodds
Under review as a conference paper at ICLR 2027

CausalGuard: Conformal Inference under Graph Uncertainty

Abstract

Current methods for estimating treatment effects harbor a fundamental vulnerability: they demand a brittle, premature commitment to a single, uncertain causal graph. If the adjustment set is misspecified, standard conformal prediction frameworks fail silently by flawlessly calibrating a causally meaningless pseudo-outcome. We introduce CausalGuard, a proactive aggregate-before-calibrate paradigm that resolves this structural vulnerability. Rather than gambling on one graph, CausalGuard utilizes LLM-elicited priors and conditional-independence screening to construct a weighted family of valid adjustment strategies, elegantly collapsing redundant graphs into shared downstream estimators for maximum computational efficiency. By aggregating graph-conditional pseudo-outcomes and bounds before applying a single conformal correction, we mathematically eliminate the interval bloat inherent to naive averaging. We prove finite-sample marginal coverage for this aggregate and formally establish its convergence to the Conditional Average Treatment Effect (CATE). Empirical benchmarking demonstrates CausalGuard dramatically tightens efficiency at comparable operating points. Crucially, in a metadata-free collider stress test, CausalGuard autonomously prevents structural bias, collapsing CATE RMSE to compared to the catastrophic error of forced-adjustment baselines, proving that robust inference requires unifying predictive calibration with causal integrity.

Primary area:
causal reasoning
Keywords:
Conformal predictions, Graph Uncertainty, Causal Methods, Causal Inference

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_YSRoOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

CausalGuard studies treatment-effect inference when the adjustment set is uncertain. It proposes candidate causal graphs using LLM-derived edge priors, screens edges using conditional-independence tests, and assigns graph weights using BIC and the prior. Graphs sharing an adjustment set share a downstream estimator. The method averages adjustment-specific AIPW pseudo-outcomes and preliminary interval endpoints, then applies split-conformal calibration to the aggregate. It establishes marginal predictive coverage for the aggregated pseudo-outcome and connects its conditional mean to CATE under additional alignment, nuisance-estimation, and weight-concentration assumptions. Experiments include causal benchmarks, collider stress tests, and component ablations.

Critical strengths
  1. The paper addresses a relevant concern: uncertainty about adjustment variables can introduce causal bias that predictive calibration alone cannot repair. Sharing estimators across graphs with identical adjustment sets is also a sensible computational design.

  2. The paper explicitly distinguishes pseudo-outcome coverage from empirical CATE coverage and includes graph-aware controls, Top-1 comparisons, prior perturbations, and multiple LLM backbones. These comparisons help assess which components contribute to the results.

Critical weaknesses
  1. The inferential objective does not clearly resolve the motivating problem. The guarantee covers a constructed pseudo-outcome, not the causal effect. Convergence of its conditional mean to CATE does not establish CATE confidence-interval coverage. Causal interpretation additionally requires weights to concentrate on target-aligned strategies, but the paper does not establish why its LLM/BIC procedure achieves this under observational ambiguity. The pipeline makes the identifying assumptions harder to interpret without clearly resolving identification uncertainty.

  2. The novelty and empirical value of the additional components remain unclear. Once the aggregate is fixed using training data, calibration follows standard split-conformal reasoning, while score domination follows from convexity. Empirically, Full and Top-1 have nearly identical widths on several datasets. On Jobs, Uniform BMA has similar width and coverage; in the IHDP prior comparison, uniform proposals have lower RMSE than the LLM prior (.859 versus .904). Four LLM backbones yield identical reported width and RMSE, which may indicate limited LLM influence rather than a demonstrated benefit. The collider result is promising, but does not isolate the necessity of LLM guidance or aggregation. Several central comparisons lack uncertainty estimates for paired differences.

Questions for authors
  1. Why is prediction of the aggregated pseudo-outcome the appropriate scientific objective, rather than confidence intervals for the causal effect? What guarantee addresses the causal estimand when adjustment uncertainty remains unresolved?

  2. What interpretable identification assumptions justify the aggregate, and what establishes that the proposed weights favor target-aligned strategies when different graphs fit the observational data equally well?

  3. What is technically new beyond ensembling followed by split-conformal calibration, and what matched empirical evidence establishes benefits over Top-1 selection and uniform proposals?

  4. Do different LLMs meaningfully change adjustment sets or weights? Do the nearly identical results demonstrate robustness, or little influence from the LLM?

Reviewer_77sJOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies treatment-effect inference when the causal graph and adjustment set are uncertain. CAUSALGUARD uses LLM-derived edge priors and conditional-independence screening to generate candidate DAGs, collapses graphs that imply the same adjustment set, assigns posterior-like weights using working BIC scores and structural priors, and fits an AIPW-based learner for each unique strategy. It aggregates strategy-specific pseudo-outcomes and preliminary bounds before applying a single split-conformal correction. The paper proves finite-sample marginal coverage for the resulting weighted pseudo-outcome and gives a conditional-mean consistency result for the CATE under target-alignment, nuisance-stability, and weight-concentration assumptions. Experiments use standard causal benchmarks, synthetic structural diagnostics, and an injected-collider stress test.

Critical strengths

S1. The paper highlights an important distinction between conformal validity and causal validity: conformal calibration can reliably cover a target produced by an invalid adjustment strategy without making that target causally meaningful. The manuscript keeps the finite-sample pseudo-outcome guarantee formally separate from its empirical CATE diagnostics.

S2. Aggregating graph masses at the adjustment-set level is computationally sensible, and freezing graph generation, weights, nuisance models, and preliminary bounds before calibration gives a clean split-conformal construction. The collider and candidate-support diagnostics are potentially useful starting points for studying structural uncertainty.

Critical weaknesses

W1. The central causal guarantee assumes rather than resolves the difficult part of graph uncertainty. Theorem 1 provides standard split-conformal marginal coverage only for the fitted weighted pseudo-outcome γˉ\bar\gamma, not for τ(x)\tau(x). Corollary 2 assumes a nonempty target-aligned candidate class satisfying the desired conditional moment identity, and Assumption 4 further assumes that working BIC scores separate every aligned graph from every misaligned graph by a diverging gap. Theorem 3 then mechanically converts this assumed gap into weight concentration. No result connects the proposed LLM sampling and Fisher-z pruning to target-aligned candidate support or establishes that observational BIC can distinguish causal alignment; tied scores are explicitly resolved by the semantic prior. Thus the method can still conformally cover a structurally biased aggregate, which is the failure mode the paper claims to eliminate.

W2. The experiments do not establish that structural aggregation or the LLM-guided graph mechanism causes the reported gains. On most full-data benchmarks, Full CAUSALGUARD, Top-1, no-LLM-prior, and no-CI-pruning variants have nearly identical coverage and width, while reported structural uncertainty is essentially zero; PC, GES, and CAUSALGUARD are also nearly identical in Table 19. The strongest prior benefit uses an oracle-informed synthetic prior rather than an elicited LLM prior. The collider study reports only final RMSE without candidate graphs, weights, adjustment sets, or a matched mechanism ablation. Comparisons additionally mix CATE and pseudo-outcome targets, use independent alpha sweeps with only three seeds in Table 2, and include unmatched achieved coverage. These results cannot support the broad claims of autonomous structural protection or robust causal intervals.

Questions for authors

Q1. Assessment-changing theory: Is there an existing result that derives target-aligned candidate support and BIC weight concentration from the actual LLM-proposal and Fisher-z-pruning procedure, rather than assuming both properties? How are Markov-equivalent or tied-score graphs with different adjustment implications handled without relying on an unverifiable semantic prior?

Q2. Clarification: Does any theorem establish coverage of the CATE τ(x)\tau(x) itself, rather than coverage of γˉ\bar\gamma or convergence of its conditional mean? If not, please state precisely what inferential claim a practitioner may attach to C^(x)\hat C(x) when target alignment or BIC separation fails.

Q3. Assessment-changing evidence: For the collider stress test, can the authors provide the candidate adjustment sets, their weights before and after pruning, the exact variable names and LLM prompt, and matched Full/Top-1/uniform-prior/no-pruning results? This existing trace would determine whether the gain is attributable to graph-aware aggregation, semantic leakage, BIC selection, or the downstream learner.

Reviewer_ZSQHOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

The paper proposes CausalGuard, a framework for conformal inference on treatment effects under uncertainty about the underlying causal graph. The method aggregates multiple causal graphs based on LLM-elicited priors and independence screening and then performs the conformal calibration step on aggregated outcomes. The paper establishes marginal predictive coverage for the aggregated pseudo-outcome and conditional-mean consistency for the CATE under additional assumptions.

Critical strengths
  • The paper addresses an important problem in that predictive calibration alone does not ensure causal validity when the adjustment strategy is misspecified.
Critical weaknesses
  • The provided guarantees concerns pseudo-outcome coverage, not CATE coverage.
  • Importantly, the guarantees do not transfer to an interventional distribution (as would be necessary in causal settings), as calibration and test data are assumed to be exchangeable
  • The structural (causal) identification is only assumed. The BIC separation does not hold for observationally equivalent graphs. Similarly, the mere inclusion of candidate graphs does not ensure causal validity. As the paper was motivated by the problem of defining the correct udnerlying causal graph, the provided solution appears insufficient. Overall, I see this as the strongest weakness.
  • The paper overstates the contributions (e.g., flawless protection should be stated with regard to predictive not causal guarantees, pseudo-outcome–CATE discrepancy conflates structural error with outcome noise and nuisance-estimation error)
Questions for authors
  • Is the intended inferential object a prediction interval for the aggregated pseudo-outcome or a confidence interval for the CATE? What additional result would justify CATE coverage for the reported interval?
  • Under what concrete data-generating assumptions does the implemented BIC score satisfy Assumption 4? What happens when observationally equivalent candidates have different causal implications?
  • How are the operating levels selected on the tuning split? Are true CATE labels used, and, if so, how would this selection be performed in observational applications without such labels?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.