acceptodds
Under review as a conference paper at ICLR 2027

ConfoundingSHAP: Quantifying confounding strength in causal inference

Abstract

In causal inference, confounders are variables that influence both treatment decisions and outcomes. However, unlike as in randomized clinical trials, the treatment assignment mechanism in observational studies is not known, and it is thus unclear which covariates act as confounders. Here, we aim to generate insight for causal inference and answer: which of the observed covariates act as confounders? We introduce ConfoundingSHAP, a Shapley-based method for attributing confounding strength to individual covariates. Our contributions are twofold. First, we propose a Shapley game targeted to infer the confounding strength of the covariates. Our resulting Shapley values differ from the standard applications of SHAP explanations on causal targets, such as understanding treatment effect heterogeneity, which are ill-suited for our task. Second, as our task requires evaluating the value function over many adjustment sets, we provide a scalable TabPFN-based estimation that avoids exhaustive refitting. We demonstrate the practical value across various datasets, where ConfoundingSHAP provides informative explanations of which observed covariates drive confounding and thereby helps to provide more insight for causal inference in practice.

Primary area:
interpretability and explainable AI
Keywords:
SHAP, feature attribution, explainable AI, causal inference, confounding

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_ePpqOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper asks an important question: Which observed covariates are responsible for the confounding bias in an observational causal analysis? The authors argue that conventional SHAP applied to propensity-score models, outcome models, or CATE models does not directly answer this question. For example, a variable can be highly predictive of treatment but be an instrument rather than a confounder, while a genuine confounder can have little importance for CATE heterogeneity because its effects on the two potential outcomes cancel. I consider the conceptual contribution novel and meaningful.

Critical strengths
  1. Clear distinction between different notions of feature importance. This is much stronger than simply applying standard SHAP to a causal ML model and claiming that the result explains confounding.
  2. The Shapley formulation is mathematically elegant. Because the full adjustment set has zero residual bias, the Shapley values decompose the bias associated with moving from no adjustment to full adjustment. This gives the attribution a principled accounting interpretation. In particular, the paper is not merely taking an arbitrary predictive score and putting SHAP on top of it. The authors deliberately define the causal quantity first, then use Shapley decomposition to explain that quantity. This is a methodological strength.\
  3. The distinction from nuisance-model SHAP is convincing. The paper derives an alternative expression showing that residual bias depends on conditional covariance between treatment assignment and potential-outcome regressions. This is useful because it explains why independently explaining the propensity model or outcome model cannot reproduce the proposed attribution.
Critical weaknesses
  1. The paper may be overclaiming what it measures. This is my biggest concern. The method does not directly identify “confounders” in the causal-graph sense. What it actually measures is closer to the contribution of observed covariates to residual bias relative to the selected full covariate set. The authors themselves acknowledge this in the limitations: if important confounders are unobserved, the attribution is only relative to the observed reference set.
  2. Dependence on the full adjustment set is fundamental. The entire method assumes that the full XX is a valid adjustment set. The paper explicitly defines the full-covariate CATE as the reference causal quantity. But this creates a potentially serious problem. It is bias relative to an estimated reference adjustment strategy.
  3. Shapley attribution does not equal causal responsibility. Shapley values are excellent for decomposing a cooperative game, but the resulting phi_j should not automatically be interpreted as the causal contribution of X_j to confounding. Consider highly correlated variables: X_1 \approx X_2. Both may represent the same underlying latent factor. Shapley symmetry will tend to distribute attribution between them. But this does not mean that each variable independently causes the confounding. It means that the two variables share the ability to reduce the bias under the chosen game.
Questions for authors
  1. What happens if the full covariate set contains a collider?
  2. How should proxy confounders be interpreted?
  3. Why use signed bias as the primary notion of “strength”?
3 comments

Thank you for the thoughtful feedback and for finding our conceptual contribution novel and meaningful and our Shapley formulation principled. We addressed all of your points (changes in blue\color{blue}{\text{blue}}). Your comments also motivated us to extend ConfoundingSHAP to Shapley interactions, and we now validate it against an exact oracle game.

Response to weaknesses

W1: What the method measures

We agree that this distinction is important. In our formulation, ϕj\phi_j is the average change in the signed residual bias when XjX_j is added to an adjustment set, and we now state explicitly that it is not a label for the role of XjX_j in the causal graph. We also added two results on this: covariates jointly independent of (A,X−j)(A,X_{-j}) receive exactly zero attribution, whereas an instrument can receive a nonzero one (bias amplification).

Action: We made this explicit throughout our entire revised paper. In our new Supplement C, we added the definition, the above condition, and an instrument example.

W2: Dependence on the full adjustment set

A valid reference set is the standard assumption in adjustment-based effect estimation, and our goal is to explain confounding under this assumption. With an insufficient reference set, the attributions sum to the unadjusted contrast minus the full-adjustment contrast. Sensitivity analysis can assess bias in the reference contrast (Eq. 5). At n = 5000, the end-to-end normalized error against the oracle is 0.07 for 11 covariates and 0.13 for 25.

Action: For space reasons, the connection to sensitivity analysis was previously in the appendix. We moved it to the main paper (see Sec. 4) and added the oracle-based evaluation (see our new Supplement E).

W3: Shapley attribution and causal responsibility

We agree that ϕj\phi_j should not be read as causal responsibility, and the equal split among covariates with the same information is a property of the Shapley value itself. Your example motivated us to extend our method to pairwise Shapley interactions, which quantify how the contribution of one covariate changes once another one is adjusted for. For two noisy proxies of one latent confounder, each proxy removes 0.44 of the bias on its own but only 0.09 once the other is adjusted for. Hence, each receives 0.27, and the interaction I12=−0.34I_{12} = -0.34 shows this overlap. We are careful with the interpretation: interactions describe the dependence on the adjustment context but not where it comes from (e.g., we obtain a comparable value for two independent confounders under non-linear treatment assignment).

Action: We added the extension and its interpretation in our new Supplement C and refer to it in Sec. 4.

Response to questions

Q1: Collider in the full set

A reference set containing a collider can remain valid if it blocks all relevant noncausal paths. In our M-structure example, the collider receives −β/3-\beta/3, where β\beta is the average bias from adjusting for the collider alone. If the collider path remains open under full adjustment, the reference contrast can be biased, and the attributions are still defined relative to that contrast.

Action: We added both cases with an example (in our new Supplement C) and state the assumptions explicitly in Sec. 3.

Q2: Proxy confounders

A proxy receives credit for the bias that is removed by adjusting for it, and several proxies of the same confounder share this credit. The credit becomes smaller when the proxies are noisier: each proxy receives 0.30 for exact copies and 0.27 when 5% of the values are misrecorded. The bias that imperfect proxies cannot remove is not attributed and is left to sensitivity analysis (see Sec. 4).

Action: We added this example (see our new Supplement C).

Q3: Signed bias

The signed bias keeps the direction: the global marginal contribution is the adjusted contrast before adding XjX_j minus the adjusted contrast after adding it. Hence, ϕj\phi_j shows how strongly and in which direction omitting XjX_j shifts the estimate, and the attributions sum up to the signed confounding bias. Taking absolute values of stratum-level biases before averaging prevents cancellation between strata and discards direction. Cancellation can still occur within strata or across coalitions.

Action: We added the motivation in Supplement C.

Thank you again for the careful reading, which helped us to state more precisely what our attributions measure.

I have read the rebuttals by the authors, the authors have presented substantial refinement for their paper, and addressed several of my concerns. after careful consideration, I will keep my score unchanged.

Thank you for taking the time to read our rebuttal and for acknowledging the improvements to our paper. Please feel free to share any concerns that remain open, and we will be happy to address them.

Reviewer_wovLOpenReview ↗
Rating
2
Confidence
1
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

ConfoundingSHAP attributes residual confounding bias under partial adjustment to individual covariates using Shapley values, providing local and global explanations. Assuming that the full observed covariate set is sufficient for adjustment, it combines TabPFN with existing Shapley approximators and evaluates the framework on synthetic, semi-synthetic, and real data.

Critical strengths
  1. The paper clearly explains the difference between attributing confounding bias and explaining treatment-effect heterogeneity. It shows why explaining a CATE model may miss important confounders. This is a useful practical question.

  2. The method compares observational and causal contrasts under the same conditioning variables. It also measures each covariate’s contribution across different adjustment sets. This gives the attribution a clear definition.

  3. The evaluation includes several data types and tests approximation accuracy, stability, and computational cost. The authors also explain the assumptions and the limits of interpreting the results.

Critical weaknesses
  1. The problem is practically relevant, but the method assumes that the observed variables are sufficient to control confounding. I find this assumption strong for real observational studies. It is unclear how useful the rankings remain when important confounders are missing.

  2. TabPFN avoids retraining for each adjustment set, but its practical advantage over other regressors is not directly tested. Avoiding retraining alone does not show that the full procedure is faster or produces more accurate attribution scores.

  3. The method estimates confounding bias through differences between fitted quantities. These differences may also reflect model errors. The randomized-treatment experiment provides some reassurance, but inaccurate models could still produce misleading confounding scores.

Questions for authors
  1. What checks or supporting evidence can help assess whether the observed variables are sufficient to control confounding in the applications studied?

  2. How do total runtime and attribution accuracy change when TabPFN is replaced with another regressor?

  3. How sensitive are the attribution scores to errors in the fitted models? Could these errors lead to misleading rankings even when the causal assumptions hold?

3 comments

Thank you for the helpful feedback and for acknowledging that our paper clearly separates the attribution of confounding bias from explaining treatment effect heterogeneity. We incorporated your suggestions (changes in blue\color{blue}{\text{blue}}).

W1 & Q1: Sufficiency of the observed covariates

Sufficiency is the standard identification assumption for adjustment-based ATE and CATE estimation, and it cannot be tested from data alone. In our experiments, the assumption holds by construction (synthetic and semi-synthetic data) or by randomization (ACTG trial), and SUPPORT is a qualitative illustration relative to the selected covariates. In practice, we recommend justifying the covariate set with domain knowledge or a causal diagram and complementing our method with sensitivity analysis. Both fit together: ConfoundingSHAP decomposes the unadjusted contrast minus the full-adjustment contrast, and sensitivity analysis can assess how far the full-adjustment contrast is from the true ATE (see Eq. 5). We added an example with two imperfect proxies of an unobserved confounder, where our attributions account for 0.534 of a total bias of 0.600 (see Fig. 8b).

Action: Due to space constraints, the connection to sensitivity analysis was previously only in the appendix. We have now moved it to the main paper (see Sec. 4). We also state the assumption more explicitly in Sec. 3 and added the proxy example in our new Supplement C.

W2 & Q2: Comparison with other regressors

We followed your suggestion (see our new Supplement E). Importantly, ConfoundingSHAP is defined by its game and not by the regressor. Hence, it is still ConfoundingSHAP when we use another regression backbone such as XGBoost, and TabPFN is our default choice. We ran ConfoundingSHAP with XGBoost and random forests, each with default hyperparameters and tuned once, and compared all backbones on identical coalitions against the exact oracle attributions. The results for 25 covariates and n = 5000 are (see Table 2, Fig. 10c):

BackboneNormalized errorTime per run
TabPFN (our default)0.13≈ 2400 s (one shared GPU)
XGBoost (tuned)0.15≈ 1300 s (4 CPU threads)
XGBoost (default)0.31≈ 240 s (4 CPU threads)

We find that TabPFN has the lowest attribution error in most settings, and the margin is larger for smaller sample sizes (e.g., 0.16 vs. 0.23 for the tuned XGBoost at n = 3000). XGBoost comes close only after tuning, while XGBoost with default hyperparameters and random forests (error of at least 0.58 for 25 covariates) are clearly less accurate. This confirms our default choice, as TabPFN is accurate without any per-coalition training or tuning. XGBoost ran faster on the tested hardware (Fig. 10c). The computational savings of our method come from the Shapley approximation. Thank you for bringing this up. We revised the main text to make clear that ConfoundingSHAP is agnostic to the regression backbone. If runtime is the priority, a tuned XGBoost is a competitive alternative (see Supplement E.4).

Action: We added the backbone comparison in our new Supplement E and guidance on how to choose the backbone. We now clarify that ConfoundingSHAP can be used with any regression backbone and that TabPFN is our default.

W3 & Q3: Sensitivity to model errors

Our oracle evaluation (Supplement E.3) measures end-to-end attribution error when the causal assumptions hold. For TabPFN, normalized error fell from 0.16 to 0.07 for 11 covariates and from 0.25 to 0.13 for 25 as n increased from 1000 to 5000. Some backbones underestimated the confounders’ attributions (Fig. 10a). At n=5000, confounders had the highest mean attribution by role for every tested backbone. Small attributions should not be interpreted individually: covariates with an oracle attribution of zero receive estimates of about 0.03 on average and up to 0.11, compared to 0.90 for the confounders (11 covariates, n = 5000). As the prediction accuracy is observable in practice, alternatives to our default TabPFN can be compared by their cross-validated prediction error.

Action: We added a new oracle-based evaluation (Supplement E.1–E.3), the above recommendation (Supplement E.4), and a note on small attributions in the main paper (see Sec. 4).

Thank you again for your review, which helped us to make the assumptions and the practical use of our method clearer!

Thank you for taking the time to respond to each of my questions in detail. Your responses have helped clarify some of my concerns, and I appreciate the additional explanations. After considering the responses, I will keep my score unchanged.

Thank you for your reply and for engaging with our responses. Please feel free to let us know if any of your concerns remain open, and we will be happy to clarify them further.

Reviewer_Lk6cOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes ConfoundingSHAP, a Shapley decomposition of the signed bias induced by restricting the adjustment set. For each covariate coalition, it compares the observational treated–control contrast with the full-adjustment CATE projected onto the same covariates. Local and population games then attribute changes in this bias to individual variables. Coalition-specific regressions are estimated with TabPFN, and budgeted Shapley estimation reduces the number of coalition evaluations. Evaluation includes synthetic data with designated variable roles, randomized and semi-synthetic ACTG data, a qualitative SUPPORT application, and an ACIC feature-removal experiment. The intended use is diagnostic attribution relative to an assumed sufficient observed adjustment set.

Critical strengths
  1. The explanation target is well motivated and meaningfully different from SHAP on a CATE predictor. Comparing observational and causal contrasts at the same information level separates confounding-related adjustment changes from treatment-effect heterogeneity. The resulting efficiency identity makes the signed decomposition interpretable.

  2. The empirical checks address several complementary aspects of this target: known-role simulations, disappearance of attribution under randomized treatment, and downstream feature removal on ACIC. The explicit discussion of cancellation and the limits of the full-covariate reference helps establish a useful diagnostic interpretation.

Critical weaknesses
  1. Nonzero attribution does not generally identify a confounder, and instruments need not be null players. Consider independent C,Z ~ Bernoulli(1/2), Y(a)=C+a+ε with independent mean-zero ε, and treatment probabilities 0.1, 0.6, 0.4, 0.9 at (C,Z)=(0,0),(0,1),(1,0),(1,1). The full set is valid, Z is a pure instrument, and the global biases are b_empty=0.3, E[b_Z]=0.4, and E[b_C]=E[b_CZ]=0. The proposed game therefore gives φ_Z=-0.05 and φ_C=0.35. This is a valid attribution of adjustment effects, but it contradicts a general interpretation as identifying causal variable roles; absolute-value rankings also count the instrument. The claims about recovering confounders and guiding minimal adjustment sets should be narrowed accordingly.

  2. The experiments do not sufficiently isolate nuisance-estimation error from Shapley approximation error. Appendix F uses plug-in predictions on the context data and leaves cross-fitting as a possible variant. Most role-recovery simulations use independent, specially structured covariates; the jointVIP comparison explicitly uses raw variables in a quadratic DGP unfavorable to its linear score. Oracle-functional attribution error, correlated/proxy-variable stress tests, and a matched regression-backbone comparison would better establish the reliability and computational advantage of the proposed estimator.

Questions for authors
  1. Do you agree that the instrument example above is within the stated assumptions? Can you distinguish “contribution to signed adjustment bias” from “being a confounder,” and state conditions under which variable-role recovery is justified? A population calculation on a correlated/proxy-variable example would also clarify the intended interpretation.

  2. Can you report attribution error against the oracle coalition game with and without cross-fitting, together with a matched-budget comparison against another regression backbone? This would determine whether the observed recovery and runtime gains come from the proposed estimation scheme rather than the favorable DGP or coalition sampler.

1 comment

Thank you for the constructive feedback and for finding our explanation target well motivated. We followed your suggestions closely (see the changes in blue\color{blue}{\text{blue}}). In addition, we now validate ConfoundingSHAP against an exact oracle game and extend it to Shapley interactions.

W1 & Q1: Instruments and interpretation

Yes, the example is within our assumptions, and our method handles it as intended. We obtain ϕC=0.35\phi_C = 0.35 and ϕZ=−0.05\phi_Z = -0.05 (see Fig. 8a). The negative value for ZZ is the known bias amplification by instruments [1]: adjusting for ZZ increases the bias by 0.10 when CC is omitted and has no effect once CC is adjusted for. Hence, the attribution correctly shows how ZZ changes the adjustment bias, including the direction, which is different from its causal role. We now state this distinction explicitly.

For role recovery, we show that XjX_j receives exactly zero global attribution if it is jointly independent of (A,X−j)(A,X_{-j}). This helps explain why attributions concentrate on the confounders in our synthetic benchmarks. We further added exact population calculations for proxies and correlated covariates. Finally, we extended ConfoundingSHAP to pairwise Shapley interactions, which show how the contribution of a covariate depends on what else is adjusted for. In your example, we obtain ICZ=0.10I_{CZ} = 0.10, i.e., ZZ only matters as long as CC is omitted.

Action: We sharpened the wording on confounder identification and adjustment-set guidance throughout the manuscript. We added a new Supplement C with your example, the sufficient condition, the population calculations, and the extension to interactions.

W2 & Q2: Oracle error, cross-fitting, and other backbones

We followed all of these suggestions (see our new Supplement E). We derived the exact oracle game for our synthetic DGP and find that ConfoundingSHAP recovers the oracle attributions with small error: at n = 5000, the normalized error is 0.07 for 11 and 0.13 for 25 covariates. Importantly, ConfoundingSHAP is defined by its game and not by the regressor. Hence, it is still ConfoundingSHAP when we use another regression backbone, and TabPFN is our default choice. We also ran ConfoundingSHAP with XGBoost and random forests, on identical coalitions, with and without cross-fitting.

To separate the error of the Shapley approximation, we also ran the approximator on the exact oracle coalition values ("approximation only"). The results for 25 covariates (B = 1024, no cross-fitting) are in our new Table 2:

Backbonen = 1000n = 3000n = 5000
Oracle values (approximation only)0.060.060.06
TabPFN0.250.160.13
XGBoost (tuned)0.310.230.15
XGBoost (default)0.530.420.31
Random forest (tuned)0.590.610.58
Random forest (default)0.660.640.61

The approximation-only error is below 0.005 for 11 covariates (B=128) and about 0.06 for 25 (B=1024). TabPFN gave similar results with cross-fitting. For 11 covariates and n=5000, the error was 0.073 without cross-fitting, 0.076 with K=2, and 0.072 with K=5 (Fig. 10b). Regarding other backbones, TabPFN has the lowest error in most settings, which confirms our default choice. XGBoost comes close only after tuning (on par for 11 covariates and n = 5000) and is clearly less accurate with default hyperparameters. XGBoost ran faster on the tested hardware (Fig. 10c). Hence, the benefit of TabPFN is accuracy without per-coalition training or tuning, while the computational savings of our method come from the Shapley approximation. If runtime is the priority, a tuned XGBoost is a competitive backbone with lower compute (see new Supplement E.4). Thank you for bringing this up.

For jointVIP, we added a second input representation and a linear DGP. In our experiments, jointVIP works well when its input representation matches the DGP but misses confounders otherwise. ConfoundingSHAP ranked the designated confounders highest using raw inputs on both benchmarks.

Action: We added a new Supplement E with the oracle game (E.1), the setup with cross-fitting and tuning protocol (E.2), the results (E.3, Fig. 10, Table 2), and guidance on how to choose the backbone (E.4), with a summary in Sec. 5.1 (d). We now clarify that ConfoundingSHAP can be used with any regression backbone and that TabPFN is our default (see Abstract, Sec. 1, and "Choice of regression backbone" in Sec. 4). We extended the jointVIP comparison in Supplement D (Fig. 9).

Thank you again for the detailed and constructive review. We are confident that our paper has improved substantially as a result.

[1] Pearl J. On a class of bias-amplifying variables that endanger effect estimates. UAI, 2010.

Comments on the paper

1 comment
AuthorsOpenReview ↗

General response

We thank all reviewers for their thoughtful reviews and actionable suggestions! We appreciate their recognition of the motivation and technical contributions of our work.

We have addressed all weaknesses and questions in our individual responses and made the following main improvements to our revised paper (highlighted in blue\color{blue}{\text{blue}} in the PDF):

  1. Clearer interpretation and scope. We spelled out more clearly what ConfoundingSHAP measures in the motivation of our paper. We also followed suggestions from the reviewers and brought the connection to sensitivity analysis into the main paper (Sec. 4).
  2. Examples. We added a new Supplement C to provide examples of ConfoundingSHAP for instruments, proxies, correlated covariates, and colliders. We also added pairwise Shapley interactions (new Fig. 8).
  3. Oracle validation and cross-fitting. We added a new Supplement E to derive the population oracle and report attribution error with and without cross-fitting. We also measure Shapley approximation error on oracle coalition values.
  4. Broader backbone and baseline comparisons. We compared TabPFN with default and tuned XGBoost and random forests on identical sampled coalitions, reporting both attribution error and runtime in our new Supplement E. We also extended the jointVIP comparison to raw and squared inputs on quadratic and linear benchmarks (Supplement D).

Our revisions, thanks to your helpful feedback, clarify the interpretation of ConfoundingSHAP and strengthen the empirical validation. We believe our revised paper makes a contribution to explainability in causal machine learning and is a strong fit for ICLR 2027.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.