acceptodds
Under review as a conference paper at ICLR 2027

Learning Probabilities of Causation with Mask-Augmented Data

Abstract

Probabilities of causation play a central role in modern decision making. Tian and Pearl first introduced formal definitions and derived tight bounds for three binary probabilities of causation (PoCs), such as the probability of necessity and sufficiency (PNS). However, estimating these probabilities requires both experimental and observational distributions specific to each subpopulation, which are often unreliable or impractical to obtain from limited population-level data. To solve this problem, we propose a machine learning (ML) framework that learns from a small set of reliable subpopulations and predicts PNS bounds for all subpopulations. We further introduce a mask-based augmentation strategy that augments the data and enables prediction of masked subpopulation queries. In experiments across five ML models, four Structural Causal Models (SCMs), and four data budgets, mask-augmented neural models consistently achieve the lowest average errors. Mask-Transformer and Mask-MLP obtain average MAEs of 0.0290 and 0.0341 on masked queries, compared with 0.0985 for the plug-in baseline, and also improve exact-query MAE from 0.1866 to 0.0469 and 0.0538. These results demonstrate both the feasibility of ML models for learning PoCs and the effectiveness of mask-based augmentation.

Primary area:
probabilistic methods (Bayesian methods, variational inference, sampling, UQ, etc.)
Keywords:
Probabilities of Causation, Structural Causal Models, Counterfactuals, Causal Machine Learning

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_s965OpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper studies how to estimate Tian–Pearl bounds on PNS when subpopulations have insufficient experimental and observational data. The authors use the existing formulas to compute bounds for well-supported subpopulations, then train regression models to predict bounds for other subpopulations. Mask augmentation extends the query representation to include subpopulations with unspecified covariate values. Evaluations on synthetic data and the Hillstrom randomized email experiment show that the framework improves bound prediction accuracy.

Critical strengths

1. The paper addresses a clear and practically relevant problem: fine-grained subpopulations often have too few samples to reliably estimate bounds on PNS.

2. The mask-augmentation strategy is straightforward and easy to implement.

3. The experiments are reasonably comprehensive, covering five regressors, four SCMs, and multiple data budgets.

Critical weaknesses

1. The experiments lack comparisons with recent relevant baselines.

2. The framework mainly uses the existing Tian–Pearl formulas to generate training labels and regression models to predict PNS bounds. The main addition is mask augmentation, but its methodological novelty relative to existing approaches remains unclear.

3. The paper aims to address data scarcity, but the Hillstrom test queries all contain relatively large numbers of customers and therefore mainly evaluate well-supported subpopulations. The authors acknowledge this limitation, but it still limits the evidence supporting their claims about data-scarce subpopulations.

4. Algorithm 1 reports errors over the full query sets without explicitly stating whether training and validation queries are excluded from evaluation.

Questions for authors

1. Why should patterns learned from well-supported subpopulations generalize to data-scarce subpopulations?

2. How are queries split into training, validation, and test sets in the synthetic experiments? Please clarify this in the experimental setup.

3. Please include additional relevant baselines. If they are not directly comparable, please explain the differences in assumptions or estimation targets.

4. Please provide the appendix referenced in the paper, including the SCM equations, model configurations, and other implementation details.

1 comment

Thank you very much for your thoughtful review and conductive advices for our paper. We appreciate the opportunity to clarify several points raised in your comments.

Regarding your specific questions and concerns:

  1. W1&Q3 We also tried to find a suitable baseline. However, as discussed in our Related Work, existing PNS research mainly focuses on theoretically deriving tighter bounds, usually assuming access to oracle distributions, or on using PNS to support machine learning. Closely related work uses ML to estimate conditional causal effects such as ATE. However, ATE and PNS belong to different levels of Pearl’s causal hierarchy, so their estimates cannot be directly compared, as also discussed in our Related Work. We have not found a directly applicable method for estimating PNS bounds for subgroups with no data. Therefore, we use plug-in estimation and other ML models as baselines.

  2. W2 We agree that the ML models themselves are standard. Our primary goal is to investigate the learnability of PNS bounds, a counterfactual defined in Structural Causal Models that is rarely served as a target function in ML. Our main contributions are therefore the formulation of the ML pipeline for learning PNS bounds and the proposed mask augmentation.

  3. W3 We agree that this is a limitation, as we mentioned in the paper. The Hillstrom experiment provides an initial real-world evaluation but still suffers from selection bias. To address this issue, as mentioned in the conclusion, an external audit could be conducted, although it would be expensive. Conducting such an audit is beyond the scope of the current study and remains an important direction for future work.

  4. W4 Thank you for the question about training and validation queries exclusion. We keep all queries for evaluation because the labels for evaluation (oracle) and training are different. Therefore, we still need to check whether the model can make accurate predictions on those seen queries. In addition, even if we exclude all seen queries, the MAE increases by about 15%, as shown in the table below, while our conclusion remains unchanged.

    ModelExact: OriginalExact: Held-outMask: OriginalMask: Held-out
    MLP0.05380.06010.03410.0448
    Transformer0.04690.04780.02900.0335
    RF0.07760.07960.05170.0632
    GBDT0.07430.07560.05070.0587
    SVM0.09380.09460.06640.0726
  5. Q1 At a high-level, individuals within the same SCM share the same structural equations. These shared mechanisms can introduce learnable structure into the mapping from covariate configurations to PNS bound endpoints. By learning this mapping from well-supported subpopulations, a neural network may also predict bounds for data-scarce subpopulations.

  6. Q2&Q4 Thank you for raising this point. We apologize that we did not make this clear enough in the main paper. The appendix is included in the supplementary material, and Section B provides detailed information on the train-validation-test split, SCM equations, and model configurations.

Reviewer_LNy7OpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes methods for estimating bounds on the probabilities of necessity and sufficiency in situations in which conventional methods do not apply.

Critical strengths

The authors show empirically that ML approaches can provide more accurate bounds on PNS for populations for which there exists insufficient data to use alternative estimators. The approach appears to be innovative and the improvements over the plug-in baseline are large.

Critical weaknesses

The authors do not provide sufficient context for the goal of estimating bounds on PNS. Specifically, they don’t contrast this estimate with a seemingly sensible alternative: estimating a posterior distribution on outcome given an intervention on treatment. What are the circumstances under which PNS is the preferred estimand for practitioners?

Relatively little intution is provided for why the machine learning estimators work well. What is it that the ML estimators are doing that allows improved bounds on PNS? What’s the computational mechanism by which the estimates are improved?

In the title, abstract, and introduction, the authors use terms such as “mask-based augmentation strategy”, “masked subpopulation queries”, and “mask-augmented models” without defining what these terms mean. No indication of meaning is given until the first paragraph of p. 4.

The AI Use Statement on p. 10 is incomplete and still contains the original template text.

Questions for authors

What’s the case for estimating bounds on the PNS rather than just estimating a posterior distribution on the effects of intervention?

At a high-level, what is the intuition for why machine learning approaches work well on this task?

1 comment

Thank you very much for your thoughtful review and for supporting our paper. We appreciate the opportunity to clarify several points raised in your comments.

Regarding your specific questions and concerns:

  1. W1&Q1 Thank you for requesting this clarification. We didn't provide sufficient context due to page limitation. And we now explain more from line 49 to 53.
  2. W2&Q2 Our goal is not to tighten the theoretical PNS bounds, but to estimate them when data for particular subgroups are limited or unavailable. We use ML to learn mappings from available data to approximate the relationship between covariates and PNS bounds, aiming to predict bounds for subgroups without data and improve accuracy over plug-in estimation when data are sparse.
  3. W3 We agree an earlier definition of mask queries can be better, which is also mentioned by Reviewer FmhC (Actually, we deleted it due to page limitation). Thereby, we add a simple definition from line 65 to 68.
  4. W4. Thank you for pointing this out. We have corrected the AI Use Statement and removed the remaining template text.
Reviewer_FmhCOpenReview ↗
Rating
2
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

The proposed method uses PNS bounds as training labels for subpopulations and applies machine learning methods to predict bounds for all subpopulations. The authors obtain PNS bounds from observational and interventional data. The authors also show empirical performance on multiple synthetic and real-world datasets.

Critical strengths

I really appreciate the human touch in the writing of the paper and thank the authors for using less AI. In this era of AI incomprehensible text in conference submissions, I value the author’s original writing effort towards the paper quality. The paper is written in very intuitive manner. Figure 1, helps to understand the full pipeline. The authors provided intuitions or examples time to time which helps to understand the approach. I appreciate the authors extensive experiments on synthetic and real-world datasets.

Critical weaknesses

Below I share my concerns:

  1. What is the guarantee of the correctness of the bounds predicted by the proposed approach? For example, how do the authors guarantee that the true value is not outside the predicted bound? Otherwise, the bound will be meaningless (by definition it should contain the true value).
  2. To my understanding, if two query with similar covariate values such as [1,1,1,1,0] and [1,1,1,1,1] will have similar predicted bounds. What if the last covariate is something important like age and play significant role in the PNS? Suppose, we are predicting the effectiveness of a treatment on recovery which works only for adult (age=1). The proposed approach will predict similar bounds for both adult and child. However, this could be an incorrect prediction leading to a very wrong decision.
  3. How does this approach generalize to arbitrary dataset? For any unseen dataset, we do not have access to their true SCMs. How does the pipeline work?
  4. The set of SCMs used for evaluation is very limited. Larger SCMs (for example, w/ multiple mediators) should be used.
  5. How does the approach work for SCMs where we dont have closed form bounds for PNS?
  6. The authors should define exact and mask queries earlier in the paper since they used these terms multiple times.
  7. [Minor] Line 240, we aim.
  8. The midpoint projection appears to be a heuristic. Whats the justification for that?
Questions for authors

Below I share my questions:

  1. How large the number of covariates, d can be?
  2. How does the authors deal with positivity violation?
  3. Whats the role of the threshold tau?
3 comments

Thank you very much for your thoughtful review and recognition of our human touch efforts. We appreciate the opportunity to clarify several points raised in your comments.

Regarding your specific questions and concerns:

  1. W1 Thank you for raising this concern about the validity of the bounds. Even the true value is outside, its distance from the bounds is at most the corresponding endpoint’s AE, with equality only when PNS equals that theoretical endpoint (This is also the reason why we use MAE to represent the error). In addition, without the oracle distribution, we can never guarantee the coverage. For example, in point-identified cases (Direct scenario), the theoretical bounds collapse to a single point. Hence, minor error will lead to outside.
  2. W2 Our model can distingue the situation as your concern about similar covariates. To exemplify, in our existing Covariate experiment, queries 0010111110 and 1010111110 differ in only one coordinate, yet their theoretical bounds are [0.3236, 0.6604] and [0.0576, 0.3271], respectively. At the 500k budget, the Transformer predicts [0.3296, 0.5980] and [0.0822, 0.3309].
  3. W3 Our approach can be directly applied to a new dataset. As described in Algorithm 1 and illustrated by the example of the Hillstrom dataset, the training process does not require any prior knowledge of SCMs. In our synthetic experiments, the known SCMs and causal equations are only used for data generation and oracle evaluation.
  4. W4 Thank you for pointing out larger SCMs. However, in our current work, the covariates are multidimensional, which means our SCMs already have at least 22 nodes to represent the necessary information. More sophisticated SCMs could be researched in future work.
  5. W5 Our approach requires computable bound labels, but not necessarily closed-form expressions for them. Moreover, closed-form PNS bounds can always be obtained for finite discrete treatment and outcome variables, as shown in "Identification of Probabilities of Causation: from Recursive to Closed-Form Bounds".
  6. W6 We agree an earlier definition of exact and mask queries can be better (Actually, we deleted it due to page limitation). Thereby, we add a simple definition from line 65 to 68.
  7. W7 Good catch for the typo! We have correct this.
  8. W8 Midpoint projection is the exact Euclidean projection onto the order constraint set C={(l,u):l≤u}C=\{(l,u):l\le u\}. For any ordered reference bounds, this operation won't increase either the combined squared endpoint error or the joint MAE.
  9. Q1 We evaluate up to d=10d=10 observed covariates. In addition, we expect increasing dd to challenge data availability more than model capacity, as the exponentially growing query space requires more data to maintain sufficient subgroup support.
  10. Q2&Q3 The threshold τ\tau is used to select groups for training. To be more specific, only groups with more than τ\tau samples can be regarded as reliable for training. Thereby, we can ensure sufficient data for training and solve practical positivity violations.

I thank the reviewer for their rebuttal response. However, I am yet not convinced with the response for my concerns in W1, W2 ,W4 and W5.

W1: The authors mentioned that the true value can be outside the predicted bound. But then the term bound becomes meaningless since, to my understanding, by definition it should contain the true value. The approximated bound could be wider but should contain the true value.

W2: The authors gave a specific example but that does guarantee that the case I mentioned will always be addressed. The approach might consider two covariate combination similar due to having almost same values. But just difference in one index might represent significant difference.

W4: The authors pointed out that the covariates are multidimensional. However, I was referring to mediators for example Z1,Z2 in: X->Z1 ->Z2 ->Y + X<-Z3->Y

W5: Can we always get closed-form PNS bounds for setups when we have more variables than just treatment, covariates and outcomes?

I would request the authors to clarify the above points.

Thank you for your response. We are glad to clarify more on W1, W2 ,W4 and W5:

  1. W1 As mentioned earlier, we do not claim 100% coverage. However, from a decision-making perspective, accurate estimates of PNS bounds can still provide useful information even without guaranteed coverage. In addition, we are not proposing new or tighter theoretical bounds, but rather aim to approximate the established Tian–Pearl bounds when subgroup data are sparse or even unavailable, aiming to improve accuracy over direct plug-in estimation. Furthermore, the table below reports our bound expansion and coverage experiments using MLP and Transformer with 200k budgets, across four SCMs and five training seeds. Even when ignoring the subgroups for which plug-in estimates are unavailable (that is, only compare on shared groups), both models require substantially smaller bound expansions, on average, to achieve the same coverage as plug-in estimation in both queries. Therefore, we think our model can provide more meaningful PNS bounds for decision-making than plug-in estimation.

    QueryCoverageMLP (full)Transformer (full)MLP (shared)Transformer (shared)Plug-in (shared)
    Exact95%0.08050.07390.06060.05740.4877
    Exact99%0.13310.13830.10390.11530.7377
    Exact100%0.28160.23530.23900.22920.9688
    Mask (including exact)95%0.04500.03610.03800.02990.2893
    Mask (including exact)99%0.09040.08130.07550.06770.6149
    Mask (including exact)100%0.28270.23550.24460.23280.9809
  2. W2 We agree that a single example is insufficient. However, we still claim that covariate similarity does not imply similarity in our predictions, because the model allow a single covariate to strongly affect the output. For example, f(z)=a+bzjf(z)=a+bz_j changes by bb when only zjz_j changes, regardless of how many other covariates match. We therefore evaluated all exact-query pairs under 200k budget that differing in one covariate, with an oracle LB or UB difference of at least 0.1, 0.2, and 0.3, respectively. In the table, Direction accuracy means the bound changed to correct direction, while Bound MAE is the MAEs of selected queries. These results show that the models can distinguish subgroups with substantially different bounds despite differing in only one covariate.

    Oracle differenceModelDirection accuracyAvg. bound MAE
    ≥0.1MLP95.44%0.0470
    ≥0.1Transformer94.98%0.0478
    ≥0.2MLP98.42%0.0500
    ≥0.2Transformer97.70%0.0498
    ≥0.3MLP99.56%0.0513
    ≥0.3Transformer97.77%0.0488
  3. W4 We agree that serial mediators are valuable. Thereby, we add an additional experiment with ten serial mediators, and retaining the direct X→YX \to Y path (200k budgets, 5 training seeds). In this setting, the MAEs changed little, which means our model can deal with multiple mediators.

    ModelTest queries1 mediator10 mediatorsChange
    MLPExact0.06050.0625+3.3%
    TransformerExact0.05290.0587+11.0%
    MLPAll0.03540.0364+2.8%
    TransformerAll0.03290.0365+11.0%
  4. W5 Yes, for any discrete PoCs, once we have oracle obs and exp distributions, we can get closed-form bounds. If we have more causal knowledge such as covariates, we can further tighten the bounds.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.