acceptodds
Under review as a conference paper at ICLR 2027

Causal EpiNets: Scalable Inference for Bounds on the Probability of Positive Individual Treatment Effects

Abstract

Does a treatment help this individual? For binary treatments and outcomes, the Probability of Necessity and Sufficiency (PNS) formalizes this question as the probability of a positive Individual Treatment Effect. In general, PNS is not point identified, but prior work shows that sharp bounds can be obtained by combining randomized and observational data without assuming observational unconfoundedness or positivity. These identification results characterize what can be learned given exact knowledge of the underlying distributions; in practice, however, PNS bounds must be estimated from finite samples. We show that naive plug-in estimation of PNS bounds suffers from two distinct finite-sample pathologies and develop a principled, scalable framework that translates the identification theory into practical estimation with high-dimensional covariates while explicitly addressing both.

Primary area:
other topics in machine learning (i.e., none of the above)
Keywords:
Causal Inference, Probability of Causation, Probability of Benefit, Partial Identification, Epistemic Neural Networks

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_vdq5OpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

The paper estimates the Tian-Pearl bounds on the conditional probability of necessity and sufficiency, PNS(z) = P(Y₁ = 1, Y₀ = 0 | Z = z), from observational and randomized data with binary treatment and outcome. A single neural network with three output heads estimates the observational probabilities P(X = x, Y = y | Z = z) and the experimental probabilities P(Y = 1 | do(X = x), Z = z). The experimental outputs are parameterized so that they always satisfy the compatibility constraints between the two data sources (Eq. 5), for example P(X = 1, Y = 1 | Z = z) ≤ P(Y = 1 | do(X = 1), Z = z) ≤ 1 − P(X = 1, Y = 0 | Z = z). The bound terms are then combined with the precision correction of Chernozhukov, Lee and Rosen (2013), with standard errors and critical values taken from Epistemic Neural Network draws.

Critical strengths

Easy to understand.

Critical weaknesses

W1. The ENN spread is not a standard error. The precision correction requires standard errors and a critical value from the sampling distribution of the term estimators. Here they come from the spread of ENN draws, which is an approximate posterior spread whose size is set by the prior scale, a tuning parameter selected by validation loss and calibration error (1.25 in the benchmarks and 2 in the case study, Appendix E). These standard errors are therefore not accurate, and the resulting correction is not theoretically sound.

W2. The coverage guarantee is for a different method. The proposed method takes its standard errors from ENN draws, but the coverage theorem (Theorem B.1, Corollary B.4) is proved for an influence-function multiplier bootstrap, which the experiments use only as a baseline (MB Full), as Section 3.2 itself states. That theorem also assumes the network is a correctly specified parametric model with an invertible Hessian, which overparameterized networks do not satisfy, so no method in the experiments has a coverage guarantee.

Missing comparisons and literature. The experiments compare only S- and T-learner plug-ins and the authors' own variants. Existing methods for the same quantities are cited but not compared: debiased and smoothed inference for bounds with guarantees (Semenova, 2023; Whitehouse et al., 2026), machine-learning estimation of probabilities of causation (Wang and Li, 2025), and bounds on the fraction harmed (Kallus, 2022). Closely related work is not cited: the nonparametric efficient estimator for the probability of causation (Cuellar and Kennedy, JRSS-A 2020), debiased estimation of covariate-assisted bounds (Levis et al., 2023), and partial counterfactual identification from observational and experimental data, including PNS (Mueller, Li and Pearl, IJCAI 2022; Zhang, Tian and Bareinboim, ICML 2022; Xia, Pan and Bareinboim, ICLR 2023).

Questions for authors
  1. Do the ENN standard errors match the standard deviation of the estimates across the 1,000 replicates?
  2. How do coverage and width change with the prior scale of the ENN?
Reviewer_HV4jOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies finite-sample estimation and inference for Tian–Pearl bounds on the probability of necessity and sufficiency (PNS) using observational and experimental data. It combines a constrained neural architecture that enforces compatibility between estimated probabilities, ENN-based uncertainty estimation, and precision correction for the max–min bound expressions. Validation includes synthetic and semi-synthetic benchmarks with known ground truth and a product-release case study.

Critical strengths
  1. A useful solution to probability compatibility. Section 3.1 uses the estimated ob- servational joint probabilities as anchors and reparameterizes the interventional probabilities within their feasible intervals in Equations (7)–(8). This enforces the Tian–Pearl compati- bility constraints by construction, including for each ENN draw. For bound estimation, this matters because independently fitted probabilities can produce internally inconsistent inputs to the bound expressions. The architecture therefore addresses a concrete statistical problem and provides a coherent starting point for combining observational and experimental data.

  2. A relevant inference formulation with encouraging low-dimensional results. The paper addresses how max–min operations can make plug-in bounds spuriously tight and connects PNS inference to precision correction for intersection bounds. In Table 1, low-dimensional identified-set coverage is 95.59% with mean width 0.3303, compared with 97.40% and 0.3695 for the full-network multiplier bootstrap. These results show a promising coverage–width trade-off in a setting where a full-bootstrap comparison is feasible. They provide concrete evidence of the potential practical value of the proposed combination.

Critical weaknesses
  1. The training procedure and epinet implementation are difficult to reconstruct. Sections 3.1 and 3.2.1 describe the losses and the general ENN framework, but the actual training procedure remains unclear. In particular, the loss weighting, the role of the ran- dom index, and the combination and training of the base network and epinet need explicit specification. These details are necessary to understand and assess the proposed method.

  2. ENN-based inference needs stronger justification and calibration evidence. Lines 327–339 on p. 7 distinguish the influence-function coverage guarantee from the em- pirically assessed ENN approximation. The connection between the actual ENN training procedure and the sampling error of the fitted estimates remains unclear. In Table 1 on p. 8, high-dimensional identified-set coverage is 86.00% [85.3%, 86.7%], below the 95% target. The 97.30% PNS point coverage does not resolve this issue. The conclusion on p. 9, lines 467–470, should make the same qualification when describing coverage as “near-nominal.” Lines 438–439 on p. 9 mention residual probability-estimation error. If the main cause of undercoverage remains unclear, this could be acknowledged as a limitation.

  3. The experimental interpretation needs clarification. Lines 398–401 on p. 8 attribute the gap between the anchored baseline and the full method to precision correction, without making clear whether training and point estimation are otherwise the same. The real-data analysis is also largely confined to Figure 4 and needs fuller presentation of estimate stability and practical interpretation. These gaps limit what can be concluded about the contribution of the correction and the usefulness of the estimated bounds. The problem is interesting, but I lean towards rejection because (1) the implementation is still hard to follow, (2) the use of ENN draws for inference needs more justification, and (3) high-dimensional identified-set coverage is well below the target. Confidence: moderate.

Questions for authors

The following are clarification and validation requests relevant to the current assessment.

  1. Training and implementation. Please provide more detail on the training objective in Section 3.1 and the ENN construction in Section 3.2.1. How are the observational and experimental losses weighted? Where does the random index enter the network, how are the base network and epinet combined with Equations (7)–(8), and how are they trained? Please specify Pζ, the index dimension, and the number of draws M.

  2. ENN calibration and undercoverage. How does the actual training procedure sup- port using ENN draws to approximate sampling error? Can the existing repetitions be used to compare ENN standard errors with the standard deviations of estimates at fixed test points? What explains the 86.00% identified-set coverage in the high-dimensional experi- ment? If the main cause remains unclear, please acknowledge this as a limitation.

  3. Ablation and influence-function baseline. Do the anchored baseline and full method use the same training procedure and point estimates apart from correction, or does ENN training or averaging over draws also differ? If ENN training or averaging over draws also differs, the authors could compare the same ENN with and without correction or qualify this attribution. Stop-gradient is used on p. 4, lines 178–181. When introducing MB Full (p. 8, lines 401– 404), please explain how the influence-function calculation handles this operation and which loss is used to form the Hessian on p. 5, lines 253–257.

  4. Validation protocol and application. Which data are used to train the S- and T-learners for μ① ? Please clarify the ACIC sampling and refitting steps, the calculation of confidence intervals and error bars, and how any intervals with a lower endpoint above the upper endpoint are handled in coverage and interval scores. The absolute width |ui _ li| alone does not readily reveal empty intervals. In the application, how stable are the reported PNS-bound differences, and what do they tell us in practice? Since true PNS is unobserved, the application checks should focus on stability and practical interpretation.

2 comments
AuthorsOpenReview ↗

Validation Protocol, Crossed Intervals, and Case-Study Stability (W3, Q4)

  • Data used to train S- and T-learners (Section 4.2, ll. 374–393; Appendix E.6, ll. 1768–1770): For both plug-in baselines, μ^0(z)\hat\mu_0(z) and μ^1(z)\hat\mu_1(z) are trained exclusively on the randomized experimental data (the S-learner fits one network with treatment XX as an input; the T-learner fits separate networks on treated and control arms). Independently, p^xy(z)\hat p_{xy}(z) is estimated by a separate four-class multinomial network trained on the observational data. Hyperparameters for the experimental and observational networks are tuned separately by grid search.
  • ACIC sampling and refitting steps (Sections 4.1, 4.3; Appendices D, E.3–E.4): In each of the 1,000 replicates, we draw new observational (nobs=2Mn_{\rm obs}=2\rm M) and experimental (nexp=1Mn_{\rm exp}=1\rm M) training samples from the fixed 3M/3M super-populations and refit every model from scratch, evaluating on a disjoint test split of ntest=2,000n_{\rm test}=2,000 units (holding tuned hyperparameters fixed across replicates).
  • Confidence intervals and error bars (Section 4.3, l. 427; Appendix G, Table 6): Each refitted replicate r∈1,..,1,000r\in\\{1,..,1,000\\} is treated as one independent unit. We first average each metric over the 2,000 test points within replicate rr to obtain qrq_r, and report the across-replicate mean qˉ=1R∑r=1Rqr\bar q=\frac{1}{R}\sum_{r=1}^R q_r with 95% Monte Carlo confidence intervals qˉ±t999,0.975sq/1,000\bar q\pm t_{999,0.975}s_q/\sqrt{1,000} in Table 1 and as error bars in Figures 2–3.
  • Handling of crossed intervals (ℓi>ui\ell_i>u_i) in coverage, width, and interval score:
  1. Zero crossings for all anchored methods: As shown in Section 2.1.1 (ll. 117–124), the Tian–Pearl constraints in Eq. (5) are necessary and jointly sufficient for a non-empty feasible set, guaranteeing 0≤L(z)≤U(z)≤10\le L(z)\le U(z)\le1. Because the bound terms and Eq. (5) are linear in (pxy,μx)(p_{xy},\mu_x), averaging across ENN draws preserves Eq. (5) and Lˉ(z)≤Uˉ(z)\bar L(z)\le\bar U(z), while CLR (Eq. 15) adds non-negative outward shifts (καLsℓz(j),καUsuz(k)≥0\kappa_\alpha^L s_{\ell_z}(j),\kappa_\alpha^U s_{u_z}(k)\ge0), yielding L^α(z)≤Lˉ(z)≤Uˉ(z)≤U^α(z)\hat L_\alpha(z)\le\bar L(z)\le\bar U(z)\le\hat U_\alpha(z). Thus, ℓi≤ui\ell_i\le u_i holds for 100% of test points across Anchored, MB Full, MB Last Lyr., and Anchored EpiNet (0% crossed intervals).
  2. Unconstrained S- and T-learners: Crossed intervals (ℓi>ui\ell_i>u_i) occur only when Eq. (5) is violated in the unconstrained S-learner (2.36% of test points) and T-learner (2.56%). In evaluation (Appendix E.6), coverage checks 1[ℓi≤yi≤ui]\mathbf{1}[\ell_i\le y_i\le u_i] and 1[ℓi≤L(zi) and U(zi)≤ui]\mathbf{1}[\ell_i\le L(z_i)\text{ and }U(z_i)\le u_i] directly, so any crossed interval (ℓi>ui\ell_i>u_i) automatically evaluates to 0 (non-coverage). For the Winkler interval score ISi=(ui−ℓi)+2α(ℓi−yi)++2α(yi−ui)+\mathrm{IS}_i=(u_i-\ell_i)+\frac{2}{\alpha}(\ell_i-y_i)_++\frac{2}{\alpha}(y_i-u_i)_+, endpoints are passed unchanged without reordering, so any crossed interval incurs a net penalty of at least (2α−1)(ℓi−ui)=39(ℓi−ui)(\frac{2}{\alpha}-1)(\ell_i-u_i)=39(\ell_i-u_i) at α=0.05\alpha=0.05. We will state this explicitly in Appendix E.6 and report the crossed-interval rates alongside % Valid.
  • Application stability and practical interpretation (Appendix H.3–H.8, pp. 36–40):
  1. Full details in Appendix H: Appendices H.5–H.8 (Tables 7–8, Figures 5–6) provide a detailed breakdown beyond Figure 4, showing that non-overlapping corrected PNS intervals within the same CATE bin have median positive separations of 4.0–5.9 percentage points (90th percentile 13.2–15.7 pp), and that for 37.6% of devices (163,642 units, Table 8) the corrected PNS lower bound exceeds the CATE point estimate. Practically, because CATE(z)=P(benefit∣z)−P(harm∣z)\text{CATE}(z)=P(\text{benefit}|z)-P(\text{harm}|z) nets out benefit and harm whereas PNS(z)=P(benefit∣z)\text{PNS}(z)=P(\text{benefit}|z) isolates the probability of benefit, two devices with identical net effects can have substantially different probabilities of benefiting when one subgroup experiences both higher benefit and higher harm.
  2. New 100-refit stability analysis: To quantify stability directly, we repeated the entire 3-fold cross-fitted case study 100 times, retraining every fold model from scratch under new stratified fold assignments and random seeds:
Estimated CATE rangeOriginal Figure 4 (%)Mean over 100 refits (%)Across-refit SD (%)Min–Max over 100 refits (%)
[−1,0)[-1,0)13.713.90.812.5–15.1
[0,0.2)[0,0.2)36.636.21.134.4–38.0
[0.2,0.4)[0.2,0.4)60.960.50.959.0–62.1
[0.4,0.6)[0.4,0.6)57.557.81.056.1–59.4
[0.6,1][0.6,1]51.250.81.348.7–52.9

The refit means differ from Figure 4 by at most 0.4 percentage points with standard deviations of only 0.8–1.3 percentage points, demonstrating that aggregate within-bin PNS non-overlap rates are stable across data splits and training seeds.

I am satisfied with the authors' respone. I will raise my point by 1

Reviewer_iubwOpenReview ↗
Rating
4
Confidence
4
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary
  • This paper introduces a solution to avoiding conflicting constraints from finite sample estimates and maximum-bias.
  • Finite sample estimate conflicts are circumvented with a joint model for the joint probabilities anchors
  • Maximum-bias is mitigated via an uncertainty weighting approach using so called epistemic neural networks
  • Results are promising and limitations discussed sufficiently

Clear acceptance

Critical strengths
  • Clear execution: the paper has a clear idea and description of the problem
  • Clear solution: both the joint model via neural networks, and the ENNs for maximum-bias correction are reasonable fixes to the problems
  • Clear presentation: notation is introduced diligently, papers discussed adequately and results described in detail
  • Table 1 results are very convincing, and make a convincing ablation study, e.g it’s encouraging to see the anchor network raise constraint violations to 100% valid
Critical weaknesses
  • Another fully real world datasets evaluation, possibly even with causal chambers would be of great value
  • Table 1: column “valid” I assume refers to the constraints violations “constraint violation rate”, “valid” isn’t clear enough
  • Readers with less background might benefit from a figure for the DAG, observational and experimental, in question, as well as a schema or diagram for the structure of the anchor network
Questions for authors
  • How 'useful' are these bounds for decision making? E.g. how often are the bounds informative, i.e. exlucidng 0?
  • Can you comment further on the scalability of this approach?
  • Would this approach translate to other bounding methods with constraints, e.g. https://arxiv.org/pdf/2304.2023, https://arxiv.org/pdf/2202.13851

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.