- Rating
- 2
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper proposes CAFE, a prior-data fitted network for individual counterfactual prediction. The method trains a transformer on synthetic data generated from fully observed additive noise SCMs with varying causal graphs. Given observational data, an individual's factual observations, and an intervention, CAFE directly predicts the Bayesian counterfactual posterior without explicitly estimating the causal graph. Experiments evaluate prediction accuracy in identifiable settings, uncertainty quantification under graph non-identifiability, and robustness on realistic simulated datasets.
Critical strengths
-
Relevant and challenging problem: The paper addresses individual-level counterfactual prediction without requiring knowledge of the underlying causal graph.
-
Strong empirical performance: CAFE demonstrates consistently competitive or superior counterfactual prediction accuracy across diverse benchmarks, including settings that violate its modeling assumptions.
Critical weaknesses
-
Limited novelty: The proposed approach closely follows existing causal PFNs (e.g., Ma et al., CausalFM; Robertson et al., Do-PFN), which already amortize Bayesian causal inference over SCM priors, including uncertainty over causal graphs. These existing frameworks could be straightforwardly adapted to individual counterfactual prediction by conditioning on the observed factual outcome , modifying the prediction target, and restricting the SCM prior to identifiable classes such as additive noise models. It remains unclear what fundamentally new methodological contribution CAFE provides beyond this adaptation. Additionally, these straightforward adaptations could be considered strong experimental baselines that are ommitted.
-
Insufficient discussion of related work and missing baselines: Crucial related work on causal PFNs is discussed only in the appendix. The authors primarily distinguish these approaches by their prediction targets, without explaining the technical contributions of CAFE relative to existing methods. Moreover, the main experiments do not compare against causal PFNs such as CausalFM or Do-PFN, nor with regular TabPFN.
-
Theory: The theoretical results seem largely straightforward adaptations of known results. It would be helpful if the authors could elaborate on their contribution here.
-
Assumptions. Individual counterfactual outcomes are generally not identifiable even when interventional distributions are known. CAFE circumvents this by imposing strong structural assumptions, particularly fully observed additive noise models with independent exogenous noise. Such assumptions may be unrealistic in many practical applications, limiting the applicability of the proposed framework. This is primarily a limitation of the considered problem setting rather than the specific methodology.
-
Prior dependence and uncertainty evaluation: When the causal graph is non-identifiable, the proposed counterfactual posterior depends on the assumed SCM prior rather than being uniquely determined by observational data. The non-identifiability experiment primarily evaluates whether CAFE approximates the posterior induced by its own prior, without assessing sensitivity to prior misspecification. Furthermore, experiments violating the ANM assumptions evaluate point-prediction accuracy but not the reliability of the predicted counterfactual uncertainty.
Questions for authors
It would be helpful if the authors could elaborate on their contribution. What changes conceptually beyond "conditioning on Y" and restricting the prior?
2 comments
Thank you for the detailed response and additional experiments. I appreciate the authors' efforts to address my concerns.
-
Novelty (W1): My main concern remains unresolved. CAFE appears to follow the established causal PFN paradigm, primarily changing the conditioning inputs, SCM prior, and prediction target. The authors do not clearly identify a methodological contribution beyond this straightforward adaptation.
-
Related work and baselines (W2): The additional comparisons with CausalFM, Do-PFN, and TabPFN are valuable and demonstrate strong empirical performance. However, adapting pretrained L2 models through CATE adjustment does not address whether existing causal PFN approaches could be straightforwardly retrained for L3 prediction.
-
Theory and assumptions (W3–W4): I appreciate the clarifications and acknowledgment of restrictive assumptions. However, the theoretical contributions remain largely adaptations of established results.
-
Prior sensitivity and uncertainty (W5): The additional experiments are useful. Nevertheless, some prior changes substantially affect performance, and the uncertainty evaluation shows imperfect calibration (e.g., 96.9% coverage at a nominal 90% level for pH).
Overall, the rebuttal substantially strengthens the empirical evaluation, but my central concern regarding methodological novelty remains. I therefore maintain my rating.
Response to Reviewer AEKi
We thank the reviewer for suggesting closer comparisons with existing causal PFNs and broader evaluation of prior sensitivity and counterfactual uncertainty. These comments helped us improve the paper’s positioning and make the evaluation more comprehensive. In the revised PDF, we have expanded the related-work discussion, added foundation-model baselines, prior-sensitivity experiments, and uncertainty metrics beyond the ANM setting.
W1 & Q1: What does CAFE contribute beyond existing causal PFNs?
A: CAFE’s contribution is to develop an effective PFN-based model for individual counterfactual prediction and establish its value through extensive experiments. PFNs provide a reusable learning paradigm, but existing pretrained causal PFNs do not directly solve this challenging counterfactual prediction task. CAFE demonstrates that this paradigm can deliver accurate individual predictions in identifiable settings and capture posterior uncertainty in non-identifiable settings. Its consistent improvements over CATE-adjusted foundation-model baselines (Table 4) further support the value of developing and training a model specifically for counterfactual prediction task.
W2: Could the authors expand the discussion of causal foundation models and add comparisons with them in the main text?
Yes. We have added discussion of these related work in the Introduction and add these causal foundation models as import baselines in Section 4.4. We recognize causal foundation models, including Do-PFN and CausalFM, as important methodological contributions to the field of causal inference. They helped pioneer a new paradigm which uses pretraining to amortize Bayesian posterior inference for causal queries. They target L2 causal queries. We adapt their predictions through CATE-adjustment to incorporate our L3 task formulation. We evaluate them on binary-treatment datasets to accommodate their supported treatment settings. Table 4 shows that CAFE outperforms these baselines.
Table 4: RMSE on C-Suite and MAPE (%) on Sangiovese.
W3: What does the theoretical analysis contribute beyond established results?
A: Building on established counterfactual identification theory, our analysis characterizes CAFE’s target posterior and explains the uncertainty that remains when compatible graphs imply different counterfactual outcomes. Our main contribution is a pretrained method for individual counterfactual prediction, together with extensive empirical validation of the PFN approach on this task.
W4: How do CAFE’s assumptions affect its practical applicability?
A: Counterfactual prediction necessarily relies on assumptions about the underlying data-generating process. We agree that CAFE’s fully observed ANM assumptions can be restrictive in practice. Therefore, we evaluate CAFE in real world scenarios, providing empirical evidence of its predictive robustness when these assumptions are violated (Section 4.3).
W5.1: How sensitive are CAFE’s predictions to the pretraining prior?
A: We vary the pretraining graph and noise priors while evaluating all variants on the same datasets. Performance remains broadly stable under moderate prior changes (Appendices D.7.2–D.7.3).
W5.2: Could the authors evaluate the reliability of predicted counterfactual uncertainty beyond the ANM setting?
A: Yes. We have expanded the Sangiovese evaluation to include CRPS and Coverage@90 in Section 4.4. CRPS evaluates each predicted distribution against the ground-truth counterfactual outcome, rewarding accuracy while penalizing excessive uncertainty. Coverage@90 is a uncertainty calibration method that measures the fraction of counterfactual outcomes within the predicted 5th–95th percentile interval, with a nominal target of 90%. CAFE’s coverage is closest to this target on Brix. On pH, CAFE is second only to Do-PFN.
Table 5: Uncertainty quantification on Sangiovese.
If the reviewers have any remaining questions, we would be glad to discuss and clarify them further.