- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper studies identifiability in causal Prior-Fitted Networks under unobserved confounding and establishes conditions under which source-labeled randomized evidence identifies causal effects. It proposes RA-PFN, which adapts CausalPFN using mixed-source prior fitting and corresponding architectural adaptations. Experiments on three semi-synthetic benchmarks report improved CATE and ATE estimation.
Critical strengths
- The mixed observational-randomized identification strategy provides a transparent way to construct identifiable synthetic training tasks. Although the conditional mean generalizability from RCT to the observational population generally cannot be verified from the available real-world data, it can be enforced exactly in the simulator. This creates a clear connection between the identification assumptions and the training-task construction.
- Reasonable empirical improvement.
Critical weaknesses
-
There seems to be limited evidence for deployment robustness. Enforcing identification assumptions during synthetic training does not ensure that independently collected observational and randomized datasets satisfy them at deployment. The paper's reported evaluations preserve these assumptions by construction. A controlled sensitivity analysis, particularly introducing violation of conditional mean generalizability from trial participants to the target population, would clarify the method’s practical robustness.
-
Inspired by equation (5), we can further decompose the error by: \begin{aligned} \left|\widehat{g}_N-g(\psi)\right| \leq \left|\widehat{g}_N-m_N\right| + \left|m_N-\bar{g}_{\pi}(R)\right| + \left|\bar{g}_{\pi}(R)-g(\psi)\right|. \end{aligned} The last two terms are examined by Theorem 3.2.(i) and Theorem 4.1, respectively, but the control of the first term, which is essentially network approximation error, is not established in the paper. Consequently, it does not establish consistency guarantee for the implemented estimator. Maybe an RA-PFN-specific error analysis would help facilitate better understanding of what drives the network approximation error.
-
The adaptation prior seems to cover a narrow family of hidden-confounding mechanism and mainly focus on low-dimensional settings. Even starting from CausalPFN does not necessarily make the adapted predictions retain the pretrained model’s generalization capabilities. We either need a sensitivity analysis on the prior coverage, or extend the coverage of adaptation priors to more scenarios, e.g. high-dimensional covariates, correlated covariates, multiple latent confounders, nonlinear latent outcome effects, or latent treatment-effect modification.
Questions for authors
- Under the identification strategy of causal data fusion, observational outcomes are no longer needed for identification, but they might be helpful for estimation. How much does access to observational treatment and outcome information improve estimation, beyond access to the same randomized sample and observational covariates?
- A potential advantage of PFN-based causal estimation is its ability to provide posterior distributions, yet the experiments evaluate only point estimates derived from posterior means. Could the authors also examine the posterior distribution of CATE?
- How do accuracy, source weighting, and computational cost change outside the 128-row context window setting?
- What explains CausalFM’s relative ATE error of on both CPS and PSID in Table 1?
- For Table 1, Appendix C.5 selects among ten configurations for each fusion baseline using source-treatment-stratified validation and a source-balanced R-loss. Given that the main experiment contains only 13 randomized observations, how sensitive are the selected configurations and reported performance to the validation split?