acceptodds
Under review as a conference paper at ICLR 2027

Unit-level aggregation breaks coverage in causal foundation models

Abstract

Causal foundation models (CFMs) show promising results for treatment-effect estimation, with unit-level estimates that are competitive on standard benchmarks. Many causal analyses, however, inform decisions through effects averaged over groups of units, such as the full sample, the treated units, or a subgroup, and for these estimands a CFM is useful only if its credible intervals are calibrated. We examine these intervals and find that their coverage degrades, both in simple synthetic settings and on standard benchmarks. On ACIC 2016, for example, temperature-calibrated 95% intervals from CausalPFN fall below nominal coverage beyond groups of 30 out of 4802 units, and its interval for the full-sample average contains the true value in only 20% of datasets. We trace this problem to how these models are trained and how their intervals are formed. They output a predictive distribution for one unit at a time and, at inference time, combine the unit-level variances as if the errors on different units were independent. We show formally that posterior uncertainty over the data-generating process makes these errors correlated. As a result, the combined variance is too small by a factor that grows with the size of the group. Empirically, this factor predicts where coverage fails and by how much. We then show that two neural process models, which make joint predictions for a group of units, account for this correlation and substantially improve coverage on synthetic tasks.

Primary area:
causal reasoning
Keywords:
causal inference, treatment effect estimation, prior-data fitted networks, Bayesian inference, uncertainty quantification

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 4 reviews, above 85% of the others.
Reviewer_PrsfOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

The paper shows that causal foundation models produce poorly calibrated uncertainty intervals for average treatment effects and subgroup effects. The main issue is that the predictions are correlated and the variance estimates considers them as independent. The authors theoretically characterize this problem and propose two neural-process models that account for these correlations.

Critical strengths
  1. The paper identifies an important limitation of existing CFMs and demonstrates it across six standard benchmarks.

  2. The theoretical analysis clearly explains how correlated estimation errors lead to poor interval coverage.

  3. The experiments compare independent and joint aggregation using the same trained models and show that accounting for correlations substantially improves coverage.

Critical weaknesses
  1. The proposed models are evaluated only on a simple synthetic setting with three covariates. They are not evaluated on standard benchmarks such as IHDP and ACIC 2016, where the paper demonstrates poor coverage for CausalPFN.

  2. The main theoretical result is based on a standard variance decomposition for correlated errors, closely related to the well-known design effect in survey sampling. While applying this analysis to causal foundation models is useful, the paper could better clarify what is theoretically novel beyond this application.

Questions for authors

Can the authors evaluate their proposed methods on standard benchmarks such as IHDP and ACIC 2016? Since the proposed models are tested only on a relatively simple synthetic setting, it is unclear whether the improvements in coverage generalize to more realistic problems.

1 comment
AuthorsOpenReview ↗

Response to Reviewer Prsf

Thank you for the review.

W1 and Q: the models improve coverage on IHDP and ACIC 2016. As heads on a frozen CausalPFN backbone (general response), their full-sample coverage on IHDP is 0.94 for the CNP head and 0.89 for the latent NP head, against 0.60 for CausalPFN. On ACIC 2016, coverage is 0.71 and 0.78, against 0.20 for CausalPFN. Unit-level aggregation from the same CNP and latent NP head weights gives 0.30 and 0.48 on IHDP and 0.10 and 0.18 on ACIC 2016, so the gain comes from the aggregation step (Figure 3 in the revision).

W2: the new theory is the source of the error correlation and its consequence. Proposition 1 is the design effect, and the paper credits it to Kish (1965). The new theoretical content is Proposition 2. The error correlation equals the posterior covariance of unit effects over the SCM, so every CFM that reports unit-level distributions has it. The unit-level marginals do not determine this covariance, so no interval built from them alone can account for it. The new exact-posterior experiment shows the consequence directly. Unit-level aggregation of the exact posterior's marginals has coverage of 0.25 at k = n on our synthetic family, while the exact posterior of the subset-CATE has nominal coverage (Figure 2 in the revision). The revised paper states these points more directly in Section 4.

Reviewer_pQ8XOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper identifies an important failure mode in Causal Foundation Models (CFMs), where naïve aggregation over individual causal effect predictions (CATEs) to obtain group-level parameters, such as ATE, severely degrades the coverage of predicted intervals. The root cause of the identified issue is that errors on individual units are treated as independent, where they are in fact correlated, as shown in the paper.

The paper attributes this problem to the entire class of CFMs and demonstrates it empirically with CausalPFN across a number of datasets commonly used in the field. The proposed solution is to make joint predictions for a group of units, which is demonstrated work well empirically on small-scale synthetic tasks.

Critical strengths

Important CFM failure mode identification. With CFM and PFN-style methods becoming more popular in the space of treatment effect estimation, identifying and addressing such failure modes is very important, especially if said models could be used in sensitive decision-making domains, such as healthcare. And I agree that group-level causal parameters are often of analysts’ interest, so their validity is very relevant.

Strong technical contributions. The theoretical explanation clearly links the problem to the entire family of CFMs, and the empirical results on CausalPFN confirm this is a real issue. The proposed two neural process models are conceptually well motivated, and their effectiveness is demonstrated empirically.

Critical weaknesses

Framing broader than empirical evidence. The paper discusses the failure mode of “causal foundation models”, but the empirical analysis includes only one model (CausalPFN). I agree that the theoretical arguments are model-agnostic, but the empirical evidence confirming this problem is widespread across all CFMs is currently missing.

Proposed solution feels preliminary. The two built neural models (CNP and latent-NP) are relatively small and validated in simplified synthetic settings. This leaves a gap between the setting where the problem is shown (the benchmark with CausalPFN) and where the solution is demonstrated to work. As a result, there is currently limited evidence that the proposed solution will scale well to the size of the problems CFMs are intended for.

Questions for authors
  1. Can the authors provide results for additional CFMs, such as Do-PFN and CausalFM?
  2. Can the authors evaluate the proposed CNP/latent-NP approaches at a larger CFM-level scale?
2 comments
AuthorsOpenReview ↗

Response to Reviewer pQ8X

Thank you for the review and for the positive assessment of the problem and the theory.

Q1: CausalFM fails the same way. With unit-level aggregation, its full-sample interval contains the true value in 9% of the 240 datasets, against 53% for CausalPFN. The failure appears even where its unit-level intervals are calibrated. On Hill-A, Friedman, and Kang–Schafer, its single-unit coverage is at or above nominal (1.00, 1.00, and 0.97), and its full-sample coverage is 0.00, 0.25, and 0.00. On IHDP, Hill-B, and ACIC 2016, its single-unit coverage is already below nominal (0.60, 0.86, and 0.92), and its full-sample coverage is 0.05, 0.23, and 0.00. Its intervals are built the same way as CausalPFN's, by averaging independent draws from each unit's mixture, with no calibration, since CausalFM has none. To check our use of CausalFM, we ran it on its own test data and obtained a PEHE of 0.459 under its protocol, against 0.515 ± 0.20 reported in its paper. We chose CausalFM because, like CausalPFN, it returns a predictive distribution over each unit's CATE, so unit-level aggregation applies to it directly, and because it comes from a different group, with a different prior and a TabPFN-based architecture.

Do-PFN fails the same way. Do-PFN predicts the outcome distribution under each treatment and provides no interval for a group average, so we formed one from independent draws under each treatment for each unit, with unit-level aggregation. Its single-unit coverage is at or near nominal (0.947 to 1.00), yet its full-sample interval contains the true value in 68 of the 240 datasets, against 126 for CausalPFN and 21 for CausalFM. On four benchmarks the data have more covariates (25 to 58) than Do-PFN was pre-trained on (at most 6 columns), and its unit-level error there is large. On Friedman and Kang–Schafer, where its error is low, full-sample coverage is 0.28 and 0.33. The revised appendix reports these results.

Q2: the models work at full scale. Both models now run as heads on a frozen CausalPFN backbone, trained on a broad prior with up to 5000 units and 60 covariates and evaluated on the six benchmarks (general response). Full-sample coverage rises from 53% of 240 datasets for CausalPFN to 90% and 89%. On ACIC 2016 (n = 4802), the largest benchmark, it rises from 0.20 to 0.71 and 0.78 (Figure 3 in the revision).

Thank you for addressing my questions, I do appreciate the work. Seeing empirical evidence that CausalFM and Do-PFN exhibit the same failure behaviour helps, even if it's obvious from the theoretical perspective. Thank you also for running the heads on the six datasets (Figure 3). This gives me reassurance that the method works on datasets where the problem was demonstrated in the first place.

My only remaining concern, as other reviewers also pointed out, is that the oferred solution (heads on frozen PFNs) to this correctly identified problem is rather patchy and does not address it on the fundamental level of PFN design, which would be a much stronger take.

Just to be clear, I think this is an interesting and important problem, but I'm not confident that the proposed solution will have the desired impact. For this reason, I will keep my initial score unchanged.

Reviewer_PTizOpenReview ↗
Rating
3(+1)
Confidence
4
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

The paper diagnoses a coverage deficiency of causal foundation models when evaluated for group-average effects such as ATE (or e.g. any averaging of CATE on any subset) arising from per-conditioning point evaluations with independent uncertainties which fail to take into account any covariance and result in systematic under-coverage. This is empirically demonstrated with CausalPFN on several common benchmarks and a remedy is proposed using Neural Processes that resolves this.

Critical strengths

The paper does identify an important issue that is not sufficiently addressed in existing literature (although the authors are not the first to raise it - which they acknowledge).

Critical weaknesses

While an important gap is correctly identified, I unfortunately find the paper has several crucial weakness that do not leave me thinking the gap has be well addressed, namely the current version suffers from:

Unclear and lacking contribution:

  • The raised uncertainty problem is not specific to causal foundation models. Correct positioning would accurately describe the broader scope where this is applicable.
  • Proposed NP solutions are defensively hedged as "not aiming to match CausalPFN". I expect the gap to be addressed by actually offering a competitive replacement. I see this as more than a pedantic point: scaling of proposed NP solution to the relevant sizes is not discussed and empirical validation that the solution performs at the appropriate scale is also missing. In its current form, the proposed solution is clearly not a replacement for existing causal foundation models
  • I fail to see how training an alternative model that directly amortizes the inference of the averaged effect θS\theta_S presents a meaningful innovation.

Insufficient exposition:

  • Sections 4.1 and 4.2 could be significantly clearer and shorter (complementary objectives in my view) - the theory is quite straightforward, just written in an overly complex manner in my opinion (for example exchangeability certainly does not need to appear in main body of paper).
  • Conversely, more space should be given to differentiating from existing pfn corrections literature (which are cited) as well as justification of novelty of the NP solution in relation to existing neural processes literature (I suspect there is barely any).
  • I would expect proper exposition of existing corrections to also reveal relevant baselines that are currently missing.

Finally, if your three-covariate synthetic SCM is a Gaussian linear regression, then analytical posterior is possible, revealing proposed cnp achieves nominal coverage at 14-15 times the width of the exact posterior, a stark shortcoming. I see this primarily as indication of broader deficiencies of your method.

In summary, I wish the authors were substantially more ambitious in their effort to resolve the gap that they correctly identified. In its current form, I am not convinced of the contribution of this work.

Questions for authors

Please see weaknesses above.

6 comments
AuthorsOpenReview ↗

Response to Reviewer PTiz

Thank you for the detailed review. Our paper diagnoses a general issue, and its solution now runs at the full scale of existing CFMs. Even a model with exact unit-level uncertainty undercovers when it combines units as independent. We respond in the order of the review.

The proposed models now work at the full scale of existing CFMs. The paper shows that current CFMs undercover group averages because they combine unit-level predictive distributions as if the units' errors were independent, while posterior uncertainty over the SCM makes these errors correlated. It proposes SCM-marginalized aggregation, which makes joint predictions for a group of units, and not a new CFM. The sentence the review refers to ("we do not claim they match CausalPFN's accuracy or generality") described small models trained on a synthetic family, and the revision removes it. We have now trained both models as heads on a frozen CausalPFN backbone, on a broad prior with up to 5000 units and 60 covariates, at about 4 to 6% of CausalPFN's own training cost. On all six benchmarks, they raise full-sample coverage from 53% of 240 datasets for CausalPFN to 90% and 89% (general response). We believe this addresses the review's main objection, since an existing CFM with these heads recovers most of the lost coverage for group averages.

The mechanism applies beyond CFMs, and the revision says so. We agree that it applies to any model that reports unit-level marginals and combines them as independent, including PFNs used to average predictions over a set of query points, and Nagler and Rügamer (2025) note the same limitation for their pointwise procedure. The revision states this scope in the related work. We focus on CFMs because causal inference is mostly about averages (the ATE, the ATT, and subgroup effects), and CausalPFN's aggregate estimator combines unit-level draws in this way.

Amortizing the averaged effect follows from the diagnosis. We do not present the direct-decode CNP as a new kind of model; it is a standard CNP. Its use here follows from Proposition 2. The variance of a group average depends on posterior covariances between units that the unit-level marginals do not determine, so a model trained only on unit-level targets cannot report it. Targeting the averaged effect directly is the simplest way to account for these covariances, and the latent NP is the other standard way. This observation motivated the implementation, and the heads show that it works when added to an existing CFM.

Sections 4.1 and 4.2 are now simpler. We agree, and the revision reduces the notation around Propositions 1 and 2. We would like to keep the exchangeability argument in the main text, stated more plainly. It explains why the errors of any CFM are correlated through the shared SCM, and it helps readers with less background in causal inference or Bayesian nonparametrics follow Proposition 2. Related papers state it in the main text for the same reason (Garnelo et al., 2018; Falck et al., 2024; Ye and Namkoong, 2024).

The revision adds the related work and the baselines. It expands the related work on PFN corrections (martingale posteriors, TabMGP, and OSPC) and on neural processes that model dependence between target points (Gaussian NPs, autoregressive CNPs, and convolutional latent NPs). The general response compares OSPC and the martingale posterior with the heads on the six benchmarks. If the reviewer has other baselines in mind, we would be glad to add them.

The exact posterior separates aggregation from accuracy. The reviewer is right that the synthetic family has a closed-form posterior. Unit-level aggregation of the exact posterior's own marginals has coverage of 0.25 at k = n, so exact marginals do not prevent the failure. The CNP's interval is 14 to 15 times wider than the exact posterior's, as the reviewer states, because its point estimate has 15 to 18 times the root-mean-square error of the posterior mean, and its coverage is close to nominal (0.92 at k = n). As heads on a frozen CausalPFN backbone, the same models have errors comparable to CausalPFN's (2.8 to 4.8 times the posterior mean's, against 2.3 to 3.3 times), and their full-sample intervals are 2.1 (CNP head) and 1.7 (latent NP head) times the exact width (Figure 2 and the appendix of the revision). In short, the width gap reflects the accuracy of a small model, and the coverage failure reflects aggregation.

Thank you for your response and for addressing most of my concerns. I will be raising my score from Clear to Weak reject. I am open to further raising my score.

My remaining concerns are whether the improvements are attributable to additional training and whether the proposed approach offers a meaningful advantage over simpler alternatives. I would therefore appreciate a computationally fair comparison against (1) CausalPFN trained longer with its original objective (to match budgets) and (2) a simple learned correction to CausalPFN's aggregate uncertainty, retaining its point estimates. The comparison should consider coverage, interval width, and estimation accuracy together.

AuthorsOpenReview ↗

Response to Reviewer PTiz: matched-budget comparisons

Thank you for the follow-up and for raising your score. At the heads' training budget, neither training CausalPFN further nor a learned correction matches the heads: further training leaves coverage unchanged, and the corrections reach nominal coverage only with intervals several times too wide. The gain comes from learning the variance of the subset-CATE, not from the extra training. The revision adds these results to Appendix F (Tables 14 and 15, Figure 8).

Every new model trains on the heads' prior with a frozen CausalPFN backbone and three seeds, on the paper's 40 datasets per benchmark with the whole sample as context. No benchmark data is used for training, fitting or checkpoint selection. The table gives means over the six benchmarks, in units of sd(y). The interval score (Winkler, α = 0.05) is the width plus a penalty for each miss, so it weighs coverage and width together; lower is better.

datasets covered at k=nk=n (of 240)width at k=nk=nnRMSE at k=nk=ninterval score at k=nk=ninterval score over subset sizes
CausalPFN, temperature-calibrated1290.0380.0250.330.47
CausalPFN, temperature 12120.0820.0250.200.81
output layer trained further1240.0390.0270.370.47
whole model trained further1290.0370.0250.330.45
constant-ρ\rho correction2400.6150.0250.620.77
binned correction2400.6690.0250.671.25
CNP head2160.1180.0370.200.32
CNP head, CausalPFN's mean2350.1480.0250.160.31
latent NP head2130.1250.0370.210.51

(1) Training CausalPFN further at the same budget does not improve coverage. With its temperature calibration, its full-sample interval contains the true value in 124 of 240 datasets when only its output layer is trained further and 129 when the whole model is, against 129 before. Unit-level accuracy improves slightly (on Hill-A, unit-level nRMSE falls from 0.072 to 0.045 for the whole model), but the subset intervals are still formed by unit-level aggregation, so coverage stays low, as the exact-posterior result predicts. We trained its output layer (1.1M parameters, the same 5M prior tasks and steps as the CNP head) and, separately, the whole model (18.8M parameters, the same GPU time), with its own per-unit objective. All rows share one full-context evaluation (the paper's 126 uses CausalPFN's public interface).

The gain comes from the learned variance of θS\theta_S, not from the extra training. With its mean fixed to CausalPFN's subset-average estimate, the CNP head contains the true value in 235 of 240 datasets, keeps CausalPFN's accuracy, and has the lowest mean interval score of all rows, both at k=nk=n and over subset sizes. This differs from the CNP head only in its mean, so it is a variant of the proposed approach rather than an alternative to it, and the paper reports it as an ablation. It supports the paper's motivation: the variance of a group average cannot be recovered from unit-level outputs, so it has to be learned, and once learned it can be paired with an existing CFM's point estimate.

(2) Corrections that depend only on kk and nn reach nominal coverage with intervals 2.5 to 16 times wider than needed. Both contain the true value in all 240 datasets and have the worst interval scores of all rows on Hill-A, Friedman and IHDP. The widening each benchmark needs at k=nk=n ranges from 1.3 to 7.7 times and is set by its ρS\rho_S and mSm_S, which Equation 1 relates to coverage (it predicts the needed widening to within a factor of 1.4). A correction that sees only kk and nn cannot adapt to them (Figure 8). Both keep CausalPFN's point estimate and widen its temperature-calibrated interval, with factors fitted on held-out prior tasks: a constant-ρ\rho correction, 1+(k−1)ρˉ\sqrt{1+(k-1)\bar\rho}, the design effect's form with one correlation ρˉ\bar\rho for every dataset, and one factor per bin of k/nk/n and nn (33 parameters).

Without calibration, CausalPFN's intervals are too wide for small subsets and too narrow for large ones. At temperature 1, its interval narrows as 1/k1/\sqrt{k}, while the width needed for 95% coverage levels off (Figure 8 in the revision). It is about six times too wide at the smallest subsets and narrows to about the needed width only near the full sample. That is why it contains the true value in 212 of 240 full-sample datasets but has coverage of 0.53 on ACIC 2016, the largest benchmark. Its interval score over subset sizes is 0.81, against 0.31 to 0.51 for the heads.

Neither more training nor a correction that ignores each dataset's error correlation recovers coverage at a reasonable width. Learning the variance of the subset average does, which is the step unit-level aggregation misses.

Reviewer_rdDBOpenReview ↗
Rating
3
Confidence
3(+1)
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper investigates the poor coverage of aggregated treatment-effect intervals in causal foundation models. It attributes the problem to ignored correlations between unit-level estimation errors, derives a theoretical explanation, and explores two neural process models to improve uncertainty estimation.

Critical strengths
  1. The paper identifies an important limitation of existing causal foundation models that is often overlooked in unit-level evaluations.
  2. The theoretical analysis provides a clear explanation for coverage degradation as subgroup size increases.
  3. Experiments across six benchmarks demonstrate the problem, and the controlled synthetic experiments support the proposed explanation.
Critical weaknesses
  1. The proposed solutions mainly adapt existing CNP and latent NP architectures. The main contribution lies in diagnosing the problem rather than developing a substantially new method.
  2. While the coverage problem is demonstrated across six benchmarks, the proposed models are evaluated only on a relatively simple synthetic family. Their effectiveness on more realistic datasets remains unclear.
  3. The paper discusses existing methods for joint posterior inference but does not experimentally compare them with the proposed solutions.
  4. The proposed models require separate training, while the variance correction relies on quantities unavailable in practice. It remains unclear how the findings can directly improve existing pretrained CFMs.
Questions for authors
  1. Can the proposed models be evaluated on standard benchmarks such as IHDP and ACIC 2016
  2. How do the proposed methods compare with existing joint posterior inference approaches
2 comments
AuthorsOpenReview ↗

Response to Reviewer rdDB

Thank you for the review and the clear questions.

Q1 and W2: the models improve coverage on the standard benchmarks. As heads on a frozen CausalPFN backbone (general response), they raise full-sample coverage on IHDP from 0.60 for CausalPFN to 0.94 (CNP head) and 0.89 (latent NP head), and on ACIC 2016 from 0.20 to 0.71 and 0.78 (Figure 3 in the revision). Unit-level aggregation from the same head weights gives 0.30 and 0.48 on IHDP and 0.10 and 0.18 on ACIC 2016, so the gain comes from the aggregation step.

Q2 and W3: joint-posterior methods recover coverage, but not with amortized inference. OSPC applied to CausalPFN's posterior draws contains the true full-sample value in 226 of the 240 datasets (213 with an estimated propensity score), against 216 and 213 for the heads, with narrower intervals on most benchmarks. It requires fitting and validating a propensity model on each dataset, the per-dataset nuisance estimation that a CFM is meant to avoid, and it gives intervals only for fixed estimands such as the ATE and ATT, while the heads give an interval for any queried subset from the same forward pass. The martingale posterior contains the true value in at least 93% of datasets on every benchmark, at 11 minutes per dataset of 1,000 units and 55 minutes per ACIC 2016 dataset on one GPU, against 0.014 seconds for CausalPFN with the CNP head.

W1: the contribution is the diagnosis, and the models follow from it. We agree that the main contribution is the diagnosis, and the paper presents it as such. Its new theoretical content is Proposition 2, which traces the error correlation to posterior uncertainty over the SCM, so no interval built from unit-level marginals alone can account for it. The two models are standard neural process families, and we do not claim a new architecture, as the revised Discussion states. They are the two ways of placing the integral over the SCM outside the product, and Proposition 3 shows that the direct-decode objective targets the posterior variance of the subset-CATE. The new results show that the same models work at full scale.

W4: the heads are cheap to train and use no data from the target dataset. Training takes 6.3 to 6.5 hours for a CNP head and 9.2 to 9.4 hours for a latent NP head on one L40S GPU, against seven days on one A100 reported for training CausalPFN itself, or about 4% and 6% of that time. The heads add 2.5% (CNP) or 3.3% (latent NP) to the backbone's parameter count, use no benchmark data, and at inference add one forward pass of a small network (0.014 seconds per dataset of 1,000 units for CausalPFN and the CNP head together). The pretrained CFM is unchanged, so the heads apply to it directly. The design-effect correction in Appendix C requires the true effects, as the reviewer notes. We present it as a diagnostic that separates unit-level calibration from the aggregation step, and we do not propose it as a method.

Thank you for your effort to explain. I prefer to keep my positive rating

Comments on the paper

2 comments
AuthorsOpenReview ↗

General response to all reviewers

We thank the four reviewers for their careful reviews. The reviews agree that the paper identifies an important limitation of causal foundation models (CFMs) that unit-level evaluations miss. We have run the requested experiments and uploaded a revised paper that includes them. The main results are below.

The proposed models recover most of the lost coverage at full scale. We trained both models as heads on a frozen CausalPFN backbone, used as released, on a broad prior with 100 to 5000 units and up to 60 covariates. No benchmark is used for training or checkpoint selection. On the six benchmarks, the full-sample interval contains the true value in 90% of the 240 datasets for the CNP head and 89% for the latent NP head, against 53% for CausalPFN. On ACIC 2016, coverage rises from 0.20 to 0.71 and 0.78, and on Hill-B from 0.53 to 0.83 and 0.85 (Figure 3 in the revision). Unit-level aggregation from the same head weights gives 25% and 52%, so the gain comes from the aggregation step.

Two more CFMs fail the same way, so the failure is not specific to CausalPFN. CausalFM and Do-PFN provide no interval for a group average, so any such interval must combine their unit-level distributions. With unit-level aggregation, their full-sample intervals contain the true value in 9% and 28% of the 240 datasets, against 53% for CausalPFN.

Even exact unit-level posteriors lose coverage when aggregated. On our synthetic family, where the posterior is available in closed form (Corollary 1), unit-level aggregation of the exact marginals has coverage of 0.25 for the full sample (k = n = 500) and 0.50 at k = 125. The exact posterior of the subset-CATE has coverage between 0.95 and 0.96 at every subset size (Figure 2 in the revision). Since these marginals are exact, more accurate unit-level predictions cannot fix the problem, which comes from combining the marginals as if their errors were independent.

Joint-posterior methods recover coverage but give up amortized inference. We compared the heads on the full sample with two methods of Melnychuk et al. (2026), the one-step posterior correction (OSPC) and the copula martingale posterior.

Datasets covered (full sample)Interval widthEstimandsPer-dataset cost
CNP / latent NP heads216 / 213 of 240referenceany subset0.01–1.7 s, no nuisance model
OSPC on CausalPFN226 of 240 (213 with a fitted propensity)narrower on most benchmarksATE, ATTpropensity model fitted and checked per dataset
Copula martingale posterior (TabPFN)207 of 210 run (10 of 40 on ACIC 2016)up to about 2× the heads'subsets possible from joint draws11–55 min
CausalPFN, unit-level126 of 240narrowestany subset1–8 s

OSPC requires a propensity model fitted and validated on each new dataset, the per-dataset nuisance estimation that a CFM is meant to avoid, and it gives intervals only for fixed estimands such as the ATE and ATT. The martingale posterior must be rerun for every new dataset. The heads give an interval for any queried subset from a single forward pass.

The new theory in this work is the source of the error correlation and its consequence. Group averages (the ATE, the ATT, and subgroup effects) are the standard targets of causal inference, and the CFMs we examine form intervals for them by combining unit-level distributions as if the units' errors were independent. CausalPFN's own interval for its ATE contains the true value in 8 of 40 ACIC 2016 datasets. The paper explains why. Proposition 1 is the classical design effect, credited to Kish (1965), and it predicts the observed coverage to within a median absolute error of 0.021 to 0.043. The new theoretical content is Proposition 2, which traces the error correlation to posterior uncertainty over the structural causal model. As a result, every CFM that reports unit-level distributions has it, and no interval built from the marginals alone can account for it. This motivates models that predict the group average jointly, and the new results show that they can be added to an existing CFM at full scale.

Revision. The revised paper adds the head results and the exact-posterior experiment to Section 5, and the CausalFM and Do-PFN results and the comparisons with joint-posterior methods to the appendix. It also simplifies the notation of Section 4, states the scope beyond CFMs, and expands the related work on PFN corrections and neural processes.

AuthorsOpenReview ↗

Summary of the discussion and revisions

We thank the reviewers for the discussion. This comment summarizes what the paper shows, what the discussion added, and how the revision reflects it.

What the paper shows

  • CFMs' intervals for group averages undercover, by an amount predicted by the design effect. CFMs form these intervals by combining unit-level distributions as if the units' errors were independent. Posterior uncertainty over the structural causal model makes the errors correlated (Proposition 2), so coverage falls as the group grows. CausalPFN loses nominal coverage on all six standard benchmarks, including for the full sample, and Proposition 1 predicts coverage on the synthetic family to within a median absolute error of 0.05.
  • The failure is in the aggregation step, not in the unit-level estimates. Unit-level aggregation of the exact posterior's marginals has full-sample coverage of 0.25. With the same head weights, unit-level aggregation contains the true full-sample value in 25% and 52% of the 240 benchmark datasets, against 90% and 89% with SCM-marginalized aggregation. CausalFM and Do-PFN, aggregated the same way, reach 9% and 28%.
  • Our procedure for learning the variance of the group average recovers coverage, and can be used on top of existing CFMs. Unit-level estimates do not determine this variance, so it has to be learned. With the CNP head's mean fixed to CausalPFN's estimate, the full-sample interval keeps CausalPFN's accuracy and contains the true value in 235 of 240 datasets.

What the discussion added

  • Full-scale results. As heads on a frozen CausalPFN backbone, trained on a broad prior with no benchmark data, both models raise full-sample coverage on IHDP from 0.60 to 0.94 and 0.89 and on ACIC 2016 from 0.20 to 0.71 and 0.78 (Reviewers rdDB and Prsf).
  • Matched budget. Training CausalPFN further at the heads' budget leaves its coverage unchanged (124 and 129 of 240 datasets, against 129). Corrections that depend only on subset and sample size reach nominal coverage only with intervals 2.5 to 16 times wider than needed (Reviewer PTiz).
  • Joint-posterior methods. OSPC matches the heads' full-sample coverage but needs a propensity model fitted to each dataset and gives intervals only for fixed estimands. The copula martingale posterior has coverage of 0.93 to 1.00 at 11 to 55 minutes per dataset (Reviewers rdDB and PTiz).
  • Other CFMs and the exact posterior. CausalFM (Reviewer pQ8X) and Do-PFN fail the same way, and so does unit-level aggregation of exact marginals (Reviewer PTiz).

Trade-offs. CausalPFN's point estimates are the most accurate, but its calibrated intervals are too narrow for large groups, and without calibration they are too wide for small ones. The heads widen their intervals to about the width needed and are less accurate on three benchmarks; however, keeping CausalPFN's estimate and learning only the variance achieves CausalPFN's accuracy, 98% full-sample coverage, and the lowest interval score in the matched-budget comparison.

Scope. As Reviewer PTiz notes, the contribution is methodological. The heads do not reach nominal coverage on every benchmark (0.71 on ACIC 2016 for the CNP head). The corrections we tested in this work are non-adaptive, so a correction that estimates the error correlation for each dataset is not ruled out. Building SCM-marginalized aggregation fully into the pre-training of CFMs is left to future work.

Revision. The abstract and introduction now state the takeaway as a simple procedure: a small head on a frozen CFM, trained on simulated tasks, predicts a credible interval for any group average, centered either on its own estimate or on the CFM's. Section 5.2 adds the result with CausalPFN's estimate and the trade-off, and Section 6 the scope of the corrections. Appendix F opens with a summary of its conclusions and is grouped into five subsections. Some appendix tables and figure numbers changed as a result. The matched-budget results remain Tables 14 and 15 and Figure 8.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.