- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper addresses an important limitation of current causal effect foundation models (CE-FMs): models such as Do-PFN and CausalPFN are designed primarily for binary treatments, while many real-world decisions involve a continuous treatment dose, intensity, or exposure. The authors propose TAP-CEFM, a lightweight trainable plugin that adapts a frozen binary-treatment CE-FM to estimate continuous dose-response functions. The central design consists of three components, Treatment-anchor construction, Treatment-aware input plugin, Adaptive Gaussian aggregation. A trainable encoder combines the original covariates, a treatment/anchor embedding, and covariate–treatment interactions. The pretrained causal foundation model remains frozen, while the plugin is trained using weighted factual prediction loss and a smoothness regularizer. Multiple anchor-specific predictions are combined, with greater weights assigned to anchors closer to the queried treatment. The bandwidth adapts to the local spacing between retained anchors. The authors evaluate TAP-CEFM using Synthetic, IHDP, and News benchmarks, with Do-PFN and CausalPFN as the frozen backbones. The reported results show that TAP-CEFM with CausalPFN achieves the lowest mean AMSE among the compared methods across the reported settings.
Critical strengths
- It addresses a genuine gap in causal foundation models. Binary treatment models are not directly suitable for estimating a full dose-response curve. Simply replacing a continuous dose with a binary label loses distinctions between treatment values within each group. The paper correctly identifies this problem and attempts to retain the continuous information in a learned representation while preserving the foundation model's binary interface. This is a worthwhile direction because foundation models become more useful when they can be adapted to new causal estimands and treatment types, rather than being restricted to the exact tasks represented during pretraining.
- The frozen-backbone design is practical. Keeping the causal foundation model frozen reduces the need for expensive foundation-model retraining and potentially limits catastrophic changes to its pretrained representations. The task-specific plugin offers a relatively compact adaptation mechanism. The paper also evaluates the approach with two different backbones, Do-PFN and CausalPFN. The fact that the same broad design can be applied to both is a positive indication of modularity, although the difference in their results also shows that the method's performance depends substantially on the selected backbone.
- The method combines local learning with global integration. The method has a coherent high-level structure, Train each anchor model using observations concentrated around its anchor. Preserve treatment variation within each anchor through the encoder. Combine predictions from multiple anchors instead of relying on one threshold. This addresses a genuine tension in continuous-treatment estimation. Very local models may have low bias but high variance, whereas broad neighborhoods may improve stability at the cost of local approximation error. Aggregation offers a plausible compromise.
Critical weaknesses
- Factual prediction loss does not guarantee causal response estimation. The plugin is trained using observed outcomes and a locally weighted factual squared-error loss. But a continuous-treatment causal model aims to estimate \mu(x,t) for treatment levels that an individual may not have received. Good factual prediction does not automatically imply accurate counterfactual response estimation. The paper states standard consistency, conditional ignorability, and positivity assumptions. These are appropriate for identification, but their presence in the preliminaries does not demonstrate that the proposed learned representation and training procedure adequately address confounding or weak treatment overlap. The central question is: what does the pretrained causal model contribute that makes the learned response curve more causally reliable than a conventional outcome model? The foundation model's causal priors may help, but the paper does not appear to directly test whether these priors improve identification-sensitive aspects of estimation.
- The binary anchor construction may create artificial treatment discontinuities. The encoder retains continuous treatment information, which is a sensible mitigation. Nevertheless, the frozen backbone still receives a binary treatment input whose value changes abruptly at the anchor. The smoothness regularizer penalizes second differences of predictions at neighboring doses. However, a second-difference penalty does not guarantee continuity or differentiability, and it may not adequately control a jump at an anchor. Its effectiveness also depends on the increment \delta, the regularization coefficient, and how the neighboring doses are selected. This matters because continuous-treatment estimation requires a meaningful response curve, and artificial discontinuities can distort both local effects and downstream dose selection.
- The theoretical results are weaker than the empirical framing suggests. Theorem 1 gives a local error bound based on Lipschitz continuity of the true response and the anchor model's prediction. Theorem 2 gives an aggregation inequality, and Theorem 3 combines the two. These results are mathematically reasonable, but their implications should not be overstated. First, the bounds depend on the reference error, the response smoothness constant, and the model smoothness constant. The paper does not establish that these quantities are small in practice or provide a procedure to estimate them. Second, the error analysis does not establish that Gaussian weighting is optimal. A bound depending on distance from the anchor motivates local weighting, but it does not uniquely imply a Gaussian kernel or the particular adaptive bandwidth. Third, the results are not causal identification guarantees. They analyze prediction error relative to the true conditional response under regularity assumptions; they do not establish that the observed-data training procedure recovers that response under confounding, limited overlap, or misspecified priors.
Questions for authors
- What is the precise causal interpretation of the binary label when the target is a continuous dose-response function? Why should a binary-treatment CE-FM trained on threshold-based labels retain the correct causal semantics after its inputs are transformed by the plugin?
- How much of the performance improvement comes from the pretrained causal backbone, rather than the trainable encoder, local weighting, and Gaussian aggregation? Can the authors provide matched-capacity comparisons with a noncausal frozen backbone and a conventional continuous-treatment neural model?
- How does TAP-CEFM behave as confounding strength increases or treatment overlap deteriorates? Can the authors demonstrate that its advantage persists when some covariate subgroups have very limited support at particular treatment levels?
5 comments
Weakness 3: Discussion on the practical interpretation of the theoretical error bounds, and the justification for Gaussian weighting selection.
Answer: Thank you for your insightful observation.
We clarify that the primary purpose of our theoretical results is to motivate the design of TAP-CEFM by identifying the factors that should be optimized. Specifically, Theorem 3 (Section 3.2, Eq. (21)) shows that the final prediction error bound consists of two main components: the reference error and the distance-dependent term.
-
First, the reference error measures the prediction error of an anchor model at its reference anchor (Eq. (4)). To reduce this error, we design an encoder that preserves continuous treatment information and combines it with the original covariates, while employing a pretrained CEFM with strong causal priors and predictive capabilities as the frozen backbone. These two components work together to improve the anchor-level predictions and thereby target the reduction of the reference error.
-
Second, when the magnitude of the response smoothness constant and the model smoothness constants are fixed, reducing the distance directly reduces the corresponding distance-dependent term in Theorem 3. This motivates assigning greater weights to anchor models closer to the queried treatment, thereby guiding the design of our local aggregation mechanism.
-
We adopt Gaussian weighting as a practical implementation of distance-based local aggregation because it obeys our motivation, widely used in aggregation. To further examine this design choice, we conduct supplementary comparisons with other alternative local weighting strategies, while keeping the remaining experimental settings unchanged.
The results are reported in the following table. Gaussian weighting achieves the lowest estimation error on both IHDP and the synthetic dataset with . These results demonstrate the empirical advantage of Gaussian local weighting.
| Weighting Strategy | IHDP | Synthetic () |
|---|---|---|
| Gaussian | ||
| Uniform | ||
| No aggregation |
Question 1: Discussion on causal interpretation of the binary label and the causal semantics of our inputs.
Answer: Thank you for your helpful comment.
The binary label represents an anchor-specific treatment status, indicating whether an individual receives a treatment dose below or above a given threshold. Our plugin transforms the original continuous-treatment task into an anchor-specific binary causal task, thereby enabling the frozen CEFM to retain its binary causal prediction semantics.
Question 2: Discussion on the contribution of pretrained causal backbone.
Answer: Thank you for your insightful question.
To quantify the contribution of the pretrained causal backbone, we conduct additional controlled comparisons while keeping other mechanism and tuning budget unchanged. Specifically, we replace the frozen CEFM with a capacity-matched generic pretrained regressor and a conventional MLP predictor, respectively.
The results are reported in the following table. TAP-CEFM with the frozen CausalPFN achieves the lowest estimation error, demonstrating the advantage of the pretrained causal priors over generic predictive models.
| Method | IHDP |
|---|---|
| Frozen LIMIX-16M | |
| MLP (Full) | |
| CausalPFN (Full) |
Question 3: Discussion on the robustness of TAP-CEFM under different confounding strengths and treatment overlap conditions.
Answer: Thank you for your valuable comment.
Our original Synthetic experiments already examine settings relevant to confounding and overlap (Table 1; Appendix C.1). To further evaluate the robustness of TAP-CEFM under different confounding strengths and treatment overlap conditions, we conduct additional experiments by varying the treatment assignment mechanism. Specifically, we reuse the covariate score function defined in our original synthetic data generation process (Appendix C.1, Eq. (30)), and divide the samples into four subgroups according to this score. The actual treatment is then generated as
We evaluate TAP-CEFM under different combinations of and , with 50 repetitions for each setting.
The overall results are reported in the following table. TAP-CEFM consistently achieves the lowest Mean across all seven experimental settings. In particular, its advantage persists as the covariate-dependent component of treatment assignment increases and as the stochastic variation in treatment assignment decreases. These results demonstrate the robustness of TAP-CEFM under the considered confounding and treatment assignment conditions.
| VCNet | ORIC | TAP-CEFM | ||
|---|---|---|---|---|
Furthermore, we report the ADRF estimation errors for the four score-based subgroups in the following table. TAP-CEFM achieves the lowest overall RMSE across all seven settings. These subgroup-level results further demonstrate the effectiveness of TAP-CEFM under heterogeneous treatment assignment patterns.
| Method | Group 1 | Group 2 | Group 3 | Group 4 | ||
|---|---|---|---|---|---|---|
| VCNet | 0.3360 | 0.4243 | 0.3721 | 0.2940 | ||
| ORIC | 0.2520 | 0.3780 | 0.2928 | 0.2145 | ||
| TAP-CEFM | 0.2151 | 0.2962 | 0.2405 | 0.2270 | ||
| VCNet | 0.1851 | 0.2390 | 0.1760 | 0.1203 | ||
| ORIC | 0.1610 | 0.2410 | 0.1686 | 0.1094 | ||
| TAP-CEFM | 0.1239 | 0.1740 | 0.1187 | 0.1064 | ||
| VCNet | 0.1239 | 0.1559 | 0.1225 | 0.1329 | ||
| ORIC | 0.0945 | 0.1816 | 0.1298 | 0.1056 | ||
| TAP-CEFM | 0.1100 | 0.1324 | 0.0886 | 0.0832 | ||
| VCNet | 0.1406 | 0.1468 | 0.1146 | 0.1381 | ||
| ORIC | 0.0877 | 0.1900 | 0.1408 | 0.1505 | ||
| TAP-CEFM | 0.1015 | 0.1385 | 0.1049 | 0.1129 | ||
| VCNet | 0.1076 | 0.1247 | 0.0906 | 0.1114 | ||
| ORIC | 0.0901 | 0.1659 | 0.1150 | 0.1077 | ||
| TAP-CEFM | 0.1215 | 0.1273 | 0.0789 | 0.0839 | ||
| VCNet | 0.1618 | 0.1990 | 0.1410 | 0.1208 | ||
| ORIC | 0.0982 | 0.1946 | 0.1429 | 0.1059 | ||
| TAP-CEFM | 0.1199 | 0.1477 | 0.1030 | 0.0874 | ||
| VCNet | 0.1867 | 0.2269 | 0.1579 | 0.1110 | ||
| ORIC | 0.0998 | 0.2001 | 0.1462 | 0.1070 | ||
| TAP-CEFM | 0.1259 | 0.1646 | 0.1225 | 0.0930 |
Weakness 1: Discussion on TAP-CEFM’s counterfactual prediction capability, and the performance under confounding and limited treatment overlap.
Answer: Thank you for your thoughtful assessment.
The counterfactual prediction capability of TAP-CEFM is primarily provided by the pretrained causal effect foundation model (CEFM). CEFM is specifically designed and pretrained for counterfactual prediction, their pretraining takes counterfactual prediction as the learning objective. We achieve accurate counterfactual predictions by training our encoder plug-in with CEFMs serving as frozen predictor.
TAP-CEFM’s ability to account for confounding and limited treatment overlap is CEFM’s pretraining priors. Their pretraining priors explicitly incorporate causal structures and treatment assignment mechanisms relevant to confounding and limited overlap. For example, CausalPFN’s prior generation process considers different degrees of confounding and treatment overlap through covariate-dependent treatment assignment mechanisms [a]. Using these pretrained causal priors, our method can handle confounding and limited treatment overlap in the constructed anchor-specific tasks.
Weakness 2: Discussion on potential abrupt changes.
Answer: Thank you for the insightful comments.
From the multi-anchor perspective, as described in Section 3.2 (Eqs. (18)–(24)), our method aggregates predictions from multiple anchor models. This mechanism avoids relying on a single anchor model, thereby greatly reducing the influence of individual prediction discontinuities on the final response estimate.
From the perspective of an individual anchor model, the frozen backbone receives the learned representation zk(x,t) which preserves continuous treatment information.
Furthermore, we introduce a second-difference regularization term (Section 3.1, Eqs. (16)–(17)) to encourage similar prediction increments at neighboring treatment levels, thereby further reducing abrupt changes in local response slopes.
To investigate the effectiveness of the smoothness regularization and its sensitivity to the choice of hyperparameters, we conduct additional experiments by varying the treatment increment Δ and the regularization coefficient λs in Eqs. (16)–(17). Specifically, we examine how different choices of these two parameters affect the smoothness regularization and the resulting ADRF estimation performance.
The results are summarized in the following table. When the smoothness regularization coefficient λs=0.1 and the treatment increment Δ=0.02, the model achieves the lowest estimation error. These results provide empirical evidence of the effects of the finite-difference increment and regularization strength on controlling local prediction variations and mitigating discontinuities.
References
[a] Vahid Balazadeh Meresht et al. CausalPFN: Amortized causal effect estimation via in-context learning. Advances in Neural Information Processing Systems, 38:154945–154984, 2026.