acceptodds
Under review as a conference paper at ICLR 2027

Adapting causal effect foundation model from binary to continuous treatments

Abstract

Causal effects describe how outcomes differ under alternative interventions and play an important role in medicine, economics, and public policy. Accurate causal effect estimation is therefore essential for informed decision-making. Recently, several studies have introduced causal effect foundation models based on in-context learning, achieving promising performance in causal effect estimation on new tasks without updating the parameters of the foundation model. However, existing causal effect estimation foundation models are primarily designed for binary treatments and cannot be directly applied to continuous treatment settings. In this paper, to leverage the strong predictive capability of causal effect foundation models for binary treatments in continuous treatment settings, we propose a lightweight trainable plugin to adapt a frozen causal effect foundation model of binary treatments to continuous treatment effect estimation. Specifically, we use treatment quantiles as thresholds to binarize the continuous treatment, yielding a series of anchor treatment values. For each anchor, we learn a covariate–treatment interaction representation through a task-specific encoder and feed it into a binary-treatment foundation model, whose parameters are kept fixed while the encoder is fine-tuned. Since each anchor model is expected to provide more accurate counterfactual estimates for target treatments close to its corresponding anchor value, we aggregate predictions from multiple anchor models to obtain the final prediction, with larger weights assigned to anchors closer to the target treatment. We conduct experiments on synthetic and semi-synthetic benchmark datasets to demonstrate the effectiveness of our method for continuous treatment effect estimation.

Primary area:
causal reasoning
Keywords:
Continuous Treatment Effect Estimation, From binary to continuous, Causal Effect Foundation Models

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_xtMwOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper addresses an important limitation of current causal effect foundation models (CE-FMs): models such as Do-PFN and CausalPFN are designed primarily for binary treatments, while many real-world decisions involve a continuous treatment dose, intensity, or exposure. The authors propose TAP-CEFM, a lightweight trainable plugin that adapts a frozen binary-treatment CE-FM to estimate continuous dose-response functions. The central design consists of three components, Treatment-anchor construction, Treatment-aware input plugin, Adaptive Gaussian aggregation. A trainable encoder combines the original covariates, a treatment/anchor embedding, and covariate–treatment interactions. The pretrained causal foundation model remains frozen, while the plugin is trained using weighted factual prediction loss and a smoothness regularizer. Multiple anchor-specific predictions are combined, with greater weights assigned to anchors closer to the queried treatment. The bandwidth adapts to the local spacing between retained anchors. The authors evaluate TAP-CEFM using Synthetic, IHDP, and News benchmarks, with Do-PFN and CausalPFN as the frozen backbones. The reported results show that TAP-CEFM with CausalPFN achieves the lowest mean AMSE among the compared methods across the reported settings.

Critical strengths
  1. It addresses a genuine gap in causal foundation models. Binary treatment models are not directly suitable for estimating a full dose-response curve. Simply replacing a continuous dose with a binary label loses distinctions between treatment values within each group. The paper correctly identifies this problem and attempts to retain the continuous information in a learned representation while preserving the foundation model's binary interface. This is a worthwhile direction because foundation models become more useful when they can be adapted to new causal estimands and treatment types, rather than being restricted to the exact tasks represented during pretraining.
  2. The frozen-backbone design is practical. Keeping the causal foundation model frozen reduces the need for expensive foundation-model retraining and potentially limits catastrophic changes to its pretrained representations. The task-specific plugin offers a relatively compact adaptation mechanism. The paper also evaluates the approach with two different backbones, Do-PFN and CausalPFN. The fact that the same broad design can be applied to both is a positive indication of modularity, although the difference in their results also shows that the method's performance depends substantially on the selected backbone.
  3. The method combines local learning with global integration. The method has a coherent high-level structure, Train each anchor model using observations concentrated around its anchor. Preserve treatment variation within each anchor through the encoder. Combine predictions from multiple anchors instead of relying on one threshold. This addresses a genuine tension in continuous-treatment estimation. Very local models may have low bias but high variance, whereas broad neighborhoods may improve stability at the cost of local approximation error. Aggregation offers a plausible compromise.
Critical weaknesses
  1. Factual prediction loss does not guarantee causal response estimation. The plugin is trained using observed outcomes and a locally weighted factual squared-error loss. But a continuous-treatment causal model aims to estimate \mu(x,t) for treatment levels that an individual may not have received. Good factual prediction does not automatically imply accurate counterfactual response estimation. The paper states standard consistency, conditional ignorability, and positivity assumptions. These are appropriate for identification, but their presence in the preliminaries does not demonstrate that the proposed learned representation and training procedure adequately address confounding or weak treatment overlap. The central question is: what does the pretrained causal model contribute that makes the learned response curve more causally reliable than a conventional outcome model? The foundation model's causal priors may help, but the paper does not appear to directly test whether these priors improve identification-sensitive aspects of estimation.
  2. The binary anchor construction may create artificial treatment discontinuities. The encoder retains continuous treatment information, which is a sensible mitigation. Nevertheless, the frozen backbone still receives a binary treatment input whose value changes abruptly at the anchor. The smoothness regularizer penalizes second differences of predictions at neighboring doses. However, a second-difference penalty does not guarantee continuity or differentiability, and it may not adequately control a jump at an anchor. Its effectiveness also depends on the increment \delta, the regularization coefficient, and how the neighboring doses are selected. This matters because continuous-treatment estimation requires a meaningful response curve, and artificial discontinuities can distort both local effects and downstream dose selection.
  3. The theoretical results are weaker than the empirical framing suggests. Theorem 1 gives a local error bound based on Lipschitz continuity of the true response and the anchor model's prediction. Theorem 2 gives an aggregation inequality, and Theorem 3 combines the two. These results are mathematically reasonable, but their implications should not be overstated. First, the bounds depend on the reference error, the response smoothness constant, and the model smoothness constant. The paper does not establish that these quantities are small in practice or provide a procedure to estimate them. Second, the error analysis does not establish that Gaussian weighting is optimal. A bound depending on distance from the anchor motivates local weighting, but it does not uniquely imply a Gaussian kernel or the particular adaptive bandwidth. Third, the results are not causal identification guarantees. They analyze prediction error relative to the true conditional response under regularity assumptions; they do not establish that the observed-data training procedure recovers that response under confounding, limited overlap, or misspecified priors.
Questions for authors
  1. What is the precise causal interpretation of the binary label when the target is a continuous dose-response function? Why should a binary-treatment CE-FM trained on threshold-based labels retain the correct causal semantics after its inputs are transformed by the plugin?
  2. How much of the performance improvement comes from the pretrained causal backbone, rather than the trainable encoder, local weighting, and Gaussian aggregation? Can the authors provide matched-capacity comparisons with a noncausal frozen backbone and a conventional continuous-treatment neural model?
  3. How does TAP-CEFM behave as confounding strength increases or treatment overlap deteriorates? Can the authors demonstrate that its advantage persists when some covariate subgroups have very limited support at particular treatment levels?
5 comments

Weakness 1: Discussion on TAP-CEFM’s counterfactual prediction capability, and the performance under confounding and limited treatment overlap.

Answer: Thank you for your thoughtful assessment.

  1. The counterfactual prediction capability of TAP-CEFM is primarily provided by the pretrained causal effect foundation model (CEFM). CEFM is specifically designed and pretrained for counterfactual prediction, their pretraining takes counterfactual prediction as the learning objective. We achieve accurate counterfactual predictions by training our encoder plug-in with CEFMs serving as frozen predictor.

  2. TAP-CEFM’s ability to account for confounding and limited treatment overlap is CEFM’s pretraining priors. Their pretraining priors explicitly incorporate causal structures and treatment assignment mechanisms relevant to confounding and limited overlap. For example, CausalPFN’s prior generation process considers different degrees of confounding and treatment overlap through covariate-dependent treatment assignment mechanisms [a]. Using these pretrained causal priors, our method can handle confounding and limited treatment overlap in the constructed anchor-specific tasks.

Weakness 2: Discussion on potential abrupt changes.

Answer: Thank you for the insightful comments.

  1. From the multi-anchor perspective, as described in Section 3.2 (Eqs. (18)–(24)), our method aggregates predictions from multiple anchor models. This mechanism avoids relying on a single anchor model, thereby greatly reducing the influence of individual prediction discontinuities on the final response estimate.

  2. From the perspective of an individual anchor model, the frozen backbone receives the learned representation zk(x,t)z_k(x,t) which preserves continuous treatment information.

  3. Furthermore, we introduce a second-difference regularization term (Section 3.1, Eqs. (16)–(17)) to encourage similar prediction increments at neighboring treatment levels, thereby further reducing abrupt changes in local response slopes.

To investigate the effectiveness of the smoothness regularization and its sensitivity to the choice of hyperparameters, we conduct additional experiments by varying the treatment increment Δ\Delta and the regularization coefficient λs\lambda_s in Eqs. (16)–(17). Specifically, we examine how different choices of these two parameters affect the smoothness regularization and the resulting ADRF estimation performance.

The results are summarized in the following table. When the smoothness regularization coefficient λs=0.1\lambda_s=0.1 and the treatment increment Δ=0.02\Delta=0.02, the model achieves the lowest estimation error. These results provide empirical evidence of the effects of the finite-difference increment and regularization strength on controlling local prediction variations and mitigating discontinuities.

λs\lambda_sΔ\DeltaAMSE\sqrt{\mathrm{AMSE}}
00.020.1343±0.02850.1343 \pm 0.0285
0.030.020.1240±0.03150.1240 \pm 0.0315
0.30.020.1124±0.03430.1124 \pm 0.0343
0.1\mathbf{0.1}0.02\mathbf{0.02}0.1106±0.0298\mathbf{0.1106 \pm 0.0298}
0.10.010.1273±0.03210.1273 \pm 0.0321
0.10.040.1190±0.03170.1190 \pm 0.0317
0.10.080.1355±0.03280.1355 \pm 0.0328

References

[a] Vahid Balazadeh Meresht et al. CausalPFN: Amortized causal effect estimation via in-context learning. Advances in Neural Information Processing Systems, 38:154945–154984, 2026.

Weakness 3: Discussion on the practical interpretation of the theoretical error bounds, and the justification for Gaussian weighting selection.

Answer: Thank you for your insightful observation.

We clarify that the primary purpose of our theoretical results is to motivate the design of TAP-CEFM by identifying the factors that should be optimized. Specifically, Theorem 3 (Section 3.2, Eq. (21)) shows that the final prediction error bound consists of two main components: the reference error ϵk(x;t)\epsilon_k(x;t) and the distance-dependent term.

  1. First, the reference error measures the prediction error of an anchor model at its reference anchor (Eq. (4)). To reduce this error, we design an encoder that preserves continuous treatment information and combines it with the original covariates, while employing a pretrained CEFM with strong causal priors and predictive capabilities as the frozen backbone. These two components work together to improve the anchor-level predictions and thereby target the reduction of the reference error.

  2. Second, when the magnitude of the response smoothness constant and the model smoothness constants are fixed, reducing the distance ∣t−τk∣|t-\tau_k| directly reduces the corresponding distance-dependent term in Theorem 3. This motivates assigning greater weights to anchor models closer to the queried treatment, thereby guiding the design of our local aggregation mechanism.

  3. We adopt Gaussian weighting as a practical implementation of distance-based local aggregation because it obeys our motivation, widely used in aggregation. To further examine this design choice, we conduct supplementary comparisons with other alternative local weighting strategies, while keeping the remaining experimental settings unchanged.

The results are reported in the following table. Gaussian weighting achieves the lowest estimation error on both IHDP and the synthetic dataset with β=0.5\beta=0.5. These results demonstrate the empirical advantage of Gaussian local weighting.

Weighting StrategyIHDPSynthetic (β=0.5\beta=0.5)
Gaussian0.2875±0.0312\mathbf{0.2875 \pm 0.0312}0.1066±0.0291\mathbf{0.1066 \pm 0.0291}
Uniform0.3476±0.03040.3476 \pm 0.03040.1159±0.02880.1159 \pm 0.0288
No aggregation0.6549±0.04200.6549 \pm 0.04200.2104±0.03860.2104 \pm 0.0386

Question 1: Discussion on causal interpretation of the binary label and the causal semantics of our inputs.

Answer: Thank you for your helpful comment.

The binary label represents an anchor-specific treatment status, indicating whether an individual receives a treatment dose below or above a given threshold. Our plugin transforms the original continuous-treatment task into an anchor-specific binary causal task, thereby enabling the frozen CEFM to retain its binary causal prediction semantics.

Question 2: Discussion on the contribution of pretrained causal backbone.

Answer: Thank you for your insightful question.

To quantify the contribution of the pretrained causal backbone, we conduct additional controlled comparisons while keeping other mechanism and tuning budget unchanged. Specifically, we replace the frozen CEFM with a capacity-matched generic pretrained regressor and a conventional MLP predictor, respectively.

The results are reported in the following table. TAP-CEFM with the frozen CausalPFN achieves the lowest estimation error, demonstrating the advantage of the pretrained causal priors over generic predictive models.

MethodIHDP
Frozen LIMIX-16M0.3042±0.04670.3042 \pm 0.0467
MLP (Full)0.6723±0.05010.6723 \pm 0.0501
CausalPFN (Full)0.2875±0.0312\mathbf{0.2875 \pm 0.0312}

Question 3: Discussion on the robustness of TAP-CEFM under different confounding strengths and treatment overlap conditions.

Answer: Thank you for your valuable comment.

Our original Synthetic experiments already examine settings relevant to confounding and overlap (Table 1; Appendix C.1). To further evaluate the robustness of TAP-CEFM under different confounding strengths and treatment overlap conditions, we conduct additional experiments by varying the treatment assignment mechanism. Specifically, we reuse the covariate score function sβ(X)s_\beta(X) defined in our original synthetic data generation process (Appendix C.1, Eq. (30)), and divide the samples into four subgroups according to this score. The actual treatment is then generated as

T=sigmoid⁡(αsβ(X)+σZ).T=\operatorname{sigmoid}\left(\alpha s_\beta(X)+\sigma Z\right).

We evaluate TAP-CEFM under different combinations of α\alpha and σ\sigma, with 50 repetitions for each setting.

The overall results are reported in the following table. TAP-CEFM consistently achieves the lowest Mean AMSE\sqrt{\mathrm{AMSE}} across all seven experimental settings. In particular, its advantage persists as the covariate-dependent component of treatment assignment increases and as the stochastic variation in treatment assignment decreases. These results demonstrate the robustness of TAP-CEFM under the considered confounding and treatment assignment conditions.

α\alphaσ\sigmaVCNetORICTAP-CEFM
000.50.50.3494±0.10450.3494\pm0.10450.2664±0.12710.2664\pm0.12710.2212±0.1491\mathbf{0.2212\pm0.1491}
0.50.50.50.50.1659±0.04450.1659\pm0.04450.1441±0.04240.1441\pm0.04240.1065±0.0281\mathbf{0.1065\pm0.0281}
110.50.50.1066±0.02980.1066\pm0.02980.1020±0.03920.1020\pm0.03920.0761±0.0276\mathbf{0.0761\pm0.0276}
220.50.50.1037±0.02430.1037\pm0.02430.1124±0.02800.1124\pm0.02800.0884±0.0275\mathbf{0.0884\pm0.0275}
11110.0859±0.01880.0859\pm0.01880.0893±0.03130.0893\pm0.03130.0761±0.0189\mathbf{0.0761\pm0.0189}
110.250.250.1261±0.03370.1261\pm0.03370.1118±0.03970.1118\pm0.03970.0859±0.0370\mathbf{0.0859\pm0.0370}
110.10.10.1416±0.03860.1416\pm0.03860.1144±0.04160.1144\pm0.04160.0994±0.0323\mathbf{0.0994\pm0.0323}

Furthermore, we report the ADRF estimation errors for the four score-based subgroups in the following table. TAP-CEFM achieves the lowest overall RMSE across all seven settings. These subgroup-level results further demonstrate the effectiveness of TAP-CEFM under heterogeneous treatment assignment patterns.

α\alphaσ\sigmaMethodGroup 1Group 2Group 3Group 4
000.50.5VCNet0.33600.42430.37210.2940
ORIC0.25200.37800.29280.2145
TAP-CEFM0.21510.29620.24050.2270
0.50.50.50.5VCNet0.18510.23900.17600.1203
ORIC0.16100.24100.16860.1094
TAP-CEFM0.12390.17400.11870.1064
110.50.5VCNet0.12390.15590.12250.1329
ORIC0.09450.18160.12980.1056
TAP-CEFM0.11000.13240.08860.0832
220.50.5VCNet0.14060.14680.11460.1381
ORIC0.08770.19000.14080.1505
TAP-CEFM0.10150.13850.10490.1129
1111VCNet0.10760.12470.09060.1114
ORIC0.09010.16590.11500.1077
TAP-CEFM0.12150.12730.07890.0839
110.250.25VCNet0.16180.19900.14100.1208
ORIC0.09820.19460.14290.1059
TAP-CEFM0.11990.14770.10300.0874
110.10.1VCNet0.18670.22690.15790.1110
ORIC0.09980.20010.14620.1070
TAP-CEFM0.12590.16460.12250.0930
Reviewer_QzhoOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The authors propose TAP-CEFM to adapt frozen binary-treatment causal foundation models to continuous treatment effect estimation. This framework selects treatment anchors based on empirical quantiles and creates a binary treatment indicator for each anchor. A trainable encoder captures continuous treatment information and interactions with covariates, while the foundation model remains frozen. Separate models are trained at each anchor using locally weighted prediction losses and filtered based on effective sample size. Their predictions are then combined using adaptive Gaussian weights to estimate the average dose–response function.

Critical strengths

I feel that the main strength lies in combining input adaptation for frozen causal foundation models with a local anchor ensemble for continuous treatments. It seems to be a interesting direction without changing the structure of the frozen causal foundation models. And the problem is well motivated. Extending existing foundation models without repeating full pretraining could have practical value.

Critical weaknesses
  1. The representation zk(x,t)z_k(x,t ) explicitly contains treatment information, while the binary label is also a deterministic function of treatment. Consequently, the constructed input is not a conventional binary causal task with pretreatment covariates. This does not invalidate outcome regression for the original continuous-treatment problem, but it complicates the claim that binary causal priors are being transferred. Matched comparisons against generic pretrained regressors and simpler adaptation strategies are missing.

  2. The treatment-dependent representation may violate the positivity assumption, and the transfer of causal guarantees is unclear. The method defines the binary treatment as $B_k=\mathbf{1} ( T\geq\tau_k ) $, while Equation 9 includes information about the same continuous treatment in the learned representation ZkZ_k. If ZkZ_k reveals which side of the threshold the treatment falls on, then P(Bk=1∣Zk)∈0,1P(B_k=1\mid Z_k)\in{0,1}, violating the positivity assumption required by CausalPFN's binary causal framework. Balancing the binary labels or filtering anchors based on marginal effective sample size (ESS) does not address this issue. Although this violation is not inevitable, the paper does not provide conditions or empirical evidence showing that sufficient overlap is maintained. The causal interpretation of treatment-dependent covariates also needs clarification. This concern does not necessarily invalidate continuous-treatment outcome estimation, but the original identification assumptions do not automatically justify the transformed binary tasks. Moreover, the current Lipschitz and aggregation bounds do not resolve this issue or establish that CausalPFN's original causal guarantees still apply.

Questions for authors
  1. Under what conditions does the learned representation satisfy binary positivity? If the binary treatment label can be recovered from the representation, how can the frozen model's predictions be interpreted causally? Please clarify how the adapted predictions relate to E[Y(t)∣X=x]\mathbb{E}[Y(t)\mid X=x], beyond the original continuous-treatment identification assumptions.

  2. Does the proposed method outperform generic pretrained regressors and simpler treatment-input adaptations when using the same adapter architecture, local weighting, aggregation method, and tuning budget?

3 comments

Weakness 1 and Question 2: Discussion on treatment-dependent representation and two experiments (predictor comparisons and adaptation strategy).

Answer: Thank you for your valuable suggestion.

To further investigate the contribution of the pretrained causal foundation model and the proposed adaptation strategy, we conduct two additional controlled experiments.

First, we replace the frozen CausalPFN with a generic pretrained regressor, LIMIX-16M, while keeping the proposed encoder architecture, local weighting, Gaussian aggregation, and tuning budget unchanged. The results are reported in the following table. Replacing CausalPFN with LIMIX-16M leads to a degradation in estimation performance, highlighting the contribution of the pretrained causal backbone.

Second, we retain the frozen CausalPFN but replace our proposed treatment-aware representation with a simpler MLP-based adaptation, defined as

zksimple(x,t)=W2,kReLU⁡(W1,k[x,t,τk,t−τk]+b1,k)+b2,k.z_k^{\mathrm{simple}}(x,t)=W_{2,k}\operatorname{ReLU}\left(W_{1,k}[x,t,\tau_k,t-\tau_k]+b_{1,k}\right)+b_{2,k}.

The results are reported in the following table. Under this setting, the estimation performance also deteriorates, demonstrating the advantage of our proposed treatment-aware representation over the simpler adaptation strategy.

MethodIHDP
Frozen LIMIX-16M0.3042±0.04670.3042 \pm 0.0467
CausalPFN + Simple Adapter0.3981±0.04920.3981 \pm 0.0492
TAP-CEFM (CausalPFN)0.2875±0.0312\mathbf{0.2875 \pm 0.0312}

Weakness 2: Discussion on positivity assumption, the causal interpretation of treatment-dependent covariates, and theoretical analysis.

Answer: Thank you for your insightful comments.

  1. The learned representation ZkZ_k cannot reveal which side of the threshold the treatment falls on. This is because ZkZ_k jointly depends on both the original covariates XX and the continuous treatment TT, such that the same representation may arise from different combinations of covariates and treatment values on either side of the threshold. Consequently, both binary treatment groups can share the same representation values, satisfying the positivity assumption.

To empirically investigate this issue, we further visualize the distributions of learned representations for the two binary treatment groups (Bk=0B_k=0 and Bk=1B_k=1) across anchor models. The results are presented in https://anonymous.4open.science/r/iclr-figure-8060/, where the representations of the two binary treatment groups exhibit a certain degree of overlap rather than being completely separated. These results provide empirical evidence regarding the representation-level overlap between the two groups.

  1. Regarding the causal interpretation, we clarify that BkB_k serves as an anchor-specific binary treatment input and ZkZ_k is an augmented input representation that jointly encodes the original covariates and continuous treatment information.

  2. Theorems 1–3 in Sections 3.1 and 3.2 establish prediction error bounds under the stated Lipschitz regularity conditions. These theoretical results are intended to motivate the local anchor construction and adaptive aggregation in our method.

Question 1: Discussion on positive assumption, and relationship between the predictions and E[Y(t)∣X=x]\mathbb{E}[Y(t)\mid X=x].

Answer: Thank you for your insightful question.

  1. When the learned representation zkz_k is non-invertible with respect to tt, the learned representations of the two groups (Bk=0B_k=0 and Bk=1B_k=1) overlap, as illustrated in https://anonymous.4open.science/r/iclr-figure-8060/. Consequently, both groups have positive probability in the overlapping representation space, thereby satisfying the positivity assumption:

0<P(Bk=1∣Zk=z)<1.0 < P(B_k=1\mid Z_k=z) < 1.

  1. By preserving the information required for conditional mean estimation through the adapted representation, our predictor can accurately approximate the corresponding regression function, thereby enabling the adapted predictions to approximate the target response E[Y(t)∣X=x]\mathbb{E}[Y(t)\mid X=x].

Thank you for addressing my concern. I raised my score accordingly.

Thank you for your insightful comments. We have revised the manuscript accordingly.

Reviewer_bp6cOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes TAP-CEFM to adapt frozen binary-treatment causal foundation models to continuous treatment effect estimation. Treatment quantiles define anchors for binarizing doses, while trainable encoders retain continuous dose information and its interactions with covariates. A separate plugin is trained for each anchor, and predictions are combined using adaptive Gaussian weights based on the queried dose. The method is evaluated with CausalPFN and DoPFN on three datasets.

Critical strengths
  1. Reusing existing binary causal foundation models without pretraining a new continuous-treatment model is a practically relevant adaptation problem.

  2. Specific improvements were made to address the information loss and lack of local samples caused by binarization.

  3. The experiment yielded good results.

Critical weaknesses
  1. This paper sequentially maps tt to the covariate representation zk(x,t)z_k(x,t), and the label bk(t)b_k(t) is also determined by the same tt. Does this constructed input still possess the semantic meaning of a binary causal task?

  2. Theorem 1 provides an upper bound on the error when the true response and the prediction function satisfy the Lipschitz condition; however, both the anchor error and the constant Lk,bL_{k,b} may vary depending on the model. Therefore, a smaller anchor distance does not necessarily imply a smaller error.

  3. The research on ablation has not been thorough enough. It does not distinguish between the benefits of causal prior transfer, pretraining, and ensemble learning.

Questions for authors
  1. Explain whether the input constructed in this paper still carries the semantic meaning of a binary causal task.

  2. Complete the ablation study.

2 comments

Weakness 1 and Question 1: Discussion on whether the treatment-dependent representation and binary label preserve the semantics of a binary causal task.

Answer: Thank you for your insightful comment.

The binary label bk(t)b_k(t) indicates the treatment status relative to anchor kk, while zk(x,t)z_k(x,t) encodes the covariate information and the sample’s relative distance from the anchor. This construction provides a binary-task representation tailored to continuous treatment estimation, connecting the continuous treatment setting with the binary input of CEFM.

Weakness 2: Discussion on how anchor-specific prediction errors and Lipschitz constants affect the interpretation of the error bound and the role of anchor distance.

Answer: Thank you for your insightful observation.

We clarify that Theorem 1 is primarily intended to motivate our method design by identifying the factors that should be optimized. Specifically, Eq. (5) decomposes the prediction error bound into two components: the reference error ϵk(x;t)\epsilon_k(x;t) and the distance-dependent term (Lμ(x)+Lk,bk(t)(x))∣t−τk∣(L_\mu(x)+L_{k,b_k(t)}(x))|t-\tau_k|. This error decomposition motivates two complementary design choices in TAP-CEFM.

  1. First, to reduce the reference error, which measures the prediction error of an anchor model at its reference anchor (Eq. (4)), we employ an encoder that preserves continuous treatment information, together with a pretrained CEFM that provides strong causal predictive priors. These components work collaboratively to improve anchor-level prediction accuracy.

  2. Second, the distance-dependent term can be uniformly bounded by (Lμ(x)+Lˉ(x))∣t−τk∣(L_\mu(x)+\bar{L}(x))|t-\tau_k|, indicating that, under this common bound, reducing the anchor distance reduces the corresponding component of the error bound. The distance-dependent component motivates assigning greater importance to anchor models closer to the target treatment.

Weakness 3 and Question 2: Discussion on distinguishing the contributions of causal prior transfer, generic pretraining, and multi-anchor aggregation through ablation studies.

Answer: Thank you for your helpful suggestion.

To distinguish the contributions of causal prior transfer and pretraining, we conduct additional comparisons by replacing the frozen CausalPFN with a generic pretrained predictor (LIMIX-16M) and a conventional MLP, respectively, while keeping the other components unchanged.

The results are presented in the following table. Replacing CausalPFN with LIMIX-16M leads to degraded performance, indicating the benefit of transferring causal priors. Moreover, replacing LIMIX-16M with a conventional MLP results in a further performance degradation, demonstrating the benefit of generic pretraining.

MethodIHDP
CausalPFN (Full)0.2875±0.0312\mathbf{0.2875 \pm 0.0312}
Frozen LIMIX-16M0.3042±0.04670.3042 \pm 0.0467
MLP (Full)0.6723±0.05010.6723 \pm 0.0501

Regarding ensemble learning, our original ablation study in Appendix E (Tables 3 and 4) already compares the full multi-anchor model with the single-anchor variant using both CausalPFN and DoPFN. The results consistently show that the single-anchor variant performs substantially worse than the full model, demonstrating the contribution of multi-anchor aggregation.

With CausalPFN as the frozen backbone, we also evaluate the other components of TAP-CEFM (Appendix E, Table 3). We remove the treatment adapter, feature interaction, local soft weighting, ESS-based anchor selection, and local smoothness loss one at a time. Additionally, we replace the adaptive Gaussian bandwidth with a fixed bandwidth. All six variants yield higher mean estimation errors, indicating that the components contribute to varying degrees.

Thank you for your insightful comments. We have revised the manuscript accordingly.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.