acceptodds
Under review as a conference paper at ICLR 2027

Data-informed Adaptation of Foundation Models for Causal Effect Estimation

Abstract

Due to the fundamental problem of causal inference – the impossibility of observing counterfactual outcomes – there is always a gap between the training goal (outcome, propensity) and the inference target (CATE, ATE). Traditional causal effect estimation revolves around setting assumptions, selecting identification strategies, and choosing causal estimators, while more recently, Causal Foundation Models (CFMs), built on Prior-Data-Fitted Networks (PFNs), ease this process by pretraining on massive synthetic prior datasets. Despite their success, such amortization makes causal inference less transparent, obscuring which assumptions the model implicitly relies on for a given dataset. The high diversity of real-world data-generating processes (DGPs) and Structural Causal Models (SCMs) makes it hard to know, a priori, which causal inference strategy is compatible with a given dataset. In this work, we focus on this compatibility between an observational dataset and method-specific assumptions, and find that geometric characteristics of data embeddings – specifically, the variogram – provide a useful signal of whether a given causal estimation strategy is appropriate. We develop MoE-Learner, a mixture-of-causal-experts framework that embeds a dataset under multiple assumption-specific views and uses the variogram as a counterfactual-free router to assess each expert's validity. Rather than eliminating causal assumptions, this shifts their validation from a fixed, a priori choice to a criterion that can be checked directly from data geometry. MoE-Learner achieves the best average rank on benchmark datasets (IHDP, ACIC, Lalonde). Being orthogonal to (causal) foundation models, our method builds a bridge from data characteristics to causal estimation strategy rather than proposing another CFM.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Causal Effect Estimation, Foundation Models, Geometry, Mixture of Experts, Test-Time Adaptation

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_2ULFOpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper proposes a causal foundation model based on mixture of experts architecture that takes different mechanism representations from a tabpfn model and learns a router on top of them. Empirical evaluation is conducted against other causal models, both amortized and non-amortized baselines and across various benchmarks.

Critical strengths

For wider adoption of causal foundation models I believe it is important we have better interpretability of their inference - this paper recognizes this need and takes first steps in that direction. I also appreciate that it reuses representations from an already trained tabpfn, which is potentially a more scalable approach than training a causal TFM from scratch.

Critical weaknesses

I have several concerns: Contribution: Contribution section is unnecessarily long, making it paradoxically less clear what is the novelty. To my understanding the motivations include 1) method-specific assumptions while at the same acknowledging that is not a concern for end-to-end inference amortizing foundation models which leave choice of method implicit and up to the model, 2) causal identification and inference separation which is already followed by CausalPFN and CausalFM and is not novel (admittedly DoPFN does not do this), or - generally - the motivation just conflates statistical modelling choices with assumptions required for causal identification, 3) fundamental problem of causal inference making counterfactual information unavailable while this is not actually true for synthetic data training of pfns (this is exactly why causal pfns exist and work so well), and 4) confusingly claims not to be yet another CFM after primarily positioning the work as so.

Methodology: 1) The paper does not convincingly explain why correlations between embedding distances and factual outcome differences should identify representations suitable for causal effect estimation - especially if contribution is built on TTA then simple correlation might not suffice (Representations preserving factual outcome similarity need not preserve the confounding or effect-modifying information required for accurate causal effect estimation). 2) MoE-Learner underperforms its own TabPFN backbone on IHDP and ACIC despite introducing substantial additional machinery which is a concern amplified by the fact that 3) the foundation-model comparisons lack sufficient information about matching pretrained checkpoints, inference configurations, and computational budgets so it is unclear if comparisons are fair. Finally I am left unconvinced by the ablations.

In sum, the contribution is in my view less substantial than presented, theoretical motivation is partially lacking and the empirical evidence does not convincingly establish the benefit of adapting tabpfn representation as such over simply using default tabpfn.

Questions for authors

For my major points please see the weaknesses section about. Additionally, could the authors please provide an anonymized code repository for inspection?

3 comments
AuthorsOpenReview ↗

Thank you for your comments!

Thanks for your thoughtful review. We fully agree with you that interpretability of causal inference is important, and our work is only an initial step. But we hope our work can shed light on future research. We address your concerns as follows.

  1. Contribution.
    Your summarization of our contributions is basically accurate. Specifically, our work addresses a question largely neglected by existing causal estimation methods: when counterfactual labels are unavailable, how can we understand whether, why, and when a causal estimation strategy is suitable for the observed data? To our best knowledge, no previous work has tackled this important problem. One earlier related work is [1]; however, as its title suggests, it is only an empirical analysis and still didn’t answer “why”. In this work, instead, we provide mechanistic interpretability - it examines the data geometry through representations and uses the alignment between representation distance and outcome difference to provide an interpretable, data-dependent basis for selection. We will reorganize the contribution section as numbered items following your summarization in the final version.

  2. Methodology and theorems.
    You are absolutely right to ask this question: “Why correlations between embedding distances and factual outcome differences should identify representations suitable for causal effect estimation?” Theorem 1 shows that (under the stated local smoothness conditions), nearby representations can support meaningful KNN-style outcome estimation. However, we are unable to (and didn’t) mathematically prove any theorem for the use of the Spearman correlation. Theorem 1 does not provide useful signal for the selection; and we clearly stated that the correlation is just a useful “empirical trend” (page 7). We also tested other metrics in Table 5. In conclusion, we proved what we were able to prove (Theorem 1), and left what we are unable to prove to empirical. But we will continue working on the theoretical part to try to fully answer your question above in future work.

  3. Performance.
    As shown in the supplementary Figures 5 and 6, lower performance than the TabPFN backbone is due to higher variance, but mostly comparable. However, does our method truly outperform a plain TabPFN? We had the same doubt as you. Therefore, we conducted two new experiments on front-door adjustment and Instrumental Variables (IV). First, we cloned the exact same datasets, codebase, and conda environment of CausalFM [2] and reproduced its scores. We also ran Do-PFN, CausalPFN, TabPFN and compare them with CausalFM. Finally, we ran our MoE-Learner using the same setting. The results are shown as follows.

Front-door

MethodsPEHE
(Linear) Pearl (2009)1.124 ± 0.28
(RF) Pearl (2009)1.364 ± 0.52
(NN) Pearl (2009)0.889 ± 0.38
Do-PFN1.274 ± 0.24
CausalFM0.846 ± 0.34
CausalPFN0.853 ± 0.34
TabPFN0.832 ± 0.33
MoE-Learner0.827 ± 0.33

Binary IV

MethodPEHE
KIV (Singh et al., 2019)0.454 ± 0.16
DRIV (Syrgkanis et al., 2019)0.531 ± 0.18
DeepIV (Hartford et al., 2017)0.427 ± 0.15
DeepGMM (Bennett et al., 2019)0.503 ± 0.20
DMLIV (Syrgkanis et al., 2019)0.479 ± 0.23
DFIV (Xu et al., 2021)0.709 ± 0.29
MRIV (Frauen & Feuerriegel, 2022)0.688 ± 0.21
Do-PFN0.523 ± 0.20
CausalFM0.422 ± 0.16
CausalPFN0.425 ± 0.10
TabPFN0.390 ± 0.13
MoE-Learner0.389 ± 0.13

Please note that while PFNs (CausalPFN and TabPFN) are not specifically designed for front-door and IV, they show competitive performance, especially TabPFN; although it is a general-purpose tabular foundation model, it outperforms all causal foundation models. However, our method, MoE-Learner, is even better than TabPFN on both front-door and IV. These new experimental results significantly strengthen our method, making it not only the first unified MoE framework for causal effect estimation but also a strong empirically validated estimator even with unobserved confounding.

AuthorsOpenReview ↗

Question 4

  1. Ablation and code.
    We ensure a fair comparison with other foundation models. For CausalPFN, we directly used their Table 1; moreover, we also reproduced their whole experimental pipeline using their official code in https://github.com/vdblm/CausalPFN/ and no significant difference was observed. For CausalFN and Do-PFN, their official codebases were adopted:
    https://github.com/yccm/CausalFM
    https://github.com/jr2021/Do-PFN
    The exact datasets and conda environments were used to run the two methods. We first reproduced their results in their papers to ensure the environments were correctly configured. Then we exported the exact data from CausalPFN and imported them into CausalFM/Do-PFN to produce the results. Importantly, since we also had doubts about whether we had run these foundation models correctly, we investigated recently published papers where these foundation models were used/tested. Please see page 40 of reference [2] and page 33 of reference [3]: Their reported numbers from CausalFM and/or Do-PFN are very close to the numbers we report in our paper. The code to reproduce our results is uploaded in a zip file.

[1] Empirical Analysis of Model Selection for Heterogeneous Causal Effect Estimation
[2] CausalPFN: Amortized Causal Effect Estimation via In-Context Learning
[3] Use What You Know: Causal Foundation Models with Partial Graphs

Having considered the authors' rebuttal and the concerns raised by the other reviewers, I am lowering my score to 1 (Clear rejection). Unfortunately, the rebuttal has not resolved the fundamental issues with this submission and, in several respects, has reinforced my concerns. The paper suffers from an insufficiently motivated and poorly differentiated contribution, serious conceptual confusion regarding causal identification and estimation, a lack of theoretical justification for its central methodological innovation, and empirical evidence that fails to convincingly demonstrate its value over substantially simpler alternatives. The additional experiments do not meaningfully address these shortcomings, while the authors explicitly acknowledge that their theoretical analysis does not justify the proposed routing criterion. Taken together, I no longer consider these merely weaknesses in an otherwise promising contribution, but fundamental shortcomings in the scientific soundness and significance of the work. I therefore do not believe the paper meets the standards for acceptance at ICLR.

Reviewer_ChLUOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper addresses the challenge of selecting the right causal estimator when the data-generating process is unknown and counterfactual ground truth is unavailable, by proposing the MoE-Learner framework. It retrofits a tabular foundation model into four distinct causal experts and uses a ground-truth-free router based on the variogram (local smoothness of embeddings) to adaptively select the best expert, thereby eliminating the need for prior method-specific assumptions or a validation set and enabling interpretable, data-driven model selection that outperforms traditional and foundation model baselines across diverse benchmarks. However, the limitations are also significant: the framework does not address unobserved confounding, strictly relying on the unconfoundedness assumption so that routing among experts based on biased factual outcomes in the presence of hidden confounders will inevitably yield biased causal estimates without any diagnostic or mitigation mechanism such as sensitivity analysis or instrumental variables. Furthermore, the theoretical contribution is essentially incremental, as the four "causal experts" are fundamentally traditional estimators (S-Learner, IPW, T-Learner, matching) retrofitted with TabPFN; the paper does not propose a new identification strategy or estimation theory but rather an algorithm-selection framework that inherits the standard limitations of these classical methods, such as sensitivity to positivity violations and reliance on correct propensity or outcome modeling.

Critical strengths

The paper presents several key strengths. First, it introduces a novel unified framework that integrates four diverse causal modeling strategies into a single architecture, mitigating the long-standing dilemma of having to commit to a single, potentially misspecified method-specific assumption a priori. Second, the expert design is reasonable and mathematically sound: the use of the variogram as a ground-truth-free router is rigorously formalized, circumventing the "no validation set" problem inherent to counterfactuals while ensuring interpretability. Furthermore, the framework is convenient and modular, as it retrofits a general-purpose tabular backbone into four experts that directly mirror classic estimators, allowing seamless integration without requiring a specialized causal foundation model. Finally, comprehensive empirical validation across diverse benchmarks demonstrates that the framework achieves good performance, outperforming traditional estimators and causal foundation models by adaptively selecting the most suitable expert for different datasets, tasks, and treatment arms.

Critical weaknesses

The paper has several critical weaknesses that limit its practical impact and theoretical novelty. First, the framework operates strictly under the unconfoundedness assumption and completely ignores hidden confounders. If unobserved confounding exists, the router will inevitably select the "best" expert based on biased factual outcomes, yielding fundamentally flawed causal estimates without any diagnostic or sensitivity analysis mechanism. Second, the methodological contribution is essentially incremental. The four "causal experts" are simply traditional estimators (e.g., S-Learner, IPW, T-Learner) retrofitted with TabPFN embeddings, meaning the framework inherits standard limitations such as sensitivity to positivity violations and reliance on correct propensity or outcome modeling. Third, the variogram-based router relies on a strong local smoothness assumption. In highly non-smooth, noisy, or discontinuous datasets, the Spearman correlation may become uninformative, leading the router to select suboptimal experts without any robustness guarantees. Finally, the framework's success is heavily contingent on TabPFN's embedding quality, yet the paper lacks a systematic ablation replacing TabPFN with other tabular foundation models or strong non-foundation baselines. It remains unclear whether the performance gains stem from the MoE routing architecture itself or simply from the underlying power of TabPFN.

Questions for authors

I have several questions whose answers would materially affect my assessment. First, regarding the lack of theoretical novelty, since the four "causal experts" are essentially traditional estimators (e.g., S-Learner, IPW) retrofitted with TabPFN, could the authors clarify if there is any fundamental innovation in the identification strategy or estimation theory, or if this is purely an algorithm-selection framework? Second, regarding unobserved confounding, since the router evaluates experts based on factual outcomes, how would the framework behave if hidden confounders were present? Could you provide a sensitivity analysis indicating how much hidden bias the routing mechanism can tolerate before fundamentally failing? Third, concerning the routing criterion, the framework relies on a strong local smoothness assumption. How robust is the Spearman correlation in highly non-smooth or noisy tabular datasets, and are there any guarantees beyond the population-level bound in Theorem 1? Finally, regarding the backbone ablation, since the four experts heavily rely on TabPFN's embeddings, could you provide results replacing TabPFN with another foundation model (e.g., TabICL) or a strong non-foundation baseline (e.g., XGBoost)? This would clarify whether the performance gains stem from the MoE routing architecture itself or merely from TabPFN's representational power.

2 comments
AuthorsOpenReview ↗

New experimental results on front-door and IV with MoE-Learner being top-1 (code uploaded)

Thank you for your insightful review. We especially appreciate your comment on unobserved confounding, which gives us an opportunity to extend our method to broader applications. Below, we address your concerns point-by-point.

  1. Unobserved confounding.
    You are right that the current paper does not handle unobserved confounding. This is because we strictly follow the setting of the CausalPFN [1], which only considers the 4 standard datasets (back-door only), and unobserved confounding is out of scope. However, per your request, we have added two new experiments on Front-door adjustment and instrumental variables (IV). First, we cloned the exact same datasets, codebase, and conda environment as CausalFM [2] and reproduced its scores. We also ran Do-PFN, CausalPFN, and TabPFN and compared them with CausalFM. Finally, we ran our MoE-Learner using the same setting. The results are shown as follows.

Front-door

MethodsPEHE
(Linear) Pearl (2009)1.124 ± 0.28
(RF) Pearl (2009)1.364 ± 0.52
(NN) Pearl (2009)0.889 ± 0.38
Do-PFN1.274 ± 0.24
CausalFM0.846 ± 0.34
CausalPFN0.853 ± 0.34
TabPFN0.832 ± 0.33
MoE-Learner0.827 ± 0.33

Binary IV

MethodPEHE
KIV (Singh et al., 2019)0.454 ± 0.16
DRIV (Syrgkanis et al., 2019)0.531 ± 0.18
DeepIV (Hartford et al., 2017)0.427 ± 0.15
DeepGMM (Bennett et al., 2019)0.503 ± 0.20
DMLIV (Syrgkanis et al., 2019)0.479 ± 0.23
DFIV (Xu et al., 2021)0.709 ± 0.29
MRIV (Frauen & Feuerriegel, 2022)0.688 ± 0.21
Do-PFN0.523 ± 0.20
CausalFM0.422 ± 0.16
CausalPFN0.425 ± 0.10
TabPFN0.390 ± 0.13
MoE-Learner0.389 ± 0.13

Please note that while PFNs (CausalPFN and TabPFN) are not specifically designed for front-door and IV, they show competitive performance, especially TabPFN; although it is a general-purpose tabular foundation model, it outperforms all causal foundation models. However, our method, MoE-Learner, is even better than TabPFN on both front-door and IV. These new experimental results significantly strengthen our method, making it not only the first unified MoE framework for causal effect estimation but also a strong empirically validated estimator even with unobserved confounding. Initially, unobserved confounding is stated as a major limitation and left for future work (see the Limitations section). Please kindly let us know if you want us to add it to the paper (which is not the main goal of the current work and may potentially overcomplicate the current work).

  1. Methodological contribution.
    To our best knowledge, our work is the first paper that addresses a question largely neglected by existing causal estimation methods: when counterfactual labels are unavailable, how can we understand whether, why, and when a causal estimation strategy is suitable for the observed data? It examines the data geometry through representations and uses the alignment between representation distance and outcome difference to provide an interpretable, data-dependent basis for selection. No previous methods are able to achieve this. Empirically, our method has the best average rank in the back-door scenario and is the best in front-door and IV scenarios.

  2. Variogram-based router.
    In highly non-smooth, noisy, or discontinuous datasets, no existing method can reliably deliver good estimation. However, our method is the only method that can provide a diagnostic breakdown of the failure by revealing the underlying geometry. In contrast, previous methods usually silently fail without letting us know why. So your concern actually points out a strength of our method.

  3. Non-TabPFN embeddings.
    We select TabPFN as the backbone for two reasons. First, it achieves the best out-of-the-box performance among all tabular foundation models we tested; second, it has an official component for embedding extraction. The embedding extraction from a tabular foundation model is non-trivial and requires sophisticated cross-validation. However, TabICL does not have this functionality built in. To avoid unfair comparison, we did not include TabICL embedding into comparison. In future work, we plan to test with more tabular backbones and hyper-parameters, e.g., the number of folds.

Thank you again for your careful review, and we look forward to further discussion/any other concerns!

[1] CausalPFN: Amortized Causal Effect Estimation via In-Context Learning
[2] Foundation Models for Causal Inference via Prior-Data Fitted Networks

Reviewer_ChLUOpenReview ↗

Response for 16876

The additional experiments lack theoretical support, and more fundamentally the contribution itself remains insufficient. What the authors claim as a contribution, handling the common situation where counterfactual labels are unavailable, is not a new problem formulation or theoretical breakthrough; the claimed “first” is combinatorial rather than problem-level novelty. The added front-door and IV results do not change this: front-door 0.827 vs. TabPFN 0.832 and IV 0.389 vs. 0.390, both within the standard deviations, and the general-purpose TabPFN already outperforms all causal-specific models, so there is neither a significant gain nor evidence for causal-specific design. The response also does not explain how MoE-Learner exploits the mediator or the IV, nor does it state or test the identification assumptions. Without these, the new experiments merely run the same model on new datasets and cannot support extending to unobserved confounding. The contribution therefore remains insufficient. Regarding the variogram-based router, being able to diagnose failure is a feature rather than a contribution: it does not improve estimation accuracy or resolve when to use which estimator, so please give a concrete example and soften “unique” to “to our knowledge, one of the few.” Regarding non-TabPFN embeddings, the rationale is acceptable, but please revise “avoiding unfair comparison” to “not included because TabICL does not provide an official embedding interface, with this limitation noted in the appendix,” and add preliminary results with more backbones.

Reviewer_gcvDOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

The paper claims a causal effect estimation method based on the mixture-of-experts (MoE) concept. It's hard to understand enough to write a proper summary.

Critical strengths

The figures are illustrative.

Critical weaknesses
  1. No motivation for the approach. The paper does not explain why one should estimate effects this way rather than with existing estimators or CATE model-selection methods. The method is unnecessarily complicated with unclear gains.

  2. Unclear assumptions and inputs. The paper never clearly states the input, the assumptions (including identification assumptions), or the output of the method.

  3. The S/T-learner differences called "method-specific assumptions" (L84-88) are modeling choices, not assumptions.

  4. Ignorability given the embedding. The method replaces X with an embedding Z and treats E[Y | Z, T=t] as mu_t(x) (L279-297, L316). Ignorability given X does not imply ignorability given Z, and even then this identifies E[Y_1 - Y_0 | Z], not tau(x).

  5. Weak results. MoE-Learner is best on 1 of 4 datasets for CATE and 0 of 4 for ATE (Table 1), and worse than its own backbone TabPFN on IHDP and ACIC CATE. A single fixed Response Expert (Table 4) matches or beats it on CATE.

  6. Theorem 1 is not well defined. Notations for understanding Theorem 1 are not well-defined.

  7. Poor citations. Several citations do not support their claims, e.g., BART is cited to Chan et al. (2022), a paper on in-context learning in transformers, and the MoE "natural fit" claim is cited to Kunzel et al. (2019), a metalearner paper.

Questions for authors
  1. State the identification assumptions formally. For each expert, under what conditions does E[Y | Z, T=t] equal mu_t(x)?
  2. Does the router beat the best fixed expert and a cross-fitted factual-loss selector over the same experts?
2 comments
AuthorsOpenReview ↗

Unacceptable accusation and possible violation of reviewer policy

Thanks for your review questions; however, we respectfully reject your accusation that our paper is a “typical, unprepared AI slop”, which is such a serious (and baseless) defamation of our work ethic and professionalism. It is hard to imagine that you made this unsupported accusation, and even refused to give any concrete evidence.

We had honestly reported the AI use statement in the paper that AI tools are only used for grammar checking and polishing (e.g., Grammarly). The methodology development, coding, experiments, drafting, and plotting are all done by humans.

Upon AC/PC’s consent, we are going to publicly release the following evidence:

  • Data, codebase, Jupyter Notebooks, and raw Excel files.
  • Full AI chat history.
  • Full Git history of the codebase.
  • Full Git history of the manuscript.
  • Photos of ~100 pages of handwritten notes/manuscripts showing how the methodology was progressively developed over a long time by hand, not by AI.

In the meantime, we want to challenge you to provide evidence for your false accusation. If it is proven that our results are authentic and by humans, your formal apology is warranted. Our academic integrity does not allow such an unfair accusation.

Please also note that, by conference policy, reviewers who wrote such misrepresentations like yours will also face consequences, including the desk rejection of all their submitted papers or restrictions on submission to future conferences.


Point-to-point answer to your questions:

  1. Illustration.
    We appreciate that you found our figures illustrative. We spent a lot of effort plotting the most understandable figures. Hope these can help with your understanding of our framework.

  2. Motivation.
    Our work addresses a question largely neglected by existing causal estimation methods: when counterfactual labels are unavailable, how can we understand whether, why, and when a causal estimation strategy is suitable for the observed data? To our best knowledge, no previous work has tackled this important problem.

  3. Assumptions.
    You are right that those are not “assumptions”, and we didn’t call them so. Instead, we categorized them all as "method-specific assumptions" in this paper. Many methods require additional causal assumptions beyond the common ones like Unconfoundedness.

  4. Ignorability.
    You are right that “Ignorability given X does not imply ignorability given Z”, and we didn’t claim so (ignorability given the embedding) in the paper.

  5. The results are enhanced by additional experiments.
    We have added two new experiments on Front-door adjustment and instrumental variables (IV). First, we cloned the exact same datasets, codebase, and conda environment of CausalFM [2] and reproduced its scores. We also ran Do-PFN, CausalPFN, and TabPFN and compared them with CausalFM. Finally, we ran our MoE-Learner using the same setting. The results are shown as follows.

Front-door

MethodsPEHE
(Linear) Pearl (2009)1.124 ± 0.28
(RF) Pearl (2009)1.364 ± 0.52
(NN) Pearl (2009)0.889 ± 0.38
Do-PFN1.274 ± 0.24
CausalFM0.846 ± 0.34
CausalPFN0.853 ± 0.34
TabPFN0.832 ± 0.33
MoE-Learner0.827 ± 0.33

Binary IV

MethodPEHE
KIV (Singh et al., 2019)0.454 ± 0.16
DRIV (Syrgkanis et al., 2019)0.531 ± 0.18
DeepIV (Hartford et al., 2017)0.427 ± 0.15
DeepGMM (Bennett et al., 2019)0.503 ± 0.20
DMLIV (Syrgkanis et al., 2019)0.479 ± 0.23
DFIV (Xu et al., 2021)0.709 ± 0.29
MRIV (Frauen & Feuerriegel, 2022)0.688 ± 0.21
Do-PFN0.523 ± 0.20
CausalFM0.422 ± 0.16
CausalPFN0.425 ± 0.10
TabPFN0.390 ± 0.13
MoE-Learner0.389 ± 0.13

Please note that while PFNs (CausalPFN and TabPFN) are not specifically designed for front-door and IV, they show competitive performance, especially TabPFN; although it is a general-purpose tabular foundation model, it outperforms all causal foundation models. However, our method, MoE-Learner, is even better than TabPFN on both front-door and IV. These new experimental results significantly strengthen our method, making it not only the first unified MoE framework for causal effect estimation but also a strong empirically validated estimator even with unobserved confounding.

  1. Theorem 1.
    Please note that Theorem 1 is more about geometry than about causality. It shows that under the stated local smoothness conditions, nearby representations can support meaningful KNN-style outcome estimation, with a bound relating the variogram to representation distance. As a geometric analysis, it may not be commonly seen in the causal literature, but it brings geometric analysis to the causal context, which is our unique strength.
AuthorsOpenReview ↗

Questions 7 to 9

  1. Citation.
    All citations were manually added and verified. For example, please see page 6 of the CausalPFN paper [1], it cited BART in the same way as we do. We directly followed how CausalPFN cited BART.

  2. Identification assumptions.
    We follow the standard common assumptions (Consistency and SUTVA, Conditional exchangeability, and Positivity). You are right that we didn’t write them down mathematically (we only briefly mentioned them on page 2). We will add them in the Appendix in the final version.

  3. Ablations.
    As shown in Table 4 (page 15), the best single expert (i.e., Response Expert) can sometimes be very bad: ATE error is 0.30±0.14 on Lalonde CPS, while the fully-fledged MoE-Learner achieves 0.19±0.01. This is because the correlation in the variogram is worse for the Response Expert. This shows the unique strength of our framework: We don’t blindly select a method (or an expert); we can answer "why", by revealing the geometry (variogram). In our paper, cross-validation-based empirical method selection like [2] is out of scope. Splitting the data further, especially for small datasets, will hurt the training of the estimators. Moreover, empirical selection lacks the interpretability that our method has.

[1] CausalPFN: Amortized Causal Effect Estimation via In-Context Learning
[2] Empirical Analysis of Model Selection for Heterogeneous Causal Effect Estimation

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.