acceptodds
Under review as a conference paper at ICLR 2027

DAG-FM: A Foundation Model for Causal Discovery under Heterogeneous Causal Mechanisms

Abstract

Causal discovery from observational tabular data remains fundamentally challenging, primarily due to the heterogeneity of underlying causal mechanisms and the high-dimensional combinatorial search space of Directed Acyclic Graphs (DAGs). In this paper, we propose DAG-FM, a novel foundation model architecture that amortizes causal discovery. Unlike direct matrix prediction, DAG-FM decomposes the causal discovery process into two auto-regressive stages using two specialized Transformer-based sub-modules: a leaf-node predictor and a parent-node predictor. To effectively model complex row-column interactions, we adopt a robust tabular interaction block to output feature-wise representations. Crucially, to handle diverse and unknown Functional Causal Model (FCM) assumptions in real-world scenarios, we introduce Mixture-of-Leaf-Experts (MoLE), allowing the model to dynamically route and adapt to identifiable mechanism families. Through an iterative inference algorithm, DAG-FM seamlessly extracts causal orderings and constructs valid DAGs. Extensive experiments demonstrate that DAG-FM achieves state-of-the-art performance on both synthetic benchmarks and complex real-world datasets, significantly outperforming traditional classical algorithms and recent foundation models in both accuracy and scalability.

Primary area:
causal reasoning
Keywords:
causality, causal discovery, identifiability, functional causal model, foundation model, amortized inference

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_J1BzOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes DAG-FM, a foundation model for causal discovery under heterogeneous causal mechanisms. The method decomposes DAG identification into leaf prediction and parent prediction, implemented by two Transformer modules. A Mixture of Leaf Experts (MoLE) is introduced to accommodate different families of functional causal models (FCMs). The paper also establishes theoretical results on DAG identifiability and evaluates the method on synthetic and real datasets.

Critical strengths
  1. The decomposition into leaf prediction and parent prediction is intuitive and ensures that the resulting graph is a DAG.
  2. Considering heterogeneous causal mechanisms is a meaningful direction for causal discovery.
Critical weaknesses
  1. The connection between the theoretical results and the proposed method is unclear. My main concern is that the theoretical results are not closely related to the proposed method. The identifiability results characterize the prior space and appear to be applicable to other amortized causal discovery methods under the same assumptions. Moreover, DAG-FM does not explicitly use QPE or enforce the theoretical conditions during training or inference. Although MoLE is motivated by the diversity of FCM families, it is unclear how the theoretical results justify this particular architecture.

  2. The theoretical guarantees rely on restrictive assumptions. The identifiability results require additional assumptions, including the finite dimensional representation of QPEs and Assumptions A.2 and A.4. These conditions may limit the applicability of the results to more general causal mechanisms. Moreover, the training prior includes general causal mechanisms for which identifiability has not been established. The discretization procedure also raises questions about whether the smoothness assumptions in the theoretical analysis are satisfied.

  3. The experiments do not sufficiently isolate the contribution of the proposed architecture. The synthetic benchmarks are generated using the same prior construction framework as the training data. It is therefore unclear whether the improvements mainly come from the proposed architecture or the choice of training prior. It would be helpful to compare DAG-FM with other amortized causal discovery methods trained on the same prior space.

Questions for authors
  1. How do the theoretical results justify the specific architecture of DAG-FM?
  2. To what extent does the actual training prior satisfy the assumptions of Theorem 4.5?
  3. How would other amortized causal discovery methods perform when trained on the same prior space?
1 comment

We thank the reviewer for helping us clarify the connection between theory and practice, and for suggesting experimental controls to better isolate the architectural contribution.

W1: The connection between the theoretical results and the proposed method is unclear.

See C1 of the global rebuttal. Our contributions are complementary: the theory establishes sufficient conditions for identifiability under heterogeneous mechanisms, while DAG-FM introduces a mechanism-aware architecture for learning across these mechanisms.

Q1: The theory characterizes the prior space, while DAG-FM neither explicitly uses QPE nor enforces its conditions.

QPE serves as a proof tool, not an inference component. Its role is to establish when the observational distribution determines a unique DAG within the analyzed prior. A model need not explicitly compute QPE to learn this mapping, just as a statistical identifiability argument need not prescribe an estimation algorithm.

We agree that this result can also inform other amortized methods under the same assumptions. This generality is part of the theoretical contribution, rather than an architectural claim. MoLE provides a separate inductive bias: it explicitly accommodates mechanism heterogeneity through specialization. The theory motivates this setting but does not establish that MoLE is necessary or superior. Its benefit is evaluated experimentally (C5).

Q2: How restrictive are the assumptions, and to what extent does the training prior satisfy Theorem 4.5?

The QPE framework allows us to analyze LiNGAM, ANM, HNM, and PNL within a common representation, rather than requiring one homogeneous family throughout the graph. The additional finite-span and non-degeneracy conditions specify where this heterogeneous identifiability result holds.

Before adding General mechanisms and post-processing, the four-family generator matches the theorem’s mechanism-level construction. The remaining conditions concern the allowed PNL functions and the distributional non-degeneracy needed for recursive identification. They are not implied by mechanism-family membership alone.

The broader training prior deliberately extends beyond this analyzed class to improve practical coverage. See C3 for the distinction between the theoretical guarantees and empirical robustness under General mechanisms, discretization, and warping. Thus, the assumptions delimit the guarantee without restricting the empirical study to idealized data.

Q3: How would other amortized methods perform when trained on the same prior?

Please see C5 for the complementary control.


We again thank the reviewer for the constructive feedback. If these explanations and the additional experimental evidence address your concerns, we would appreciate reconsideration of your assessment.

Reviewer_dXWEOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies causal discovery when different causal mechanisms coexist. The authors propose DAG-FM, pretrained on synthetic data, which constructs an acyclic causal graph by recursively predicting leaf nodes and their parents and uses MoLE to select mechanism-specific experts. The paper evaluates the method on synthetic and real-world data and at different data scales.

Critical strengths
  1. The setting of mixed causal mechanisms is practically meaningful, and the idea of incorporating different mechanisms into a unified pretraining framework is appealing.
  2. Recursively predicting leaf nodes ensures acyclic outputs, and the results on synthetic and real-world data show the method’s potential.
Critical weaknesses
  1. In footnote 4, the authors state that Assumption 4.3(ii) and Assumption A.2(iv) still hold even though U_i and X_{N\i} are no longer independent in the rewritten representation. However, this addresses two local conditions without explaining why the QPE conditions and Corollary A.5 remain applicable after the original independence is lost. I therefore remain unsure whether Theorem 4.5 applies to the rewritten representation used in the proof.
  2. I am somewhat puzzled by the Direct-LiNGAM results in Tables 2 and 7, where its order divergence rate is only 0.02 under the LiNGAM setting, while its edge-prediction F1 is only 0.01. I would like the authors to explain this gap and check how ODR and F1 were defined and computed.
  3. Given the difference between the Direct-LiNGAM results in Tables 2 and 7, I would like to know how the synthetic cases were generated for training and testing, and whether the two tables used the same test cases and data processing. These details would help me interpret the reported difference and what it means for the experimental conclusions.
Questions for authors
  1. I would appreciate the specific derivation invoking Corollary A.5 in the proof of Theorem 4.5, explaining why the relevant QPE conditions still hold when the rewritten representation changes the noise independence.
  2. I would like to understand why Direct-LiNGAM’s order prediction and edge prediction differ so markedly, and whether the results remain the same after checking the evaluation procedures for Tables 2 and 7.
  3. In light of the Direct-LiNGAM results above, I would like to know how synthetic cases were generated for training and testing, and whether Tables 2 and 7 used the same test cases and data processing.
3 comments

We thank the reviewer for recognizing the value of heterogeneous causal discovery and our leaf–parent construction. We especially appreciate the observation about Direct-LiNGAM, which helped us identify and correct an mismatch in our appendix.

Q1: Why do the QPE conditions remain applicable when the rewritten representation changes noise independence?

See C2 of the global rebuttal for the derivation. We now invoke Corollary A.5 on a current leaf within the remaining ancestral set, rather than on an expanded neighborhood. Its conditional distribution equals that given its parents, so Table 4 establishes the finite-span condition. The recursive non-degeneracy condition excludes non-leaves. This makes the corollary’s application explicit while retaining the original QPE-based identification strategy.

Q2: Why is Direct-LiNGAM’s ordering accurate while its edge-prediction F1 is extremely low?

Our checks identified a mismatch between the adjacency convention of the lingam Python package and our edge-evaluation code, causing predicted edges to be evaluated in reverse. This affected the edge metrics for Direct-LiNGAM, ICA-LiNGAM, RCD, and RESIT.

We have corrected the conversion and recomputed the affected metrics. Direct-LiNGAM now achieves F1 = 0.91 and nSHD = 0.05 on LiNGAM data. ODR, computed separately from the estimated ordering, remains 0.02. The corrected results therefore resolve the ordering–edge recovery discrepancy.

Each entry below reports F1 / nSHD:

SettingICA-LiNGAMDirect-LiNGAMRESITRCD
LiNGAM0.710.20/0.180.220.71_{0.20}/0.18_{0.22}0.910.11/0.050.060.91_{0.11}/0.05_{0.06}0.490.20/0.410.330.49_{0.20}/0.41_{0.33}0.740.18/0.140.170.74_{0.18}/0.14_{0.17}
ANM0.310.10/0.310.170.31_{0.10}/0.31_{0.17}0.420.12/0.250.140.42_{0.12}/0.25_{0.14}0.540.17/0.270.230.54_{0.17}/0.27_{0.23}0.160.10/0.230.140.16_{0.10}/0.23_{0.14}
HNM0.340.14/0.350.250.34_{0.14}/0.35_{0.25}0.450.15/0.260.150.45_{0.15}/0.26_{0.15}0.250.15/0.720.280.25_{0.15}/0.72_{0.28}0.080.09/0.260.190.08_{0.09}/0.26_{0.19}
PNL0.360.11/0.340.220.36_{0.11}/0.34_{0.22}0.460.12/0.260.170.46_{0.12}/0.26_{0.17}0.330.11/0.500.270.33_{0.11}/0.50_{0.27}0.150.10/0.260.190.15_{0.10}/0.26_{0.19}
General0.300.09/0.440.250.30_{0.09}/0.44_{0.25}0.310.10/0.430.260.31_{0.10}/0.43_{0.26}0.260.14/0.720.230.26_{0.14}/0.72_{0.23}0.050.07/0.280.200.05_{0.07}/0.28_{0.20}
Hetero0.330.11/0.360.260.33_{0.11}/0.36_{0.26}0.460.15/0.260.190.46_{0.15}/0.26_{0.19}0.310.13/0.520.270.31_{0.13}/0.52_{0.27}0.200.12/0.250.210.20_{0.12}/0.25_{0.21}
Sachs0.34/0.780.34/0.780.36/0.650.36/0.650.21/1.070.21/1.070.00/0.360.00/0.36
Causal Chamber0.54/0.190.54/0.190.55/0.190.55/0.190.17/0.570.17/0.570.44/0.150.44/0.15

The correction restores Direct-LiNGAM’s strong performance in its matched linear non-Gaussian setting. We will update the affected tables and specify the adjacency convention in the revised manuscript.


We again thank the reviewer for the careful reading and constructive feedback. If these explanations and corrected results address your concerns, we would appreciate reconsideration of your assessment.

The results make more sense with the edge directions fixed. One thing that I am still unsure about is whether the mixed mechanisms used in the experiments satisfy the assumptions required for the recursive leaf selection procedure. I don’t think my concern on this issue was fully addressed in the response, so I will therefore maintain my current rating.

We appreciate the reviewer raising this specific follow-up. We had understood the initial review as focusing on the proof and the LiNGAM results, and had not realized that the applicability of the assumptions to the mixed-mechanism benchmarks also required an explicit response, as this point was not directly raised in the initial review. Our response is as follows.

The core mixed-mechanism benchmarks use mechanisms drawn from the four families covered by our theory. Although DAG-FM is pretrained on a broader distribution that also includes General mechanisms and post-processing, its training prior includes the theoretically identifiable mixed-mechanism settings. Thus, the model is trained on settings covered by the theory, while its performance on the extended settings is evaluated empirically. As discussed in C1 and C3, our claims for those extended settings are empirical.

Reviewer_ep6POpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes DAG-FM, a pretrained model for predicting causal DAGs from observational tabular data. It decomposes graph recovery into leaf-node prediction and parent prediction for the selected leaf. Inference repeatedly selects and removes leaves, recovering a causal order while adding parent–child relations. This construction guarantees acyclic outputs.

To handle heterogeneous mechanisms, the paper introduces Mixture-of-Leaf-Experts (MoLE), with experts for LiNGAM, ANM, HNM, PNL, and more general mechanisms. The router is trained using mechanism labels from synthetic data. The paper also presents identifiability arguments for heterogeneous mechanisms. Experiments cover homogeneous and heterogeneous synthetic data, Sachs, Causal Chamber, and additional tests of sample size, dimensionality, perturbations, and distribution shift. DAG-FM improves several structural recovery metrics over the evaluated methods.

Critical strengths

The prediction procedure directly guarantees structural validity. Repeatedly removing a leaf and adding parent edges only from remaining variables avoids the need for subsequent cycle repair. It also connects ordering and parent prediction in an inference procedure that is easy to understand and inspect.

The evaluation is broad and shows practical improvements. It includes multiple mechanism families, heterogeneous mechanisms, distribution shifts, changes in sample size and dimensionality, and two real-world datasets. On Hetero, DAG-FM achieves a mean F1 of 0.66, exceeding the other amortized methods in Table 1. Improvements on real-world data further motivate studying this approach.

Critical weaknesses

There are gaps between the theoretical claims, their proofs, and the implemented training setting. The proof of Theorem 4.4 distinguishes mechanism families through their QPE representations, but using one basis function rather than two does not itself establish that the corresponding function spaces are disjoint. Although a non-degeneracy assumption is included, the excluded overlaps and their treatment in the generator need to be specified. Theorem 4.5 also applies the preceding result after expanding mechanisms to include neighboring variables. Such sets can contain descendants, so independence between the original noise and the added conditioning variables does not follow automatically. Moreover, sampling finitely many PNL functions during training does not imply that the generating prior has finite support over functions. Training additionally includes General mechanisms, nonlinear transformations, and discretization, some of which fall outside the proof conditions. The current argument therefore does not establish broad identifiability or convergence guarantees for the implemented pretrained model.

The experiments do not sufficiently isolate the benefits of MoLE and the architecture. The main synthetic tests share most configurations with the authors’ training generator, whereas competing methods use their own public pretrained checkpoints. The comparison therefore reflects both architectural and training-prior differences. The OOD tests help, but do not fully separate these factors. The MoLE ablation bypasses routing in an already trained multi-expert model and forces a single expert to handle all samples. This exposes experts to cases outside their main training responsibility and is not equivalent to retraining a single-head model with the same data and computational budget. Without that control, the benefit attributed to mechanism specialization remains insufficiently established.

Questions for authors

Could you precisely state the conditions ensuring non-overlap of mechanism families in Theorem 4.4 and explain how the generator satisfies them? In Theorem 4.5, why do noise independence and the required identifiability conditions remain valid after neighboring variables are added?

What portion of the actual training distribution satisfies all theoretical conditions? After introducing General mechanisms, discretization, and feature transformations, which conclusions remain guaranteed and which are empirical? Does Assumption A.4 restrict the support of the prior or only the functions appearing in a finite training sample?

1 comment

We thank the reviewer for recognizing DAG-FM’s structural validity and broad evaluation, and for the constructive questions about the theory and experimental controls.

Q1: Conditions ensuring non-overlap in Theorem 4.4, and how the generator satisfies them.

Overlaps arise from special parameterizations producing exactly equivalent mechanisms. For example, exponential PNL, Y=exp⁡(g(X)+U)=exp⁡(g(X))exp⁡(U)Y=\exp(g(X)+U)=\exp(g(X))\exp(U), also admits a multiplicative-noise representation.

Our randomized construction makes such exact coincidences unlikely. Under non-degenerate continuous parameter sampling, exceptional configurations defined by specific functional identities have probability zero, provided the sampling family is not itself constrained to those identities. This is the general-position interpretation of our non-degeneracy assumption, not a claim that unrestricted mechanism families are disjoint.

We have made sufficient restrictions explicit in the revised version: non-affine gg for ANM, non-constant positive hh for HNM, and non-constant gg with hˉ∉span⁡{1,y}\bar h\notin\operatorname{span}\{1,y\} for PNL. These yield distinct QPE signatures. Importantly, DAG identification only requires a common finite span, not unique mechanism-family labels.

Q2: Why do the required conditions remain valid after neighboring variables are added?

See C2 of the global rebuttal. The QPE identification criterion is distributional once its finite-span and non-degeneracy conditions hold. To establish these conditions explicitly, the revised proof uses recursive leaf identification: each remaining set is ancestral, and a current leaf’s noise is independent of all other remaining variables. Its mechanism-specific QPE representation therefore applies directly. This preserves the QPE identification principle while avoiding conditioning on descendants and aligning the proof with DAG-FM’s inference procedure.

Q3: What portion of the training distribution satisfies all theoretical conditions?

See C3. Membership in the four mechanism families alone does not determine this proportion, where the finite-span and non-degeneracy conditions also concern the generated functions and distributions. We therefore can not infer a numerical percentage from mechanism-sampling frequencies. The theory establishes an identifiable heterogeneous class, while the broader training prior tests practical recovery beyond that class.

Q4: Which conclusions remain guaranteed after General mechanisms and post-processing?

See C3. Identifiability applies within the analyzed class satisfying the stated conditions. General mechanisms and post-processing extend practical coverage, with robustness assessed in Table 11 and Figures 5–6.

Q5: Does A.4 restrict the prior support or only a finite training sample?

A.4 restricts the generating prior’s support. We have clarified the wording. LiNGAM, ANM, and HNM already have fixed QPE spanning functions. The additional restriction incorporates PNL into the same specified finite-dimensional span.

This restriction concerns our proof framework, rather than being a general necessity for PNL identifiability. Complementary work by Kamimura and Pham (2026) develops transformed low-rank quantile surfaces and establishes further identifiability results involving unknown transformations, including PNL-HNM under its conditions. We will discuss this development alongside our assumptions.

W1–W2: Training-prior fairness and the MoLE ablation.

Please see C4 and C5 of the global rebuttal.


We again thank the reviewer for the constructive feedback. If these explanations and the additional evidence address your concerns, we would appreciate reconsideration of your assessment.

Reference: Kamimura, R., & Pham, T. (2026). Causal discovery via transformed low-rank quantile surfaces. arXiv preprint arXiv:2609.16931.

Comments on the paper

3 comments
AuthorsOpenReview ↗

Global Rebuttal (Part I)

We thank the reviewers for recognizing the value of leaf–parent prediction and causal discovery under heterogeneous mechanisms. We address the shared questions about the theoretical contribution, its connection to DAG-FM, and its practical scope.

C1. Connection between the theory and DAG-FM

Our theoretical and methodological contributions are complementary. The theory establishes sufficient conditions for DAG identifiability under heterogeneous mechanisms, rather than requiring a single mechanism family throughout the SCM. DAG-FM brings this heterogeneous perspective into an amortized architecture through mechanism-aware specialization.

Identifiability is a property of the generating distributions and can also benefit other amortized methods under the same assumptions. It does not uniquely prescribe MoLE. Rather, MoLE translates mechanism heterogeneity into an architectural inductive bias, whose practical benefit is assessed empirically (See C5).

The contribution is therefore two-fold: an identifiability analysis for a restricted heterogeneous prior, and a mechanism-aware model evaluated on broader settings. The theory provides a statistical foundation for the analyzed class. Beyond that class, including post-processing, the connection is supported empirically rather than guaranteed by the theorem.

C2. QPE identification and the conditioning argument

The original argument uses a distributional property of QPE: once the finite-span and non-degeneracy conditions hold for the relevant conditional distribution, the identification criterion does not require an independent-noise representation. To make the application of these conditions explicit, we have reformulated the graph-level argument through recursive leaf identification.

Removing true leaves leaves an ancestral variable set. For any current leaf, all other remaining variables are non-descendants, so its noise remains independent of the conditioning variables. Its conditional distribution therefore reduces to that given its parents, and the mechanism-specific QPE expressions apply, with zero components for non-parents. Under A.4, these expressions belong to a common finite-dimensional span.

The Wronskian criterion retains the true leaves, while the explicitly stated recursive non-degeneracy condition excludes non-leaves in each ancestral sub-problem. Multiple leaves may coexist, each of can be removed, and their parents are identified through conditional independences under faithfulness. Repeating the argument identifies the DAG.

This formulation retains the QPE-based identification principle while avoiding the need to establish its conditions after adding descendants. It also aligns the proof directly with the leaf–parent decomposition used by DAG-FM.

C3. Scope of the guarantees and the training distribution

The four mechanism families provide the analyzed setting, with A.4 specifying the PNL support needed for a common finite-dimensional QPE span and the non-degeneracy condition distinguishing leaves from non-leaves.

General mechanisms and post-processing deliberately broaden training coverage beyond this setting. Their role is to improve practical robustness, not to extend the theorem. This distinction is now explicit in the theoretical discussion and the Post-processing paragraph in revision.

In particular, invertible feature-wise warping preserves observational information but need not preserve the assumed mechanism family or QPE span. Discretization can lose information and violate smoothness. Accordingly, Table 11 and Figures 5–6 evaluate recovery and robustness under these extensions empirically.

AuthorsOpenReview ↗

Global Rebuttal (Part II)

C4. Additional experiments: sensitivity to training-prior shift

Training-prior shift affects DAG-FM and is a shared challenge for amortized causal discovery. Our existing experiments suggest this sensitivity: models primarily trained on linear or additive mechanisms often perform less well on HNM and PNL. To examine its effect more directly, we additionally evaluated the pretrained models on external generators not used to train DAG-FM.

These experiments assess cross-generator generalization. They complement, rather than replace, the controlled comparisons in C5, which isolate architectural contributions under matched training conditions.

External benchmarks from the AVICI generator. We evaluated gene-ecoli, lin-lc, rff-g, and rff-lc. Some corresponding generator families are represented in AVICI’s and TabCausal’s training priors, whereas DAG-FM was not trained using this generator. The comparison therefore tests sensitivity to different training priors, rather than an equally unfamiliar distribution for every model.

Each entry reports F1 / nSHD:

SettingAVICICauScaleTabCausalFoundCauseDAG-FM
gene-ecoli0.040.07/0.180.060.04_{0.07}/0.18_{0.06}0.010.03/0.160.050.01_{0.03}/0.16_{0.05}0.000.01/0.150.040.00_{0.01}/0.15_{0.04}0.030.08/0.160.070.03_{0.08}/0.16_{0.07}0.060.07/0.270.080.06_{0.07}/0.27_{0.08}
lin-lc0.620.26/0.100.060.62_{0.26}/0.10_{0.06}0.250.28/0.180.080.25_{0.28}/0.18_{0.08}0.890.09/0.040.030.89_{0.09}/0.04_{0.03}0.500.24/0.190.120.50_{0.24}/0.19_{0.12}0.620.16/0.150.090.62_{0.16}/0.15_{0.09}
rff-g0.660.15/0.100.040.66_{0.15}/0.10_{0.04}0.410.17/0.200.080.41_{0.17}/0.20_{0.08}0.820.10/0.060.040.82_{0.10}/0.06_{0.04}0.620.13/0.190.120.62_{0.13}/0.19_{0.12}0.270.12/0.500.200.27_{0.12}/0.50_{0.20}
rff-lc0.530.31/0.110.060.53_{0.31}/0.11_{0.06}0.190.22/0.190.070.19_{0.22}/0.19_{0.07}0.750.20/0.070.050.75_{0.20}/0.07_{0.05}0.420.23/0.220.130.42_{0.23}/0.22_{0.13}0.550.15/0.140.060.55_{0.15}/0.14_{0.06}

DAG-FM does not dominate these benchmarks. Its F1 is competitive with AVICI on lin-lc and rff-lc, but substantially lower on rff-g. Random Fourier feature mechanisms are not a major component of our training prior. Its performance is better on the Laplace–Cauchy settings, whose noise characteristics are closer to those covered during training. This pattern is consistent with prior sensitivity, although these comparisons alone do not isolate the contribution of mechanism versus noise mismatch. On gene-ecoli, all methods have low F1; DAG-FM’s numerically highest F1 also comes with higher nSHD.

A further external generator: TabICL v2 GraphSCM. We also evaluated a structurally different generator from TabICL v2 (Qu et al., 2026), which was not used in DAG-FM’s training and, to our knowledge, is not part of the evaluated baselines’ training generators.

SettingAVICICauScaleTabCausalFoundCauseDAG-FM
GraphSCM0.020.07/0.320.300.02_{0.07}/0.32_{0.30}0.060.10/0.350.290.06_{0.10}/0.35_{0.29}0.030.08/0.310.290.03_{0.08}/0.31_{0.29}0.110.17/0.420.300.11_{0.17}/0.42_{0.30}0.050.09/0.390.300.05_{0.09}/0.39_{0.30}

All evaluated models obtain low F1 on this generator, with no method leading across both metrics. Together with the AVICI-generator results, this illustrates that strong performance within one synthetic prior does not imply reliable recovery under an arbitrary external prior. The results support treating prior robustness as a broader challenge for amortized causal discovery, rather than a property resolved by DAG-FM.

Implications for our claims. Our main contribution targets heterogeneous mechanisms, supported by the mechanism-wise evaluations and Appendix D.2. The real-world experiments provide complementary evidence beyond our synthetic generator. We do not claim universal OOD recovery. Moreover, these external generators are not established to satisfy all our identifiability assumptions, so low recovery scores cannot be attributed to prior shift alone.

We have included these results to make the empirical boundary explicit: DAG-FM improves recovery in the studied heterogeneous settings, while cross-prior robustness remains an important direction for further work. The matched-training controls in C5 separately address whether the gains arise from the architecture rather than the training prior.

AuthorsOpenReview ↗

Global Rebuttal (Part III)

C5. Additional controls: MoLE vs. a retrained single head and another architecture

Our original ablation examined expert specialization and routing within a trained MoLE model. To complement it, following reviewers ep6P and J1Bz, we retrained a single-head leaf predictor and MoLE on the same training data, and trained a small TabCausal model using the same prior and data.

All use embed_dim=80, blocks=8, nhead=4, and 100k training steps. This also controls the backbone settings and training exposure. The reduced scale makes retraining feasible within the rebuttal period.

Ordering results (ODR ↓):

ModelLiNGAMANMHNMPNLGeneralHetero
DAG-FM [Single]0.120.090.12_{0.09}0.230.120.23_{0.12}0.230.110.23_{0.11}0.230.120.23_{0.12}0.280.130.28_{0.13}0.230.120.23_{0.12}
DAG-FM [MoLE]0.110.09\mathbf{0.11}_{0.09}0.200.11\mathbf{0.20}_{0.11}0.220.11\mathbf{0.22}_{0.11}0.190.11\mathbf{0.19}_{0.11}0.260.12\mathbf{0.26}_{0.12}0.210.11\mathbf{0.21}_{0.11}

Graph-recovery results:

LiNGAMPrec. ↑Rec. ↑F1 ↑Jac. ↑nSHD ↓
TabCausal [Small]0.690.210.69_{0.21}0.260.160.26_{0.16}0.360.160.36_{0.16}0.230.140.23_{0.14}0.220.180.22_{0.18}
DAG-FM [Single]0.750.16\mathbf{0.75}_{0.16}0.680.190.68_{0.19}0.700.16\mathbf{0.70}_{0.16}0.560.19\mathbf{0.56}_{0.19}0.170.17\mathbf{0.17}_{0.17}
DAG-FM [MoLE]0.740.160.74_{0.16}0.690.20\mathbf{0.69}_{0.20}0.700.16\mathbf{0.70}_{0.16}0.560.19\mathbf{0.56}_{0.19}0.170.18\mathbf{0.17}_{0.18}
ANMPrec. ↑Rec. ↑F1 ↑Jac. ↑nSHD ↓
TabCausal [Small]0.800.28\mathbf{0.80}_{0.28}0.120.110.12_{0.11}0.190.150.19_{0.15}0.110.100.11_{0.10}0.210.140.21_{0.14}
DAG-FM [Single]0.680.150.68_{0.15}0.580.180.58_{0.18}0.610.150.61_{0.15}0.460.160.46_{0.16}0.170.14\mathbf{0.17}_{0.14}
DAG-FM [MoLE]0.700.140.70_{0.14}0.600.19\mathbf{0.60}_{0.19}0.630.15\mathbf{0.63}_{0.15}0.480.17\mathbf{0.48}_{0.17}0.170.14\mathbf{0.17}_{0.14}
HNMPrec. ↑Rec. ↑F1 ↑Jac. ↑nSHD ↓
TabCausal [Small]0.690.27\mathbf{0.69}_{0.27}0.180.200.18_{0.20}0.250.210.25_{0.21}0.160.180.16_{0.18}0.220.160.22_{0.16}
DAG-FM [Single]0.600.150.60_{0.15}0.610.18\mathbf{0.61}_{0.18}0.590.130.59_{0.13}0.430.150.43_{0.15}0.210.160.21_{0.16}
DAG-FM [MoLE]0.620.160.62_{0.16}0.610.18\mathbf{0.61}_{0.18}0.600.14\mathbf{0.60}_{0.14}0.450.17\mathbf{0.45}_{0.17}0.200.16\mathbf{0.20}_{0.16}
PNLPrec. ↑Rec. ↑F1 ↑Jac. ↑nSHD ↓
TabCausal [Small]0.830.21\mathbf{0.83}_{0.21}0.140.140.14_{0.14}0.220.170.22_{0.17}0.140.130.14_{0.13}0.230.190.23_{0.19}
DAG-FM [Single]0.710.130.71_{0.13}0.550.190.55_{0.19}0.600.150.60_{0.15}0.450.170.45_{0.17}0.190.170.19_{0.17}
DAG-FM [MoLE]0.750.110.75_{0.11}0.580.21\mathbf{0.58}_{0.21}0.630.16\mathbf{0.63}_{0.16}0.480.19\mathbf{0.48}_{0.19}0.180.17\mathbf{0.18}_{0.17}
GeneralPrec. ↑Rec. ↑F1 ↑Jac. ↑nSHD ↓
TabCausal [Small]0.570.29\mathbf{0.57}_{0.29}0.160.160.16_{0.16}0.220.170.22_{0.17}0.140.120.14_{0.12}0.240.17\mathbf{0.24}_{0.17}
DAG-FM [Single]0.550.130.55_{0.13}0.540.170.54_{0.17}0.530.140.53_{0.14}0.380.150.38_{0.15}0.250.190.25_{0.19}
DAG-FM [MoLE]0.560.120.56_{0.12}0.550.16\mathbf{0.55}_{0.16}0.540.13\mathbf{0.54}_{0.13}0.390.14\mathbf{0.39}_{0.14}0.240.19\mathbf{0.24}_{0.19}
HeteroPrec. ↑Rec. ↑F1 ↑Jac. ↑nSHD ↓
TabCausal [Small]0.700.27\mathbf{0.70}_{0.27}0.180.140.18_{0.14}0.260.160.26_{0.16}0.160.110.16_{0.11}0.230.180.23_{0.18}
DAG-FM [Single]0.640.170.64_{0.17}0.570.180.57_{0.18}0.590.160.59_{0.16}0.440.170.44_{0.17}0.200.180.20_{0.18}
DAG-FM [MoLE]0.670.150.67_{0.15}0.590.18\mathbf{0.59}_{0.18}0.610.14\mathbf{0.61}_{0.14}0.450.15\mathbf{0.45}_{0.15}0.190.18\mathbf{0.19}_{0.18}

MoLE v.s. single head. Mean ODR decreases in all six settings when using MoLE. Graph F1 improves in five settings and ties on LiNGAM. These consistent mean improvements support specialization beyond the original forced-expert ablation, without implying statistical significance.

TabCausal v.s. DAG-FM TabCausal has higher precision in most settings but substantially lower recall. MoLE achieves higher F1 throughout and lower or equal nSHD. Since TabCausal’s full original training recipe is unavailable to us, this is a shared-configuration comparison, not a claim about its best achievable performance.

We will add these controls to the revised manuscript.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.