acceptodds
Under review as a conference paper at ICLR 2027

ALICE: Amortized Learning for Invariance-aware Causal Estimation

31 views

Abstract

Causal Discovery (CD) is fundamental across domains such as biology, finance, weather, and physical systems. Classical CD approaches often rely on restrictive assumptions about causal structure, noise distributions, and mechanisms, and require sufficiently large datasets for reliable estimation. These factors can limit their effectiveness on real-world datasets. Moreover, classical and existing zero-shot approaches do not explicitly exploit feature semantics, which can provide complementary causal information. To this end, we first introduce Amortized Learning for Invariance-aware Causal Estimation (ALICE), a causal discovery foundation model, trained entirely on synthetic datasets generated from diverse structural causal models, that maps observational data to a full directed causal graph in a single forward pass. Our diverse pretraining corpus spanning graph families, mechanisms, and noise processes enables effective zero-shot transfer to real-world datasets. Second, we propose targeted augmentations and consistency losses to enforce key graph invariances, including realization, permutation, and scale. Leveraging these invariances, we propose test-time optimization strategies that improve the F1 score by up to 14%. Finally, we incorporate feature semantics as a post-processing signal, yielding up to 18% improvement on real-world causal estimation tasks. Across synthetic, biological, physical-system, bivariate, and tabular benchmarks, ALICE achieves accurate and robust causal graph estimation without dataset-specific training.

Primary area:
causal reasoning
Keywords:
Causal Discovery Foundation Models, Causal Graph Estimation, Sim-to-Real Generalization, Graph Invariances

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_Mht6OpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

ALICE is an amortized causal discovery model trained on synthetic structural causal models to predict directed graphs from observational datasets. Its main contribution is a collection of augmentations and auxiliary objectives intended to improve consistency across datasets sharing the same causal structure. The paper also introduces target-conditioned averaging, subgraph voting, and post-processing with semantic information from an LLM. Experiments demonstrate competitive graph recovery and useful aggregation gains, although the causal validity of some components and the attribution of these improvements remain insufficiently established.

Critical strengths
  • Improving the robustness of pretrained causal discovery models is a meaningful objective. ALICE obtains encouraging results across diverse benchmarks, including the best reported AUROC on the four real-world datasets in Table 1.

  • The evaluation goes beyond aggregate accuracy to examine consistency, identifiability, graph width, and inference cost. Table 9 also provides a useful matched ablation showing that the auxiliary training objectives improve several metrics.

  • Target-conditioned inference is an interesting design choice, and Figure 6 provides evidence that different targets produce complementary local improvements.

Critical weaknesses
  • Subgraph voting needs a more careful causal justification. Section 3.4 uses the identity (AS)ij=Aij(A_S){ij}=A{ij}, which holds for an induced adjacency matrix but does not establish that the marginal dataset DSD_S supports the same causal interpretation. For example, removing the mediator from X→Z→YX\rightarrow Z\rightarrow Y leaves dependent variables whose relationship may be predicted as a direct edge; removing a common cause introduces latent confounding. Averaging these predictions can preserve systematic errors. Theorem 3.3 is a valid convex-loss averaging argument under its assumptions, but it does not resolve this issue or guarantee improved F1 over full-graph inference, as the authors appropriately acknowledge. There are also recent related works missing, for example:

Causal Discovery in the Wild: A Voting-Theoretic Ensemble Approach, ICLR 2026;

Efficient Causal Structure Learning via Modular Subgraph Integration, arXiv: 2601.21014

  • The duplicate-parity objective raises a related concern about the correctness of the training labels. Appendix D states that a near-copy of a target’s parent is appended and labeled as another parent. If the original target-generating mechanism is unchanged, a noisy copy of a cause is not automatically another direct cause. Enforcing equal causal scores could therefore teach the model to confuse predictive proxies with causal parents. The paper needs to specify the augmented SCM and explain why these labels are valid.

  • The distinctive contribution is not sufficiently isolated. AVICI already incorporates observation-permutation invariance, variable-permutation equivariance, alternating attention, and asymmetric edge scoring. ALICE’s novelty therefore rests substantially on its auxiliary objectives and target-conditioned inference. However, Table 9 removes the auxiliary losses together, including objectives involving additional mediator, confounder, and intervention supervision. This establishes the benefit of the combined training recipe, but does not distinguish consistency regularization from additional causal supervision. Moreover, the generic averaging theorem does not explain why these particular training losses improve aggregation efficiency.

  • The semantic-fusion argument is not probabilistically justified. Giving numerical values to one model and feature names to another does not establish statistical independence between their predictions. Even independence would not, by itself, justify treating two estimates of the same edge event as 1−(1−pnum)(1−pLLM)1-(1-p_{\mathrm{num}})(1-p_{\mathrm{LLM}}). Empirically, Figure 8 shows that the LLM alone outperforms ALICE’s fused prediction on Tübingen, so the highlighted improvement over ALICE alone does not establish useful complementarity on that benchmark.

Questions for authors
  • What assumptions make predictions from variable subsets comparable to the original graph’s direct-edge labels? Can you report errors specifically when mediators or common causes are omitted?

  • For duplicate-parity training, is the target regenerated to depend causally on the appended variable? Please provide the corresponding structural equations.

  • Can you separate consistency penalties from additional supervised augmentation while keeping training examples and compute matched, and evaluate these ablations on real-world benchmarks?

  • How does semantic fusion compare with the stronger individual predictor and simple alternative fusion rules? Why does combining ALICE with the LLM reduce performance relative to the LLM alone on Tübingen?

Given these unresolved issues, my initial score is 2.


Raised to 3 after rebuttal.

7 comments

We thank the reviewer for their careful observations and detailed feedback. The reviewer acknowledged the improvements that stem from the proposed target-aware design and the extensive evaluation. Below, we address the weaknesses and the questions raised by the reviewer.

(W1 and Q1) Possible Bias in Subgraph Voting

We agree with the reviewer. The identity ASij=AijA_{S_{ij}} = A_{ij} holds for the induced subgraph, but it does not imply that DSD_S supports the same causal interpretation. Removing a variable k from DSD_S does not remove its effect on the remaining variables. If k is a mediator (i→k→ji \rightarrow k \rightarrow j) or a common cause (i←k→ji \leftarrow k \rightarrow j) of a nonadjacent pair, then ii and jj seem dependent in DSD_S with k no longer present to explain that dependence, and the estimator may predict a spurious edge. Theorem 3.3 and the per-edge averaging bound do not address this bias, since it is shared across subgraphs. Accordingly, we have revised Section 3.4 (lines 370-375) and Limitations (lines 491 and 492) to state this explicitly.

When are subgraph predictions comparable to the original direct-edge labels?

A subgraph prediction for (i,j)(i, j) targets AijA_{ij} when SS retains the variables that explain any dependence between ii and jj other than a direct edge, such as mediators and common causes. When we omit such a variable, the prediction may include a spurious edge, and averaging retains this error in proportion to how often we omit the variable. However, omitting variables does not remove a dependence created by a true edge, so the bias mainly produces false positives rather than missed edges.

Quantifying errors when mediators or common causes are omitted.

We evaluate ALICE on all 450 synthetic-suite graphs with 50 or 100 variables, covering 5,619 nonadjacent endpoint pairs and 8,161 tests of an individual mediator or common cause. In each test, both endpoints appear in all 64 width-30 subgraphs, the target is fixed, and we control the fraction of subgraphs from which the explanatory node is omitted. All conditions use the same frozen threshold calibrator. Probe accuracy is the fraction of tests in which neither i→ji \rightarrow j nor j→ij \rightarrow i is selected.

ALICE conditionProbe accuracy ↑\uparrowAUROC ↑\uparrowF1 ↑\uparrow
Null-target66.2%0.87620.3716
0% omitted88.4%0.92660.5075
25% omitted84.2%0.92690.5085
50% omitted78.3%0.92700.5083
75% omitted72.7%0.92690.5070
100% omitted68.3%0.92000.4897
Random Sampling72.7%0.93390.5218

Table 1: Omitting mediators and common causes increases spurious edges locally but has a limited effect on overall graph recovery. Probe accuracy measures how often ALICE correctly rejects an edge between two nonadjacent variables as the fraction of subgraphs omitting their mediator or common cause increases.

Probe accuracy decreases from 88.4% to 68.3% as the omission rate increases. However, the effect remains local. Whole-graph AUROC and F1 are nearly unchanged up to 75% omission. Also, random sampling of 30-feature subgraphs achieves the best whole-graph AUROC and F1, since it covers all variable pairs with many votes.

The gap between random sampling and the 0% omission condition (72.7% vs. 88.4%) suggests that sampling strategies that first identify likely mediators and common causes of each pair, for example, from ALICE’s own full-graph scores or from an estimated Markov blanket as in VISTA [1] (Sun et al., 2026), and use that information in sampling could reduce these spurious edges. We leave this to future work and have added this discussion and the analysis above in Appendix G.

[1] Efficient Causal Structure Learning via Modular Subgraph Integration, arXiv: 2601.21014

(W1) Related Works

We thank the reviewer for these references and will add both. Vo et al. [1] combine different classical algorithms to hedge against their individual assumptions, whereas VISTA [2] aggregates one classical algorithm over Markov-blanket subgraphs for scalability. Neither involves amortized pretraining. ALICE, similar to CDFM, TabCausal, and FoundCause, addresses the assumption problem through pretraining on diverse SCMs, and its subgraph voting is similar to VISTA’s. The difference is that ALICE’s invariance losses during training enable consistency across subgraph views, whereas VISTA considers a fixed base learner and relies on the voting rule to filter inconsistent subgraph predictions. ALICE is complementary to both: it can serve as one of the experts in Vo et al.’s ensemble, and VISTA’s Markov-blanket decomposition can be used to further reduce omission bias in ALICE’s subgraph voting.

[1] Causal Discovery in the Wild: A Voting-Theoretic Ensemble Approach, ICLR 2026

[2] Efficient Causal Structure Learning via Modular Subgraph Integration, arXiv: 2601.21014

(W2 and Q2) Clarifying Duplicate-Parity Penalty

The duplicate-parity objective targets redundant measurements of the same quantity, for example, two calibrated sensors reading the temperature that a controller acts on (T→S1,T→S2,S1→y,S2→yT \rightarrow S_1, T \rightarrow S_2, S_1 \rightarrow y, S_2 \rightarrow y). In such settings, a model should treat the two measurements symmetrically rather than arbitrarily favoring one of them.

Starting from the original SCM, Xj=fj(Xpa(j),εj)X_j = f_j(X_{pa}(j), ε_j) and y=fy(Xpa(y),εy)y = f_y(X_{pa}(y), ε_y), we select a parent XpX_p of the target yy, generated by its own mechanism Xp=fp(Xpa(p),εp)X_p = f_p(X_{pa}(p), ε_p). We append X’=fp(Xpa(p),εp)+0.05η=Xp+0.05⋅ηX’ = f_p(X_{pa}(p), ε_p) + 0.05\eta = X_p + 0.05·\eta, with η∼N(0,1)\eta ∼ N(0, 1) independent of all other variables. X’X’ inherits XpX_p’s parents and, up to a small measurement noise, its exact realization. Accordingly, the two are siblings and share no edge between them, and we label both as parents of y. The parity penalty enforces that both XpX_p and X’X’ should be equally identified as parents of the target.

The target mechanism y=fy(Xpa(y),εy)y = f_y(X_{pa}(y), ε_y) is left unchanged after X’X’ is appended and y is not regenerated. So strictly, yy depends on XpX_p and not on X’X’. However, since X’=Xp+0.05ηX’ = X_p + 0.05\eta, the observational samples are nearly indistinguishable from those of the equivalent target mechanism y=fy(Xpa(y) Xp,½(Xp+X’),εy)y = f_y(X_{pa}(y) \ {X_p}, ½(X_p + X’), ε_y), in which y depends symmetrically on both measurements. For such near-duplicates, the direct parent is practically non-identifiable from observational data with finite/limited samples, and consequently, our parity penalty enforces the symmetric interpretation rather than arbitrarily choosing one.

(Q3, W3) Isolating the Contribution of the Invariance Losses

We thank the reviewer for this careful comparison. We agree that ALICE’s backbone follows established amortized designs: AVICI already uses alternating row-column attention, permutation symmetries over observations and variables, and asymmetric edge scoring. ALICE differs in two respects. First, these architectural symmetries cover only the reordering of rows and columns. ALICE additionally enforces invariances that architecture cannot guarantee, to new realizations, different mechanisms, added irrelevant variables, and duplicated parents, through dedicated training losses. Second, ALICE introduces target-conditioned inference and test-time aggregation, whose effects are isolated in Figure 6.

We now address the reviewer’s central question: Do ALICE’s gains come from the invariance penalties themselves or from the additional supervised samples they require?

To separate the contribution of the invariance losses from additional supervised augmentation, we trained a third matched model. It sees exactly the same augmented consistency samples as full ALICE and is supervised on them with the edge loss, but it omits the invariance penalties between views. All three models share architecture, size (~2M parameters), and training steps.

MetricNo AugmentationConsistency Samples Augmentation without Invariance LossesConsistency Samples Augmentation with Invariance Losses
Synthetic suite AUROC (null)0.87720.87150.8860
Synthetic suite F1 (null)0.44890.44070.4866
Ensembling gain, synthetic AUROC+0.0413+0.0434+0.0474
Ensembling gain, synthetic F1+0.0829+0.0825+0.1138
Causal Chambers AUROC0.96510.96150.9665
Sachs AUROC0.73880.65210.7577
Tübingen weighted acc.0.45880.60960.6431
Avg. identifiability deviation ↓\downarrow0.19210.16500.1133
Avg. consistency (invariance tests)0.75440.76910.8061

Table 2: Consistency penalties vs. supervised augmentation. All models are matched in size and training steps. Ensembling gains are for subgraph voting on the synthetic suite. Table 9 (Table 11 in the updated manuscript) has been updated with the detailed results.

Invariance losses enable gains in consistency and prediction. Adding the consistency samples without the penalties gives only small changes in consistency and slightly lowers AUROC on the synthetic suite. Adding the penalties on the same samples improves consistency, reduces identifiability deviation, and gives the best AUROC on the synthetic suite as well as real-world benchmarks.

Connection to aggregation efficiency. The reviewer correctly notes that Theorem 3.3 holds for any estimator and therefore does not by itself explain why these losses help aggregation. However, our argument is about the finite-sample approximation of the aggregate. A Monte Carlo average over views converges faster when the predictions across graph-preserving views vary less, and the invariance penalties directly reduce this view-induced variation (Table 2, page 6). With identical training samples, the model trained with penalties is the most consistent across views and also gains the most from subgraph voting.

Reviewer_J6FtOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper presents an approach called ALICE, which is essentially a supervised learning approach to causal structure prediction. ALICE is trained on a dataset of synthetic causal graphs and generated data. The authors discuss how a few invariances are specifically built into the training process. Empirically they demonstrate that this is effective for predicting the underlying causal DAG, even in transfer to real structure prediction datasets.

Critical strengths
  • I think the general empirical observation is useful: a supervised approach using only synthetic data can be SOTA on causal Discovery, if combined with enough invariance training and other tricks. Experiments generally seem like they give a pretty clean takeaway
Critical weaknesses
  • I get lost in spots: in particular around view-generation rules and target averaging. The authors should spend more time defining their terms and describing what these things are
  • The authors claim that previous methods rely on assumptions about the DGP; however by building the training set in a specific way they are also creating assumptions. It would be good to argue more clearly about why their approach is preferable or what the pros and cons are of doing it their way (a fully supervised + synthetic approach)
Questions for authors

Is Fig 3 on synthetic or real data? How does ALICE behave in unidenfiable / markov equivalent settings?

5 comments

We thank the reviewer for their careful observations and thoughtful feedback. The reviewer acknowledged the effective sim-to-real transfer for causal discovery, achieved through invariance losses and diverse synthetic pretraining. Below, we address the weaknesses and the questions raised by the reviewer.

W1: We have rewritten Section 3.4 to define the alternative views and explain how their predictions are combined.

W2: ALICE expands the mechanisms and noise distributions covered by a single estimator through a diverse synthetic prior. Our generator first samples a random causal structure, then its mechanisms and noise distributions, exposing the model to combinations that fall outside individual modeling assumptions (Section 3.1, lines 186 to 206). For example, LiNGAM relies on linear relationships and non-Gaussian noise. It fails to identify nonlinear mechanisms with additive Gaussian noise. Training across both regimes allows ALICE to learn different indicators of directional information without adhering to a particular signal for every dataset. Overall, diverse synthetic pretraining provides an accurate zero-shot causal estimator that works for real-world datasets without any assumptions on noise distributions or functional mechanisms. The only tradeoff is the dependence on the synthetic prior’s coverage, i.e., the synthetic prior distribution should be sufficiently broad to cover/overlap with real-world use cases. We have extensively assessed this through diverse real-world benchmarks that span physical systems, hydrological, and biological domains. Additionally, we show state-of-the-art performance on out-of-distribution benchmarks like CausalArena, where the dataset generation mechanism is unknown.

Q1: Figure 3 uses synthetic data to evaluate aggregation across large graphs, as they are unavailable in our real-world benchmarks. The real-world datasets available contain 2–20 variables, whereas Figure 3 examines scaling up to 100 variables and the corresponding accuracy–compute tradeoff on our synthetic suite. We have added test-time aggregation results on completely held-out CausalArena datasets to extend this analysis beyond our own synthetic evaluation suite.

DatasetNull AUROCSubgraph Voting AUROCNull F1Subgraph Voting F1
Synthetic0.84620.86580.50530.5497
Formula0.83840.84810.42130.4347

Table 1: Impact of subgraph voting on performance on held-out synthetic CausalArena datasets.

Q2: ALICE’s direction-recovery accuracy follows the expected identifiability boundary in the controlled experiments reported in Appendix F.

Our identifiability analysis (Appendix F) includes the bivariate linear-Gaussian case, where both orientations are Markov equivalent. Figure 5 evaluates Y=X+bX3+NY = X + bX^3 + N while varying nonlinearity (b) and non-Gaussianity (q). At b = 0 with Gaussian noise (q = 1), the model is linear-Gaussian, so X→YX \rightarrow Y and Y→XY \rightarrow X are Markov equivalent and the direction is unidentifiable. Away from this point, nonlinearity or non-Gaussian noise makes the direction identifiable. As expected, all methods drop to chance accuracy (0.5 probability) at the linear-Gaussian boundary. In the identifiable regimes, the methods differ substantially. ALICE performs best under nonlinearity and remains competitive under non-Gaussianity, making it the only method that reliably recovers the direction from both sources of identifiability. This is reflected in its lowest aggregate deviation from ideal behavior (Table 2).

Beyond two variables, we added a setting in which identifiable and unidentifiable edges coexist. The graphs G1G_1: A→B→C←DA \rightarrow B \rightarrow C \leftarrow D, C→EC \rightarrow E and G2G_2: A←B→C←DA \leftarrow B \rightarrow C \leftarrow D, C→EC \rightarrow E are Markov equivalent. The collider B→C←DB \rightarrow C \leftarrow D and the edge C→EC \rightarrow E are identifiable in both. The relationship between A and B is controlled using λ\lambda, which induces a linear, additive Gaussian (unidentifiable) relationship when λ=0\lambda = 0 and non-linear (identifiable) otherwise. Similar to our previous experiment, we generate A–B with a mechanism that interpolates from purely linear to increasingly nonlinear as a parameter λ\lambda grows. We report the deviation from ideal orientation accuracy, where the ideal is 0.5 for A–B at λ\lambda = 0 and 1 otherwise. Overall, ALICE achieves the lowest mean deviation, showing that it reliably exploits identifiable structure while remaining competitive on the ambiguous edge.

SettingALICEFoundCauseCDFMTabCausal
Bivariate0.11290.16310.37190.3600
Multivariate0.05940.07830.21990.0951

Table 2: Mean Absolute Deviation from ideal orientation accuracy in unidentifiable and Markov-equivalent settings (lower is better). Bivariate deviation is aggregated over the nonlinearity and non-Gaussianity sweeps (Figure 5). Multivariate deviation is averaged over the A–B, collider, and C→EC \rightarrow E edges (detailed results in Table 7 of the updated manuscript).

We refer the reviewer to Appendix F (pages 24 and 25) for a detailed explanation. These results clearly show that when directional information becomes available, ALICE exploits it better than the baselines.

Dear Reviewer,

We hope our rebuttal has adequately addressed the questions and concerns raised in your review. As we approach the end of the author–reviewer discussion period, we remain committed to clarifying any outstanding questions or addressing any further concerns regarding the manuscript or our responses.

Please let us know if any issues remain unresolved. If you feel that your concerns have been satisfactorily addressed, we would be very grateful if you could update your review and score accordingly.

Thank you again for your time and thoughtful feedback.

Reviewer_zD9kOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes ALICE, an amortized causal discovery model pretrained on diverse synthetic structural causal models to predict directed edge scores without dataset-specific retraining. Its architecture combines attention across observations and variables with an asymmetric edge-scoring head. The main contributions are targeted augmentations and auxiliary objectives intended to improve prediction consistency, test-time target averaging and subgraph voting, and optional fusion with LLM-derived semantic information. A theoretical argument motivates multi-view averaging under distribution-preserving transformations and convex losses. The evaluation covers synthetic benchmarks and four real-world datasets: Causal Chambers, Sachs, CausalRivers, and Tübingen.

Critical strengths

**The paper addresses a practically relevant problem. ** Learning to infer causal graphs from synthetic SCMs without retraining on each new dataset could reduce the computational burden of causal discovery. The investigation of transfer from synthetic training data to diverse real-world systems is valuable.

The evaluation covers diverse settings and includes competitive real-world results. Experiments span synthetic data, physical systems, biological signaling, hydrological time series, and bivariate causal direction identification, with comparisons against several recent amortized causal discovery models.

Critical weaknesses
  • The training data are generated by simulators, whereas real-world data may be considerably more complex. How can the authors prove that the proposed neural network generalizes to real-world settings?
  • The authors state that “The model should remain insensitive to input perturbations and transformations that leave the causal process unchanged.” If the predicted graphs exactly matched the ground truth, they would naturally satisfy the relevant invariances. However, when predictions are imperfect, different predicted graphs may fluctuate around the ground truth while still approximating it. Why should we enforce invariance in this case?
Questions for authors
  • Is the trained model suitable for deployment in real-world applications? Given that real-world data may often fall outside the training distribution, I question whether training on a limited set of distributions is sufficient to perform causal discovery across an unrestricted range of complex data-generating processes.
  • Could imposing the invariance constraints instead compromise prediction accuracy?
4 comments

We thank the reviewer for their careful observations and thoughtful feedback. The reviewer acknowledged ALICE's practical relevance and the extensive empirical evidence for effective real-world causal estimation. Below, we address the weaknesses and the questions raised by the reviewer.

W1 and Q1: We show ALICE’s generalization through transfer to real-world domains and improved performance on completely held-out benchmarks. Learning from synthetic data for real-world tabular prediction is an established approach, demonstrated by TabPFN and TabICL. We translate this approach to causal discovery with ALICE, where we show real-world evaluations on physical systems (Causal Chambers), biological signaling (SACHS), hydrological time series (Causal Rivers), and bivariate causal relationships (Tuebingen), without any training on these datasets. To showcase transfer beyond our training data, we also show results on held-out CausalArena datasets (Table 1). Note that we do not have access to their underlying generating mechanisms.

W2: Invariance is useful precisely when predictions are imperfect, because graph-preserving changes can otherwise introduce unnecessary variation into the estimated structure.

We agree with the reviewer that a perfect graph estimator would already satisfy the relevant invariances. However, all causal discovery foundation models (CDFMs) use neural networks as function approximators, and therefore are imperfect learners, as ground-truth supervision alone cannot develop an ideal/perfect estimator. Table 2 (page 6) in our manuscript shows significant disagreement across views of the same graph for all CDFMs, motivating the need for consistency objectives that penalize sensitivity to changes irrelevant to the causal structure.

This also matters for finite-sample aggregation, as discussed in Section 3.4. A Monte Carlo average over graph-preserving views converges faster when the predictions across views vary less. This is exactly enforced by the invariance penalties (Tables 2 and 11, higher consistency with invariance losses). With identical training samples, the model trained with penalties is the most consistent across views and also gains the most from subgraph voting.

Additionally, enforcing the consistency losses teaches the model to ignore any data-specific artifacts and biases towards particular mechanism types, noise distributions, etc. Instead, these consistency losses teach the model to only focus on the underlying causal structure, thereby improving causal discovery. This is shown in Table 11 (page 30, Table 9 previously).

Q2: Imposing invariance losses improves graph recovery. Table 11 (page 30, Table 9 previously) compares 2M-parameter ALICE models trained with and without the invariance losses, showing that their inclusion improves synthetic-suite F1 from 0.4489 to 0.4866 before ensembling and from 0.5318 to 0.6004 after ensembling, while improving across all five realization, mechanism, and width tests, along with AUROC on real-world datasets as well. We request the reviewer to check the response to Reviewer Mht6 (W3, Q3) for further details on the importance of invariance losses.

Dear Reviewer,

We hope our rebuttal has adequately addressed the questions and concerns raised in your review. As we approach the end of the author–reviewer discussion period, we remain committed to clarifying any outstanding questions or addressing any further concerns regarding the manuscript or our responses.

Please let us know if any issues remain unresolved. If you feel that your concerns have been satisfactorily addressed, we would be very grateful if you could update your review and score accordingly.

Thank you again for your time and thoughtful feedback.

I thank the authors for the clarifications. I find the explanations regarding the real-world evaluations convincing, and the experimental results support the effectiveness of imposing invariance losses in improving graph recovery.

However, I am still not fully convinced by the theoretical analysis of why explicitly enforcing consistency is necessary. While reducing variation across views can make finite-sample aggregation more stable, this does not by itself explain when or why the resulting graph estimates should be more accurate. This connection remains insufficiently justified in the current analysis.

Overall, taking the authors’ clarifications and the empirical results into account, I will raise my score to Weak Accept.

We thank the reviewer for raising their score and for the continued engagement. We agree that the theoretical analysis and the aided discussion in Section 3.4 motivate reduced prediction variance across graph-preserving views, which results in effective finite-sample aggregation. It does not, by itself, establish that consistency losses improve the causal estimation accuracy.

Instead, we hypothesize that penalizing the model for predicting different causal structures when the ground truth labels are the same teaches the model to better extract the underlying causal structure. Our key intuition is that consistency losses help the model ignore spurious correlations between a specific causal structure and artifacts such as particular noise or mechanism families.

Table 11 provides empirical evidence that the losses improve graph recovery in our experiments. Additionally, Figure 10 (page 33) provides qualitative evidence for this hypothesis. The t-SNE visualization shows similar representations for datasets with the same causal structure.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary of the Discussion Phase

We thank all reviewers for their thoughtful feedback and engaged discussion, and for raising their scores (zD9k and Mht6). The reviewers found the problem practically relevant (zD9k, Mht6), the evaluation diverse and the real-world results strong (zD9k, J6Ft, Mht6), and target-conditioned inference an interesting design choice (Mht6). Below, we list the key empirical results added to the manuscript during the discussion phase to address the reviewers’ concerns.

1. Role of Invariance Losses (Mht6, zD9k). The newly added matched ablation separates the invariance penalties from the additional consistency samples they require (Table 11). With matched compute and data, the penalties improve consistency, identifiability, aggregation gains, and real-world performance, while the augmented samples alone do not. Both reviewers found this evidence convincing.

2. Addressing the biases in Subgraph Voting (Mht6). We state explicitly that omitting mediators or common causes can introduce spurious edges (Section 3.4, Limitations). We quantify this in Appendix G and observe that random sampling mitigates this effectively, and the overall gains on the full graph far outweigh this limitation.

3. Extended Identifiability Analysis (J6Ft). We added a new multivariate setting that contains causal structure with both identifiable and unidentifiable edges. ALICE provides the lowest deviation from ideal behavior (Appendix F), thereby showcasing that it exploits directional information, when available, better than baselines.

4. Clarity (J6Ft). We rewrote Section 3.4 to define views and aggregation, which the reviewer found cleaner.

5. Semantic Fusion (Mht6). We reframed fusion as a heuristic score and added a detailed analysis of different fusion rules (Table 12).

We thank the reviewers again for their constructive engagement, which has substantially improved the paper.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.