acceptodds
Under review as a conference paper at ICLR 2027

Beyond Candidate Voting: Transformer Aggregation Meets Observational Evidence

Abstract

Causal discovery algorithms often return different graphs because they rely on different functional, distributional, and conditional-independence assumptions. Candidate-only ensembles can reduce this instability, but their predictions remain limited to information retained in the submitted graphs. We introduce CausalFusionFormer, a causal discovery framework comprising progressively richer variants: globally learned candidate weighting, edge-adaptive transformer aggregation, an independent data-driven expert, and a full hybrid that routes between transformer-aggregated candidates and observational evidence. Our primary candidate pool contains PC, GES-BIC, ICA-LiNGAM, NOTEARS, and DAGMA. Each algorithm contributes its full-data estimate and two bootstrap perturbations, yielding a shared 15-graph evidence budget. On 90 previously untouched locked tasks, the full hybrid obtains a directed F1 of \(0.661\), compared with \(0.612\) for a shared-budget adaptation of Causal-Bayes-Ensemble (CBE) and \(0.608\) for our global candidate-weighting variant (vo2026wild). Relative to the CBE adaptation, it improves directed F1 by \(0.050\) (\(p=0.0091\)) and AUPRC by \(0.094\) (\(p<10^-4\)). The framework also exposes complementary operating points: uniform candidate voting provides the strongest skeleton recovery and normalized structural Hamming distance, while our global-weighting variant provides the best calibration. On continuous structural-equation data generated over unseen Asia, CHILD, and ALARM graph topologies, the full hybrid achieves the highest macro directed F1. On Sachs, our global candidate-weighting variant ties matched full-data PC at \(0.439\) directed F1, while the full hybrid matches the CBE adaptation. Together, these results show that edge-adaptive transformer aggregation and observational correction can improve directed causal recovery beyond candidate-only Bayesian voting, while complementary variants of the framework support different accuracy, calibration, and transfer requirements.

Primary area:
causal reasoning
Keywords:
Causal discovery, causal structure learning, transformer aggregation, ensemble learning, mixture of experts, algorithm selection, robust machine learning.

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_93f7OpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper introduces a new approach to Causal Discovery ensembling that takes into account various Causal Discovery method outputs as well as observational data statistics. It evaluates five ensembling variants with increasing complexity and compares them against individual Causal Discovery methods, an alternative ensembling approach and ammortized Causal Discovery.

Critical strengths

Ensembling of Causal Discovery methods is an interesting approach, especially with the help of observational statistics. The presented method is timely and provides a meaningful alternative to other ensembling or ammortized approaches to Causal Discovery.

The approach of testing the 5 different levels of ensembling is rigorous and comparing them all with the baseline Causal Discovery methods is solid. I also like the clear six data partitions with clear roles.

Generally the manuscript is very detailed and provides a lot of information on the specific design of the proposed method.

Critical weaknesses

While the paper is on a good way, however I have a number of concerns that I list below (not ordered by importance):

  1. There are many design choices in this method that are not really explained. Examples for this are the network architecture (Why 1/2 layers? Why exactly these 8 statistical data features? δ_max and g_max. Was there HP optimization involved? A general discussion on the design philosophy of the approach would be great. Exemplary, the candidate-budget ablation (G.15) is solid.

  2. The literature review is rather sparse. Specifically I am missing 2 additional ensembling approaches for Causal Discovery that I am aware of [1][2] and some more recent amortized approaches, e.g. [3][4].

  3. The paper is very dense and has no figures whatsoever. A depiction of the proposed ensembling approach and its components would likely help with comprehensibility.

  4. "Full regularizer definitions and coefficients are provided in Appendix E": Appendix E exists but is missing content.

  5. The real world data evaluation is quite sparse. An additional dataset would make it much easier to asses the performance of the approach out-of-distribution.

  6. Finally a more fundamental concern that is partly addressed already in the text: As all learned components are only trained on a very narrow training distribution, what is the reasoning why these learned modules should behave in a meaningful manner on out-of-distribution data? The results on Sachs seem to support this issue. For new data, the conditional patterns that your method can rely on will change. E.g. if no module will ever see non-gaussian noise during training, it will not like not trust noise-based approaches like LINGAM. While you noted this as future work, some more discussion on this topic would be great!

[1] Stein, Gideon, et al. "TCD-Arena: Assessing Robustness of Time Series Causal Discovery Methods Against Assumption Violations." arXiv preprint arXiv:2605.03045 (2026).

[2] Wu, Menghua, et al. "Sample, estimate, aggregate: A recipe for causal discovery foundation models." arXiv preprint arXiv:2402.01929 (2024).

[3] Blöbaum, Patrick, Krishnakumar Balasubramanian, and Shiva Prasad Kasiviswanathan. "FoundCause: Causal Discovery with Latent Confounders from Observational Data." arXiv preprint arXiv:2606.17516 (2026).

[4] Guan, Zhengkang, et al. "DCD-PFN: A Decoupling-Aware Foundation Model for Causal Discovery." arXiv preprint arXiv:2606.21212 (2026).

Questions for authors

Could you comment on your choice of relying on an edge independent weighting instead of a joint estimation of all links as in [1] ? What could be (dis)advantages?

What prevented you from training on a much broader synthetic distribution to alleviate the sym-to-real gap at least partially?

11 comments
AuthorsOpenReview ↗

Appreciation of Review (93f7)

We thank the reviewer for recognizing the timeliness of combining heterogeneous causal discovery methods with observational statistics, the value of our progressive five-variant evaluation, the breadth of the baseline comparisons, and the clearly separated roles of the six data partitions. We also appreciate their positive assessment of the manuscript’s technical detail.

Changes in the revised manuscript (as a result of your review) can be seen in cyan.

AuthorsOpenReview ↗

W1 – What motivated the architecture, feature set, and bounded design choices?

CaFuFo was designed as a sequence of controlled, low-capacity extensions to candidate voting rather than as a large unconstrained predictor. Each stage relaxes one limitation of the preceding stage: global weights capture average constituent reliability; a single self-attention layer enables edge-specific interactions among candidates; a bounded logit correction introduces limited observation-derived evidence; an independent Data-Only Expert permits predictions beyond candidate support; and the router controls how much of this less-constrained expert contributes to the final prediction. The design therefore increases flexibility progressively while deliberately preserving candidate evidence.

The architecture and bounds were fixed during pre-lock development and were not selected using labels from the locked evaluation tasks, topology-transfer tasks, or Sachs. We did not conduct a broad automated architecture or hyperparameter sweep. Model parameters were fitted on training tasks, the stopping epoch was selected using validation tasks, and temperature and graph threshold were selected using the separate calibration split.

Table 11 summarizes the main architectural choices and their provenance. The bounded components have direct operational interpretations. The reliability adjustment changes a candidate's prior contribution by at most a multiplicative factor between e−0.75e^{-0.75} and e0.75e^{0.75}, rather than permitting unrestricted replacement of the prior. The Candidate Anchor correction changes edge odds by at most a factor of e1e^1. Finally, gmax⁡data=0.9g_{\max}^{\mathrm{data}}=0.9 ensures that the full hybrid cannot completely discard candidate-derived evidence. We regard these values as conservative safeguards rather than empirically optimized constants.

ChoiceValueDesign rationaleSelection
Hidden width / heads48/448/4Compact representation with multiple candidate-interaction channelsFixed pre-lock
Candidate-transformer depth1 layerOne interaction step models candidate agreement and conflict while limiting capacityFixed pre-lock
Data-expert depth2 layersSlightly greater capacity because this branch lacks candidate-graph scaffoldingFixed pre-lock
Pair statistics8Compact observation-derived description of pairwise association, directional asymmetry, marginal structure, and task sizeHand specified
Reliability boundδmax⁡=0.75\delta_{\max}=0.75Prevents edge-specific reliability adjustments from departing arbitrarily from the global candidate priorFixed conservative bound
Correction boundλmax⁡=1\lambda_{\max}=1Limits the Candidate Anchor correction to one logit unitFixed conservative bound
Maximum data gategmax⁡data=0.9g_{\max}^{\mathrm{data}}=0.9Guarantees at least 10%10\% Candidate Anchor contributionFixed conservative bound
Stopping epochAt most 80/10080/100Controls optimization without changing model capacityValidation selected
Temperature / thresholdOne per method and seedCalibrates probabilities and extracts the final DAGCalibration selected

Table 11. Rationale and provenance of the principal architectural and optimization choices.

AuthorsOpenReview ↗

W2 – Coverage of related work.

We thank the reviewer for identifying these omissions. We have expanded the related-work section to discuss SEA and its sample-estimate-aggregate framework, the ensemble robustness analysis in TCD-Arena, and the recent amortized discovery models FoundCause and DCD-PFN.

Reviewer_RJ8FOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes CAFUFO, a supervised framework for combining heterogeneous causal graph estimates with observational features. It introduces edge-specific transformer weighting, a bounded correction to the aggregated prediction, and a task-level router that combines this prediction with a data-only expert. Experiments show improvements in edge ranking, while the advantages in graph recovery and transfer are less consistent.

Critical strengths
  • Combining complementary discovery algorithms with information retained in the observations is a reasonable practical motivation. The progressive variants make several components individually inspectable.

  • The evaluation separates training, validation, calibration, and held-out tasks, and reports paired comparisons and multiple structural metrics. The authors also openly report the hybrid model’s poor transfer to Sachs.

Critical weaknesses
  • The representation is not clear. Supervised aggregation is a legitimate approach to causal discovery, but the manuscript does not clearly explain which assumptions make its predicted orientations identifiable, particularly in the linear-Gaussian settings. Learning simulator-specific regularities and imposing an acyclic output do not themselves resolve observational equivalence. Evaluation of equivalence classes or identifiable orientations would help distinguish improved causal recovery from better prediction under the synthetic training prior.

  • Several observational features appear to lose their intended information under the stated preprocessing. Appendix C.1 standardizes every variable before feature extraction, making the mean-difference and log-scale-ratio features approximately zero. For standardized variables, the two univariate linear regressions also have equal residual variances, making the fifth feature approximately zero. Furthermore, C.2 standardizes node features across nodes, while C.4 summarizes them using their means and standard deviations across those same nodes. These summaries are therefore essentially constants for nondegenerate coordinates. This raises a concrete concern about whether the implemented representation provides the directional and distributional information attributed to it.

  • The bounded correction is not clearly shown to address the motivating problem of edges missed by every candidate. When candidate support is zero, clipping followed by a bounded logit adjustment leaves the uncalibrated probability very small. The authors should demonstrate whether this mechanism meaningfully recovers unanimously omitted edges after calibration and decoding, and distinguish its contribution from that of the independent data-only expert.

  • The empirical support for the full framework remains mixed. Its directed-F1 improvement over uniform voting is not statistically significant, while skeleton recovery, structural Hamming distance, and calibration are worse. The significant AUPRC improvement is valuable, but primarily supports better edge ranking. Moreover, routing does not significantly improve directed F1 over the candidate anchor on the locked tasks and substantially reduces performance on Sachs, leaving its reliability under distribution shift unresolved.

  • The paper lacks discussion of recent related work and does not clearly identify limitations of existing methods or how the proposed approach addresses them.

Sample, Estimate, Aggregate: A Recipe for Causal Discovery Foundation Models. TMLR 2025

Pairwise Causal Discovery with Support Measure Machines. Applied Soft Computing, 150, 111030.

Causal Discovery for Observational Sciences Using Supervised Machine Learning. Journal of Data Science, 21(2), 255–280.

Towards a Learning Theory of Cause-Effect Inference. ICML 2015

Efficient Causal Structure Learning via Modular Subgraph Integration. arXiv preprint: 2601.21014

Stable Causal Discovery via Directed Acyclic Graph Aggregation. arXiv: 2605.18633

Causal Ensemble Agent: Hierarchical Causal Discovery with LLM-guided Expert Reweighting. arXiv preprint: 2606.10607

Questions for authors
  • Could the authors confirm the preprocessing order and report the empirical variation of the potentially degenerate features and router summaries?

  • How does CAFUFO differ from SEA beyond the candidate construction, and how does it compare with a simpler learned aggregator using identical inputs?

  • How many true edges absent from every candidate are recovered by the anchor and hybrid after calibration and decoding?

  • Which identifying assumptions support the orientation claims, and do the conclusions hold when evaluation accounts for Markov equivalence?

13 comments
AuthorsOpenReview ↗

Appreciation of Review (RJ8F)

We thank Reviewer RJ8F for recognising the key strengths of our work, particularly the practical motivation for combining complementary discovery algorithms with observational information, the transparency and inspectability of our progressive variants, and the rigorous experimental design separating training, validation, calibration, and held-out evaluation. We also appreciate the reviewer acknowledging our comprehensive structural evaluation and transparent reporting of the hybrid model's limited transfer to the Sachs dataset.

Changes in manuscript as a result of your review are marked in red.

AuthorsOpenReview ↗

W1 – What assumptions support orientation, and does performance persist under equivalence-aware evaluation?

We thank the reviewer for raising this important question. We agree that supervised prediction and acyclic decoding do not, by themselves, resolve observational equivalence. CaFuFo is a supervised graph aggregator, not a new identifiability result. Under the standard Markov and faithfulness assumptions, purely observational linear-Gaussian data generally identify only a Markov equivalence class, represented by a CPDAG. Orientations that are not compelled by the CPDAG may therefore reflect additional assumptions made by the constituent algorithms or regularities learned from the synthetic training distribution. The DAG decoder enforces acyclicity, but provides no additional identifying information. We have revised Section 1 of the paper to make this distinction explicit.

To separate recovery of observationally identifiable orientations from prediction under the simulator prior, we conducted an additional equivalence-aware evaluation. For each locked evaluation task, we convert the ground-truth DAG to its CPDAG and evaluate orientation recovery only on edges whose directions are compelled by that CPDAG. We additionally examine equivalence-class structure and report the linear-Gaussian subset separately. This analysis directly tests whether CaFuFo's apparent advantage in directed-edge recovery persists when evaluation is restricted to orientations supported by the observational equivalence class, rather than potentially arbitrary directions selected by the simulator.

Equivalence-aware orientation recovery.

Table 7 reports compelled-orientation recovery together with paired statistical comparisons. The Observable Hybrid achieves the highest overall micro-F1 (0.4340.434), outperforming the Candidate Anchor (0.3700.370), Global Candidate Weights (0.2950.295), and Uniform Vote (0.2570.257). It also achieves the highest overall precision (0.3680.368) and recall (0.5280.528). Paired task-level comparisons show significant improvements over all three alternatives (p<0.01p<0.01).

Importantly, the advantage is not universal. On the linear-Gaussian subset, Global Candidate Weights achieves a higher compelled-orientation F1 (0.3520.352) than the Observable Hybrid (0.3050.305). Thus, CaFuFo's overall orientation advantage persists under equivalence-aware evaluation, but its benefit depends on the underlying data-generating mechanism.

MethodPrecision ↑Recall ↑F1 ↑LG-F1 ↑Δ\DeltaF1 ↑pp-value
Observable Hybrid0.3680.5280.4340.305----
Candidate Anchor0.3310.4190.3700.285+0.02360.0067
Global Candidate Weights0.2710.3250.2950.352+0.06400.0097
Uniform Vote0.3540.2020.2570.223+0.11720.0015

Table 7. Equivalence-aware compelled-orientation recovery. Precision, recall, and F1 are pooled micro-averaged metrics across the 90 locked tasks. LG-F1 denotes compelled-orientation micro-F1 on the 30 linear-Gaussian tasks. Δ\DeltaF1 reports the mean paired task-level improvement of the Observable Hybrid over each comparator; positive values favour the hybrid. The pp-values are obtained from paired randomisation tests using 90, 89, and 85 valid task-level comparisons, respectively.

AuthorsOpenReview ↗

W2 (Part 1) – Are the observational features informative after preprocessing?

We thank the reviewer for identifying this important issue. We agree that the original description overstated the information available in several observation-derived features after preprocessing.

Feature audit. Because variables are standardized before pair-feature extraction, the standardized mean difference, log scale ratio, and two-regression residual-variance ratio become numerically degenerate. Similarly, standardizing node features across nodes before computing within-task means and standard deviations renders several router summaries constant or redundant.

We audited all feature coordinates across 162 training, validation, calibration, development, and router tasks. Three of the eight pair-feature coordinates are numerically degenerate, while five retain empirical variation. Among the observation-derived pair statistics, the two asymmetric cross-moment features are the remaining direction-sensitive coordinates. Numerical variation alone, however, does not establish predictive utility or causal identifiability.

Coordinate or groupEmpirical rangeNear-zero rateConclusion
Standardized mean difference8.39×10−98.39\times10^{-9}100.0%100.0\%Degenerate
Log scale ratio1.10×10−71.10\times10^{-7}100.0%100.0\%Degenerate
Log residual-variance ratio2.20×10−72.20\times10^{-7}99.99%99.99\%Degenerate
Correlation1.9980.0%0.0\%Varying
Absolute correlation1.0000.0%0.0\%Varying
Two directional cross-moments1.298 each0.0%0.0\%Varying
Log sample size0.1200.0%0.0\%Varies across tasks

Table 8. Empirical audit of the observation-derived pair features.

Control 1: Zero-masking degenerate pair features. We trained a same-dimensional control in which the three degenerate pair-feature coordinates were fixed to zero during both training and evaluation. This preserves the architecture and parameter count while eliminating any residual numerical variation.

VariantMetricDifference95% CI
Observable HybridDirected F1−0.0074-0.0074[−0.0156,  0.0004][-0.0156,\;0.0004]
Observable HybridAUPRC−0.0031-0.0031[−0.0078,  0.0016][-0.0078,\;0.0016]
Candidate AnchorDirected F1−0.0037-0.0037[−0.0110,  0.0036][-0.0110,\;0.0036]
Candidate AnchorAUPRC−0.0069-0.0069[−0.0107,  −0.0028][-0.0107,\;-0.0028]

Table 9. Same-dimensional zero-mask control. Differences are zero-masked minus original performance.

The changes are small. Importantly, the lower confidence bounds for all four comparisons remain above the predeclared non-inferiority margin of −0.02-0.02. Thus, the zero-masked variants satisfy the specified non-inferiority criterion for both directed F1 and AUPRC. Additional skeleton F1, nSHD, and ECE evaluations likewise show no practically substantial deterioration.

These results indicate that the degenerate pair-feature coordinates are not required to maintain the reported predictive performance.

Reviewer_Br47OpenReview ↗
Rating
2
Confidence
1
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes CAFUFO, which aggregates candidate graphs produced by multiple causal discovery algorithms using a Transformer and combines this aggregation with a correction based on observational data and an independent data expert. It learns a router that controls the balance between the candidate aggregation output and the data expert's predictions, and evaluates the method on a locked synthetic evaluation set, data generated from new graph structures, and the Sachs dataset. The main result is an improvement in AUPRC, a measure of the ranking quality of edge probabilities, in the synthetic evaluation. However, the method does not show consistent gains over simpler approaches in recovering the final graph structure or transferring to real data.

Critical strengths
  1. The problem setting is practically motivated. Using observational information alongside candidates from causal discovery algorithms that rely on different assumptions is well motivated. Several variants compare approaches that aggregate only candidates with those that use only data (Section 3, pp. 4–6).
  2. The evaluation procedure is relatively transparent. The paper distinguishes the roles of training, validation, calibration, development, transfer, and locked evaluation, and uses the same candidate budget. It reports paired differences and confidence intervals in addition to mean performance (p. 7, Appendix G).
  3. The improvement in edge ranking is a clear positive result. In the locked evaluation, the hybrid achieves an AUPRC of 0.759, compared with 0.702 for uniform vote and 0.722 for the anchor. In particular, the difference of 0.0364 relative to the anchor has a 95% confidence interval of [0.0267, 0.0466] (Tables 1 and 11; pp. 7 and 23). The transparent reporting of the failure on real data is also a strength.
Critical weaknesses
  1. The structural recovery gains are insufficient to justify the full model's additional complexity. In the locked evaluation, directed F1 is 0.661 for the hybrid, 0.651 for uniform vote, and 0.659 for the anchor. The paired difference between the hybrid and uniform vote is 0.0103, with a 95% confidence interval of [-0.0112, 0.0315] and a p-value of 0.5351. The difference relative to the anchor is also not significant (Table 11, p. 23). This does not establish equivalence between the methods, but it also does not provide strong evidence that the full model improves structural recovery. Uniform vote performs better in skeleton F1 and nSHD (Table 1). While the AUPRC improvement is valuable, the current results do not sufficiently isolate the need for the complex architecture. In addition to the existing variant comparisons, a controlled comparison with simple nonlinear candidate aggregation is needed.

  2. There is weak evidence that the router usefully selects experts under real distribution shift. On Sachs, directed F1 is 0.258 for the hybrid, 0.397 for the anchor, and 0.439 for global candidate weights (Table 3, p. 9). Thus, the pathway that adds observational information can actually reduce performance on real data. The Asia, CHILD, and ALARM evaluations constitute synthetic transfer using continuous SEMs on new structures, so these results alone do not resolve the transfer problem revealed by Sachs (Section 4.2, p. 8). Although the failure is transparently reported, the paper provides insufficient explanation of the observable conditions under which the router can be trusted.

  3. The central explanation of correcting shared candidate errors is not directly validated. The anchor adds a bounded correction to the logit of the aggregated candidate probability (Equation (2), Appendix D.3). Because of clipping and the correction bounds, an edge omitted by all candidates is unlikely to receive a high probability from the anchor alone. Recovery of such edges would therefore appear to depend primarily on the independent data expert. However, the data expert itself achieves a directed F1 of 0.315 in the locked evaluation (Table 1), and aggregate performance comparisons do not reveal how many commonly omitted edges it recovers or how many false positives it introduces. A branch-specific analysis of this edge set is needed to support the proposed mechanism.

Questions for authors

NA

6 comments
AuthorsOpenReview ↗

Appreciation of Review [Br47]

We thank the reviewer for the careful and constructive assessment. We appreciate their recognition of the practical motivation for combining heterogeneous causal-discovery candidates with observational evidence, the transparency of our partitioned and locked evaluation protocol, and the value of the statistically supported AUPRC improvement. In particular, the reviewer highlights that the observable hybrid improves AUPRC from 0.7220.722 for the candidate anchor to 0.7590.759. We also appreciate their acknowledgement of our transparent reporting of the mixed real-data transfer result.

The additional analyses below directly address the reviewer’s concerns about architectural complexity, router reliability under distribution shift, and recovery of edges omitted by all candidates.

Changes in the revised manuscript (as a result of your review) can be seen in blue.

AuthorsOpenReview ↗

W1a – Does the full architecture improve over a simple nonlinear aggregator?

To address this concern directly, we designed a controlled replacement experiment: the DeepSets baseline receives the same candidate tokens and observational features and uses the same task partitions, objectives, calibration split, and DAG decoder, but replaces self-attention with a shared nonlinear candidate encoder and mean pooling.

This comparison tests whether the observed gains in CaFuFo require modelling interactions among candidates through attention or can be reproduced by a simpler nonlinear set aggregator. If attention is unnecessary, DeepSets should match or outperform the transformer across the progressively richer variants. Conversely, improvements confined to the candidate-anchor and hybrid variants would indicate that attention is useful specifically when candidate agreement and conflict must inform observational correction or routing.

Table~1 reports paired differences on the 90 locked tasks -- here, positive values favour the transformer. We report paired differences rather than replacing the original headline results with absolute scores from this post-hoc rerun. It can be seen that DeepSets performs better for weights-only aggregation, showing that self-attention is not intrinsically superior for candidate reweighting. In contrast, the transformer significantly improves directed F1 for both the candidate anchor (+0.0272+0.0272, p<10−4p<10^{-4}) and observable hybrid (+0.0267+0.0267, p<10−4p<10^{-4}), while also improving skeleton F1 and nSHD. Their AUPRC values are statistically indistinguishable.

VariantΔ\Delta Dir. F1 [95% CI]ppΔ\Delta AUPRC [95% CI]Δ\Delta Skel. F1Δ\Delta nSHD
Weights only−0.0125  [−0.0188,−0.0061]-0.0125\;[-0.0188,-0.0061]0.00020.0002−0.0095  [−0.0163,−0.0027]-0.0095\;[-0.0163,-0.0027]+0.0077+0.0077−0.0216-0.0216
Candidate anchor+0.0272  [0.0169,0.0380]+0.0272\;[0.0169,0.0380]<10−4<10^{-4}+0.0023  [−0.0015,0.0060]+0.0023\;[-0.0015,0.0060]+0.0306+0.0306+0.0969+0.0969
Observable hybrid+0.0267  [0.0175,0.0360]+0.0267\;[0.0175,0.0360]<10−4<10^{-4}−0.0008  [−0.0045,0.0029]-0.0008\;[-0.0045,0.0029]+0.0207+0.0207+0.1199+0.1199

Table 1. Transformer minus matched DeepSets performance on the 90 locked tasks. Positive values favour the transformer.

This experiment refines our attribution of the gains: observable hybrid drives the AUPRC improvement, which DeepSets largely reproduces, whereas transformer-based candidate interactions improve final structural recovery, including directed F1, skeleton F1, and nSHD.

We have revised paper section 4.1 to reflect this distinction, and added the complete DeepSets protocol and paired results to the appendix.

AuthorsOpenReview ↗

W1b – Re Complexity justification

We agree that the locked comparison does not establish a statistically significant directed-F1 advantage of the observable hybrid over either uniform voting or the candidate anchor. However, we respectfully disagree that these findings leave the richer design unsupported. First, the hybrid’s AUPRC advantage is both substantial and statistically supported: +0.0564+0.0564 over uniform voting (95% CI [0.0362,0.0761][0.0362,0.0761], p=0.0105p=0.0105) and +0.0364+0.0364 over the candidate anchor ([0.0267,0.0466][0.0267,0.0466], p<10−4p<10^{-4}). This demonstrates improved directed-edge ranking even where the final thresholded F1 values are close.

Second, (as shown in W1a) our new matched nonlinear-control experiment directly isolates architectural complexity. Relative to an otherwise matched DeepSets hybrid, the transformer hybrid improves directed F1 by 0.02670.0267 ([0.0175,0.0360][0.0175,0.0360], p<10−4p<10^{-4}), together with significant improvements in skeleton F1 and nSHD, while AUPRC is statistically tied. We, therefore, interpret our results more narrowly than universal dominance: CaFuFo provides the strongest directed-edge ranking and benefits from transformer interactions in matched structural comparisons, while uniform voting remains preferable for skeleton recovery and nSHD. Its strong performance also motivates future variants with stronger regularization toward uniform candidate priors. \textbf{We have revised the manuscript to add this as future works}.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary of Revisions and New Analyses

Dear Reviewers and Area Chair,

Thank you for the detailed and constructive reviews. We have revised the manuscript substantially and summarize the principal changes below.

In response to Reviewer Br47, we added three targeted diagnostic analyses:

  1. Matched nonlinear aggregation control (Appendix H). We compare the transformer with a no-attention DeepSets control using identical inputs, training partitions, calibration, and decoding. This isolates where candidate-to-candidate attention improves decoded structural recovery.
  2. Router reliability audit (Appendix I). We evaluate whether the router’s gate tracks the relative advantage of the candidate anchor and data-only expert. The audit supports meaningful selection within represented synthetic regimes, while clarifying that this behaviour does not reliably transfer under real-data shift.
  3. Common-miss recovery analysis (Appendix J). We identify which branch recovers true edges omitted by every candidate and quantify the associated false-positive trade-off. This confirms that novel-edge evidence originates from the independent data branch, while routing substantially filters its false positives.

The main locked-results discussion has been updated to summarize these findings and narrow the corresponding claims.

In response to Reviewer RJ8F, we:

  1. Added an explicit discussion of identifiability and Markov equivalence in the Introduction, clarifying that CaFuFo is a supervised aggregation framework rather than a new identifiability result.
  2. Added a CPDAG-aware evaluation of equivalence-class structure and orientations compelled by the observational distribution.
  3. Audited the observational representation across 162 tasks, documented the empirically constant coordinates, corrected the feature descriptions, and added zero-mask and reduced-router controls. Appendix C has been rewritten accordingly.
  4. Expanded the related-work discussion to cover supervised cause–effect inference, recent aggregation methods, and causal-discovery foundation models.
  5. Revised the conclusion to identify stronger regularization toward candidate priors, including uniform weighting, as an important direction for future work.

In response to Reviewer 93f7, we:

  1. Added an illustrative architecture overview that makes the candidate-fusion, observational-correction, data-expert, and routing pathways explicit.
  2. Rewrote Appendix E to provide the previously omitted definitions of all regularizers and their coefficients.
  3. Added an architecture and computational audit reporting parameter counts, neural inference time, candidate-generation cost, hyperparameter purposes, and whether each choice was fixed or selected using validation or calibration tasks.
  4. Added a real-data stress test on the Tübingen Cause–Effect Pairs benchmark (Appendix L).
  5. Expanded the discussion of edge-wise versus joint graph estimation, the scope of the synthetic training distribution, and the limitations of synthetic-to-real transfer.

A recurring theme across the reviews was out-of-distribution generalization. We have therefore narrowed the paper’s claims. On Sachs, the observable hybrid does not improve over the more conservative candidate variants. On Tübingen, however, the CaFuFo Candidate Anchor achieves the highest weighted direction accuracy (0.6370.637), slightly exceeding ICA-LiNGAM (0.6310.631) and outperforming uniform voting (0.5540.554), PC (0.5200.520), and GES-BIC (0.5200.520). These results suggest that bounded candidate-anchored correction can transfer promisingly, while the synthetic-trained data-only branch and router remain vulnerable to unrepresented real-data shifts.

We again thank the reviewers for their careful and high-quality feedback. The revised manuscript incorporates these analyses, corrections, and clarifications, and we hope that it addresses the principal concerns raised during review.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.