acceptodds
Under review as a conference paper at ICLR 2027

Test Time Training for Supervised Causal Learning

Abstract

Supervised Causal Learning (SCL) has shown promise in causal discovery by framing it as a supervised learning problem. However, it suffers from significant out-of-distribution generalization challenges. We reveal three limitations of previous SCL practices: a significant performance gap between synthetic benchmarks and real-world data, fragility to distribution shifts, and failure in compositional generalization, collectively questioning its real-world applicability. To address this, we propose Test-Time Training for Supervised Causal Learning (TTT-SCL), a framework that dynamically generates training sets aligned with the current test dataset, on which a new supervised learner is trained from random initialization. We demonstrate the correlation between TTT-SCL and score-based methods, and design an efficient module for generating training sets based on the classic scoring function. Experiments on synthetic, pseudo-real, and real-world datasets demonstrate competitive performance and improvements over the compared baselines in challenging settings.

Primary area:
transfer learning, meta learning, and lifelong learning
Keywords:
Causal discovery, Test time training, Supervised causal learning

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_pB8DOpenReview ↗
Rating
2
Confidence
2
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper investigates the out-of-distribution generalization limitations of Supervised Causal Learning (SCL). The authors show that existing SCL methods suffer significant performance degradation on real-world data, under distribution shifts, and on unseen combinations of causal components. To address these challenges, they propose TTT-SCL, which dynamically constructs training data tailored to each test dataset through score-guided graph search and causal mechanism fitting. Experiments demonstrate competitive performance across synthetic, pseudo-real, and real-world benchmarks.

Critical strengths
  1. The paper provides an empirical analysis of the generalization limitations of existing SCL methods, including performance degradation under distribution shifts and failures on unseen combinations of causal components. These findings provide clear motivation for the task.

  2. The proposed TTT-SCL framework offers a promising approach for addressing the out-of-distribution generalization challenges of existing SCL methods by dynamically constructing instance-specific training data at test time.

Critical weaknesses
  1. The computation cost seems to be very high. It requires graph search, mechanism fitting, and retraining a supervised model for each test dataset, which raises concerns about its scalability and practical applicability to larger causal discovery problems.

  2. The paper does not fully isolate the contribution of test-time supervised learning from the score-guided graph search. Although it compares TTT-SCL with graph voting baselines, these methods use different candidate graph sets. It doesn't isolate the contribution of the supervised learner. A comparison using identical candidate graphs would provide stronger evidence for the independent contribution of the supervised learner.

Questions for authors
  1. Could the authors provide a comparison between TTT-SCL and graph voting baselines using identical candidate graph sets? This would help clarify how much of the performance improvement comes from the supervised learner itself.

  2. Given that the synthetic training data are generated from mechanisms fitted to the observed test dataset, could the authors clarify why this process should improve OOD robustness? Since the generated data are still constrained by the information available in the original observations and the assumptions of the fitted mechanisms, it is unclear how the method can handle causal mechanisms or distributional variations that are not adequately captured by the observed data.

3 comments

We are very grateful for the reviewers' efforts, and the following are our responses to the questions.

Q1/W2.Isolating the contribution of the supervised learner.

We agree that if the voting baseline uses a different set of candidate graphs than Full TTT-SCL, performance differences may arise from both candidate selection and subsequent learning, thus failing to adequately account for additional contributions during the learning phase. To address this, we supplement this with a control experiment using a fixed set of candidate graphs: based on the same batch of candidate graphs actually used in Full TTT-SCL, predictions are obtained using the following three methods:

Voting: Equal-weighted averaging of the adjacency matrices within the same candidate set.

Score-weighted voting: Weighted averaging of the same candidate set based on graph scores to obtain continuous edge scores.

Full TTT-SCL: Using the same candidate set, a supervised learner is trained on training data constructed using SIM and used to predict on test data.

Table R2. Ablation experiments with the same candidate graph set.

Method                     RFF_GLinear_UChebyshev_G   Sachs SynTReN
Top KK voting              90.1    79.8       80.2    66.7    57.9
Score weighted voting      90.6    81.9       81.3    65.8    58.6
Full TTT-SCL            91.886.3   83.078.980.1

In all five settings, Full TTT-SCL outperformed both voting baselines. Its AUROC improved by 1.2, 4.4, 1.7, 12.2, and 21.5 percentage points relative to the best-performing voting methods in each setting, with larger improvements on Sachs and SynTReN. The use of the same candidate set by all three methods supports the additional value of the supervised learning phase.

Q2.Why TTT-SCL improve OOD robustness

The potential benefit of TTT-SCL comes from adapting the learner’s training distribution to the current test dataset. A distinction is that useful statistical information may be present in the test dataset, while a model trained on a fixed distribution may not have learned how to use that information under the conditions encountered at test time.

Because reliable causal graph annotations are scarce, supervised causal learning relies heavily on synthetic training data. The synthetic generation process therefore determines which graph structures, mechanisms, and noise characteristics the learner encounters during training. When a test dataset differs from this training distribution, the learned mapping from observations to graphs may perform poorly.

TTT-SCL addresses this mismatch by using the current observations to construct the training tasks themselves. For each test dataset, we use its observations to guide candidate graph construction, fit mechanisms under these candidates, and generate corresponding graph-labeled datasets. We then train a newly initialized SCL model on these tasks and apply it to the observed dataset. TTT-SCL provides a dataset-specific training strategy within SCL, it reduces reliance on a single, fixed pretraining distribution anticipating each future task.

Thank you for the clarification and additional experiments. The new ablation study addresses my concern about isolating the contribution of the supervised learner. However, my main concern regarding OOD robustness remains unanswered. While the method may help the model better utilize information from the observed test data, it is still unclear to me why this process should consistently improve OOD robustness. Since the synthetic training data are constructed from the original observations and candidate graph assumptions, how does the method ensure that the learned relationships reflect the underlying causal structure rather than environment-specific statistical patterns that may not generalize under distribution shifts? The current explanation may provide a reasonable intuition for test-time adaptation, but I believe stronger justification or more direct experimental evidence is needed to support the claimed OOD robustness. Therefore, I will maintain the current score.

We greatly appreciate the reviewer's further description of their confusion regarding how to resolve OOD, which gives us the opportunity to explain further. We previously misunderstood your question; the core issue is that there are actually two different OODs within this topic . We will now explain this in more detail to hopefully clear up your confusion.

(1) First, we want to clarify the meaning of OOD robustness in this paper. TTT-SCL constructs training data, trains a learner, and predicts its causal structure separately for each current test dataset; when faced with another dataset, we repeat this process. Therefore, we do not require the currently generated training data or trained learner to be suitable for other testing environments, which is precisely the meaning of instance-specific. Compared to static SCL, TTT-SCL changes the way of relying on a fixed pre-trained distribution to cover subsequent test tasks from a methodological design perspective.

(2) Secondly, we understand your concerns and the OOD in your comments: even when generating training data for the current test data, a mismatch may still exist between the generated distribution and the true test distribution. This is mainly affected by the quality and coverage of the candidate graph, the ability of SIM to fit mechanisms and noise, and the limitations of limited observation data.

We do not require the two distributions to be completely identical, nor do we believe that distribution consistency is sufficient to determine the true causal graph. The role of SIM is to construct training data generated from explicit candidate causal graphs while incorporating the statistical characteristics of the current test data, so that the mapping between the SCL data and the graph structure is not overly influenced by the distribution. What needs to be preserved here is the statistical information that helps distinguish causal structures, not just general distributional similarity. The existing experiments in Table 2 of our paper, showing that TTT-SCL outperforms the static SCL baseline AVICI on both Sachs and SynTReN, provide empirical support for this approach. We do not know the mechanisms of these two test data, and the generated training data has a certain distribution mismatch with the test data, but still achieves good practical performance.

Furthermore, the two main technical aspects causing the mismatch can be improved within the TTT-SCL framework: candidate graph construction can employ better initialization and search methods, and SIM can use more flexible mechanisms and noise models that are compatible with the causal assumptions used. These improvements provide ways to further reduce the mismatch between the generated training and test distributions.

Reviewer_5nvoOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper addresses the generalization challenge in supervised causal learning (SCL) under distribution shifts. The authors identify three primary out-of-distribution (OOD) limitations inherent in existing SCL methods based on static pre-training: the generalization gap when transitioning from synthetic to real-world data, sensitivity to shifts in causal mechanisms, graph structures, or noise, and failures in compositional generalization. To overcome these issues, the authors propose TTT-SCL, a test-time training framework tailored to the specific instance encountered during testing. Specifically, the method begins by performing a score-guided local graph search on the test data to identify and retain a set of candidate directed acyclic graphs (DAGs). It then employs a "Structure-Induced Mechanism" (SIM) to fit node-level mechanisms based on the test data and generates synthetic, graph-labeled training data derived from these candidate structures. Finally, a new SCL model is trained from a random initialization to predict the final causal graph. Experiments conducted on synthetic data, the pseudo-real dataset SynTReN, and the real-world Sachs dataset demonstrate that TTT-SCL achieves competitive—and in many cases, superior—performance compared to strong baselines across a variety of challenging settings.

Critical strengths

A clear test-time training framework is proposed, which integrates traditional score-based graph search with supervised causal learning. For each test dataset, the method constructs candidate causal graphs and generates corresponding supervised training data via SIM (Structural Intervention Modeling). This effectively transforms pure graph search into a downstream learning process, resulting in a distinct two-stage search -> learning design.

The experimental evaluation is comprehensive, covering i.i.d. settings, mechanism/graph/noise shifts, and compositional generalization across synthetic, SynTReN, and Sachs benchmarks. The paper also provides extensive analyses through sparsity ablations, staged comparisons, initialization sensitivity, backbone studies, and runtime evaluation.

Critical weaknesses

The source of the core performance gains remains insufficiently isolated, while the candidate graph construction process relies on several heuristic design choices. Existing voting baselines do not use the exact same candidate pool as Full TTT-SCL, making it difficult to disentangle the contribution of the supervised learner from that of candidate construction and selection.

Moreover, the fixed-unit-variance Gaussian likelihood used for scoring, together with the stochastic acceptance rule and SIM-based data generation, may substantially affect both candidate quality and the resulting training distribution, yet the robustness of these design choices is not thoroughly evaluated.

Finally,the fairness of directly comparing AUROC/AUPRC is somewhat unclear, as some baselines output binary graphs while TTT-SCL produces continuous edge scores, and no score-based sampler with marginal edge probabilities is included. In addition, sensitivity to key choices such as KK, the edge-penalty coefficient, and SIM configurations is not systematically studied. Real-world validation is also largely limited to Sachs, without evaluation on more conservative identifiable targets such as CPDAGs.

Questions for authors

The voting baselines and Full TTT-SCL in Appendix K use different candidate pools. Could the authors compare voting, score-weighted aggregation, and SCL on the same final-KK states to better isolate the contribution of the supervised learner?

How sensitive is TTT-SCL to KK, the edge-penalty coefficient, and the score/acceptance scale? These choices are largely fixed in the current experiments, and a more systematic sensitivity analysis would strengthen the evaluation.

Both SIM and likelihood-based AD rely on GAMs with fixed unit Gaussian noise. How is this assumption justified for non-Gaussian test distributions, and how sensitive is the method to alternative regression or noise specifications?

3 comments

We are very grateful for the reviewers' efforts, and the following are our responses to the questions.

W1/Q1.Isolating the contribution of the supervised learner.

We agree that if the voting baseline uses a different set of candidate graphs than Full TTT-SCL, performance differences may arise from both candidate selection and subsequent learning, thus failing to adequately account for additional contributions during the learning phase. To address this, we supplement this with a control experiment using a fixed set of candidate graphs: based on the same batch of candidate graphs actually used in Full TTT-SCL, predictions are obtained using the following three methods:

Voting: Equal-weighted averaging of the adjacency matrices within the same candidate set.

Score-weighted voting: Weighted averaging of the same candidate set based on graph scores to obtain continuous edge scores.

Full TTT-SCL: Using the same candidate set, a supervised learner is trained on training data constructed using SIM and used to predict on test data.

Table R2. Ablation experiments with the same candidate graph set.

Method                     RFF_GLinear_UChebyshev_G   Sachs SynTReN
Top KK voting              90.1    79.8       80.2    66.7    57.9
Score weighted voting      90.6    81.9       81.3    65.8    58.6
Full TTT-SCL            91.886.3   83.078.980.1

In all five settings, Full TTT-SCL outperformed both voting baselines. Its AUROC improved by 1.2, 4.4, 1.7, 12.2, and 21.5 percentage points relative to the best-performing voting methods in each setting, with larger improvements on Sachs and SynTReN. The use of the same candidate set by all three methods supports the additional value of the supervised learning phase.

Q2. Sensitivity of hyperparameters.

We have already conducted some sensitivity experiments on the edge-penalty coefficient in our paper. Here, we will supplement this with further sensitivity analysis on K.

(1)Penalty coefficient. Section 5.2, Table 3 in paper reports a directly relevant ablation: the full method retains the default sparsity penalty, whereas TTT-SCL (Notears-s) removes this term, corresponding to setting its weight to zero. The reported AUROC values (%) are:

Table 3 in Section 5.2: Ablation experiment of sparsity.

ConfigurationRFF_GLinear_UChebyshev_GSachsSynTReN
With sparsity penalty91.886.383.078.980.1
Without sparsity penalty86.884.369.763.576.1

Removing the penalty decreases AUROC by 5.0, 2.0, 13.3, 15.4, and 4.0 percentage points across the five settings, respectively. These results support considering both data fit and structural complexity during candidate construction: using the fitting term alone does not match the downstream performance of the full method in these comparisons.

(2) Candidate count KK. We compare the default configuration, K=200K=200, with a larger candidate budget, K=2000K=2000. The AUROC results (%) are:

Table R3.Ablation experiment on K

ConfigurationRFF_GLinear_UChebyshev_GSachsSynTReN
TTT-SCL (K=200K=200)91.886.383.078.980.1
TTT-SCL (K=2000K=2000)92.187.883.673.477.8

Increasing KK tenfold improves AUROC by 0.3, 1.5, and 0.6 percentage points on the three synthetic settings, respectively, but reduces it by 5.5 and 2.3 percentage points on Sachs and SynTReN. Thus, retaining more candidate graphs does not necessarily improve downstream performance. In these evaluated settings, the default K=200K=200 already achieves competitive performance, while increasing it to K=2000K=2000 does not yield consistent gains. We believe that if the mechanism fit is good enough, increasing K will improve performance to some extent (but will increase training costs accordingly); however, if the mechanism fit is not good enough, blindly increasing K may not necessarily improve performance.

(3) Score/acceptance scale. We agree that this scale affects the probability of accepting lower-scoring proposals and can therefore influence the search trajectory, candidate pool, and resulting training distribution. A systematic comparison requires rerunning candidate search, synthetic data generation, and learner training. We were unable to complete these experiments within the rebuttal period. We will provide a score/acceptance-scale sensitivity analysis in a subsequent version, explicitly specifying the score normalization and acceptance-scale definition and comparing their effects under fixed search and training budgets.

W2/Q3.Implementation of GAMs with fixed unit Gaussian noise

We wish to clarify that the current GAM and noise model constitute a specific implementation of TTT-SCL, not a requirement of this training paradigm. The core of TTT-SCL is constructing candidate graphs based on current observations and then supervising learning during testing; this process does not require the mechanism fitter to use GAM, nor does it require the simulated noise to be fixed to a unit Gaussian distribution (as we provide several candidates for AD calculation in Appendix D).

We use GAM because it can fit smooth, nonlinear conditional means under additive structures and provides a unified node-by-node fitting process for multiple candidate graphs. The current SIM uses independent unit Gaussian noise to provide an explicit and reproducible initial implementation. This choice does not imply that we assume all test systems have the same noise distribution or scale.

We agree that this simplification may lead to model mismatch. But from a framework perspective, the mechanism module can be replaced with a regression model capable of expressing more complex parent variable interactions; the noise module can adopt a scheme that estimates the scale by nodes or fits the residual distribution. Therefore, we will more clearly distinguish the scalability of the TTT-SCL framework from the applicability of the current GAM-SIM implementation. A systematic comparison of different mechanism fitters, noise models, and their degree of mismatch will be the focus of future work; the results presented in this paper should be understood as an empirical evaluation of the current specific implementation.

W3.Fairness of metric and limitations of real-world data

(1) AUROC evaluates how well edge scores rank true edges above nonedges without selecting a single threshold, making it suitable for assessing our learner’s continuous outputs. We have provided additional standard causal discovery metrics, including ACC and F1, in Appendix F (Table 7). ACC can be viewed as a normalized version of the Structural Hamming Distance (SHD). Relative to AVICI, TTT-SCL improves F1/AUPRC (%) from 41.8/52.8 to 65.4/65.0 on Linear_U, from 23.0/31.6 to 56.4/53.6 on Sachs, and from 22.22/25.53 to 32.14/51.85 on SynTReN. These key improvements extend beyond AUROC, although rankings are not identical across every metric and setting.

(2) To reduce reliance on Sachs's data for evaluation, we supplemented our analysis with Causal Chambers' Light Tunnel experiment. This data, derived from a real physical device, provides a reference causal model that combines physical knowledge with experimental validation, offering a different test subject than biological signaling networks. All methods used the same 20 observed variables and did not provide intervention meta-information. We used five predefined, non-overlapping data blocks from the uniform reference, each containing 1,000 rows. These measure subsampling differences within the same physical system, rather than five independent physical systems. The results are as follows:

Table R1. Causal Chambers evaluation on a real physical system.

MethodAUROC (%)AUPRC (%)SHD
PC56.614.742.6
GES54.311.643.8
AVICI44.69.142.6
GraN-DAG50.411.238.6
DAG-GNN48.710.351.6
Seed graph (NOTEARS)49.710.345.4
Full TTT-SCL60.716.339.0

Values are averages over the five predefined blocks. For PC and GES, we enumerate all consistent DAG extensions of each estimated CPDAG, compute AUROC/AUPRC from binary scores and SHD for every DAG, average within the MEC, and then average across blocks. Seed and thresholded DAG-GNN use AUROC/AUPRC from binary scores, whereas Full TTT-SCL, AVICI, and GraN-DAG use continuous scores. Full TTT-SCL has the highest AUROC/AUPRC, while GraN-DAG has slightly lower SHD.

Thank you for the clarification and additional experiments. I will maintain the rating.

Reviewer_9CrVOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
1
Presentation
3
Soundness
1
Read review
Paper summary

The submission proposes the following method for causal discovery: Given a dataset initialize a graph (random or existing method, mostly NOTEARS), generate a list of graphs via sequential edits that are accepted/rejected according to a score construction (computed by fitting mechanisms to the data according to the graph structure and checking likelihoods), for each of those candidate graphs and the corresponding functions, sample new datasets. On these datasets train an AVICI style amortized supervised causal learning model and evaluate it on the original dataset to obtain the final graph.

Critical strengths
  • The paper is largely easy to follow and well written. The authors do a good job describing clearly how their method functions in most parts.
  • Bringing test time training to different domains is an area of active interest.
Critical weaknesses
  1. One of my main criticisms is that the claims made throughout the paper are too broad and not supported by evidence. I will give a few examples.

a. Primarily, the entire section 3 on issues of SCL is overstated. It is repeatedly emphasized heavily that there is a "gap" or fundamental issues with SCL style learning and that these are undisputably demonstrated here. However, this somehow just attacks the assumptions of SCL style learning. Viewed as an amortized prior fitted network approach, it is known and expected that inputs on which the prior puts little or no mass will not be handled well. The fundamental assumption of PFN style learning is that you craft your prior such that it captures all settings you expect to encounter at test time. Intentionally setting things up such that this is violated (in different ways) does not provide novel insights to me. (It's a bit like showing how linear methods fail on non-linear data.) This also holds for the composition claim: The way it is written suggests that "one should expect that the method figures out compositions, but it (surprisingly?) does not". However, this would only be possible if from the observational datasets (only input to the SCL model) one could identify the compositionality of the underlying data generating process (SCM), which is known to be impossible. It would have been interesting if there was a structured general quantification of "how" exactly methods fail, but for this the scope of the experimental setup is too narrow.

To summarize: Yes, supervised learning typically fails out of distribution (often in hard to characterize ways). The key observation here is that this would also happen to the proposed method: If certain building blocks (functional mechanisms, noise distributions/models, types of graphs) are not part of the available library (e.g., in the SIM steps or final training), it would equally suffer. The assumption must be that all relevant mechanisms, distributions, etc are known in advance, which also provides a way forward for non-ttt SCL methods by including all components and compositions thereof in a diverse prior.

b. Some of the claims are worded too definitive given the evidence: for example, the "memorization" claim in Issue 3 is not at all clear to me. Similarly, following the argument above, I disagree that the "results compellingly argue that robust causal discovery requires a shift". It also does not "fundamentally undermine the static pre-training paradigm". More strong phrasing in the experimental results discussion.

  1. Insufficient scope and empirical validation: Given that the methodological contribution of the submission must be carried by its empirical performance, the scope is too narrow in my view. Concretely:
  • Two graph families, 3 mechanism types and only additive noise models (primarily Gaussian, see below for comments on identifiability) does not cover any reasonably realistic settings. Current state of the art (including AVICI) goes far beyond this.

  • A big burden of evidence rests on Sachs as the only real-world dataset. As Sach herself (as co-author) claimed [1], this dataset sholud not be treated as definitive ground truth. It likely violates various assumptions of causal discovery algorithms (starting with acyclicity) and it has been criticized as a "ground truth" dataset repeatedly in its many decades of use as essentially the one and only real world dataset we have for causal discovery. So agreement with one disputed reference graph on a single dataset is not sufficient evidence of claiming superior real-world performance. Even then, the it is unclear what AUROCs of 60 vs 80 mean on a graph with 11 nodes. This points at another fundamental challenge for causal discovery: statistical reports of metrics like F1, AUROC, AUPRC, etc hide the difficulty of interpretation: Is an F1 of 0.8 on a graph with 20 nodes good or bad? Can we tolerate 1 wrong connection among 400 possible ones or 20? How useful is a graph with 5 false edges out of 15 true edges compared to one with 10 wrong edges out of the 15?

[1] Brouillard, P., Squires, C., Wahl, J., Kording, K. P., Sachs, K., Drouin, A., & Sridhar, D. (2024). The landscape of causal discovery data: Grounding causal discovery in real-world applications. arXiv preprint arXiv:2412.01953.

  1. Insufficient discussion of causal discovery (and its limitations/challenges) more broadly: Connecting also to the above points, causal discovery (as intruiging as it is) is very hard and as a community we understand very well that in general, there is not much we can hope for. Even general SCMs are incredibly narrow models (in particular the acyclicty assumption), which the literature further constrains to few graph-families, restrictive classes of functional relationships, and restrictive noise models (additive) and distributions. Increasing flexibility leads to difficult to counter identifiability issues, but maintaining identifiability requires extremely restricted models. (In this submission the authors seem to have gone the route of preserving identifiability, but do not discuss why it makes sense to claim applicability while simultaneously restricting linear relations to additive uniform noise and other relations to additive Gaussian noise.) Paired with the fact that we essentially still have no good benchmarks (see critique about Sachs dataset above) means that all synthetic empirical evaluation can be tuned to reveal whatever the authors please.

I acknowledge that large parts of my review are a general critique of causal discovery, but given that these have been stated repeatedly openly, I insist that we should take them seriously as a community and must be more thorough in our work on causal discovery. For example, the fact that the current submission uses only additive Gaussian noise with unit variance despite [2] being 5 years old and having been discussed extensively in the literature calls for more caution in my view (see also [3]).

[2] Reisach, A., Seiler, C., & Weichwald, S. (2021). Beware of the simulated dag! causal discovery benchmarks may be easy to game. Advances in Neural Information Processing Systems, 34, 27772-27784. [3] Reisach, A., Tami, M., Seiler, C., Chambaz, A., & Weichwald, S. (2023). A scale-invariant sorting criterion to find a causal order in additive noise models. Advances in Neural Information Processing Systems, 36, 785-807.

Questions for authors
  • It would be nice to provide some intuition or justificaiton for the many heuristics (e.g., keeping last K out of N candidate graphs, the entire data generation pieces, the score derived probability \alpha).
5 comments

We are very grateful for the reviewers' sincere and constructive comments, which are of great help to us in promoting the development of TTT-SCL more rigorously. Below are our responses to your concerns and questions:

W1. The criticism of the current limitations of SCL is too strong and the necessity of TTT-SCL is not fully demonstrated.

(1) Although the failure of OOD generalization in supervised learning is a trivial intuition, but not every researcher in our field accepts and realizes what this means for causal discovery. First, one of the earliest studies of the SCL paradigm, CSIvA[1], attempted to test the model outside the training data distribution to prove that it still has high performance. This to some extent leads people to believe that a basic model similar to GPT has been established in the field of causal discovery. At the same time, AVICI[2] and CDFM[3] training on a wide distribution further improves performance, leading people to believe that the field of causal discovery, like other fields, has established a scaling law, where as long as enough distributions are included, it will perform well in most scenarios. In fact, a very important difference between the field of causal discovery and other fields is that "causal graphs" are very scarce. Therefore, the lack of ground truth for pre-training leads to a heavy reliance on synthetic data for current training data. In real-world scenarios, performance heavily depends on the similarity between the training data and the real data distribution. One of the key points of this paper is to advise those in this field against over-reliance on (static) SCL.

(2) However, our initial intention was not to close off SCL. We believe the key to further developing SCL is defining a coordinate system or space to systematically describe graph distribution, mechanism distribution, and noise distribution, moving beyond heuristic selection of combinations. This provides a tool for quantitatively or incrementally improving the coverage of the training distribution.

(3) Furthermore, we aim to provide an alternative to SCL, eliminating the need to pre-assume that the training data distribution covers all possible subsequent test distributions. Instead, we generate training data specifically for each new test data and train the SCL model. This addresses, to some extent, the theoretical problem that synthetic data cannot cover all function spaces, focusing on the current test data each time – this is the idea behind TTT-SCL. Of course, the specific performance of this paradigm is highly dependent on the fitting of the test data distribution. However, based on the currently widely available fitting tools (such as neural networks), it is possible to capture most of the information of the test distribution to a certain extent. Using this information to generate training data is another approach to addressing the OOD problem faced by SCL.

(4) Our current GAM-based SIM is only one implementation of the TTT-SCL paradigm, intended to initially demonstrate its applicability. We acknowledge that the current SIM has a limited coverage, but it still outperforms existing traditional causal discovery methods and SCL on a limited test distribution, demonstrating the potential of TTT-SCL. Designing a more general mechanism fitter will be the focus of future research on TTT-SCL.

[1] Ke, N. R., et al. (2023). Learning to Induce Causal Structure. International Conference on Learning Representations (ICLR).

[2] Lorch, L., et al. (2022). Amortized Inference for Causal Structure Learning. Advances in Neural Information Processing Systems, 35, 13104–13118.

[3] Qiao, J., et al. (2026). CDFM: Towards a General-Purpose Causal Discovery Foundation Model. arXiv preprint arXiv:2607.11508.

W2.Insufficient scope and empirical validation.

(1) Our experiments primarily aim to demonstrate whether the performance of static SCL models is affected by variations in the training distribution, and whether constructing training data using current test data can improve SCL performance. Synthetic data allows for direct evaluation of causal graphs, and controlled variations in graphs, mechanisms, and noise make performance differences more interpretable. These experiments support the core arguments under the evaluated conditions, and their effectiveness does not depend on covering all real-world systems. Of course, for synthetic data, the wider the distribution range, the better.

(2) We are aware that Sachs is controversial, primarily due to its potential violation of the assumptions of causal discovery and the debate surrounding actual causal graphs. However, given the severe lack of real-world data in the field of causal discovery, we still hope to utilize it to demonstrate the capabilities of TTT-SCL to some extent. We acknowledge that performance on Sachs cannot be considered definitive evidence of the ability to recover true causal relationships.

(3) To reduce reliance on Sachs's data for evaluation, we supplemented our analysis with Causal Chambers' Light Tunnel experiment. This data, derived from a real physical device, provides a reference causal model that combines physical knowledge with experimental validation, offering a different test subject than biological signaling networks. All methods used the same 20 observed variables and did not provide intervention meta-information. We used five predefined, non-overlapping data blocks from the uniform reference, each containing 1,000 rows. These measure subsampling differences within the same physical system, rather than five independent physical systems. The results are as follows:

Table R1. Causal Chambers evaluation on a real physical system.

MethodAUROC (%)AUPRC (%)SHD
PC56.614.742.6
GES54.311.643.8
AVICI44.69.142.6
GraN-DAG50.411.238.6
DAG-GNN48.710.351.6
Seed graph (NOTEARS)49.710.345.4
Full TTT-SCL60.716.339.0

Values are averages over the five predefined blocks. For PC and GES, we enumerate all consistent DAG extensions of each estimated CPDAG, compute AUROC/AUPRC from binary scores and SHD for every DAG, average within the MEC, and then average across blocks. Seed and thresholded DAG-GNN use AUROC/AUPRC from binary scores, whereas Full TTT-SCL, AVICI, and GraN-DAG use continuous scores. Full TTT-SCL has the highest AUROC/AUPRC, while GraN-DAG has slightly lower SHD.

(4) We agree with your comments regarding the interpretation of the metrics. AUROC evaluates how well edge scores rank true edges above nonedges without selecting a single threshold, making it suitable for assessing our learner’s continuous outputs. We provide additional metrics (F1 and ACC) for the consistent experimental setup in the text in Appendix F and Table 7, and SHD in the Causal Chambers experiment (Table R1). These metrics help to intuitively understand the number of correct edges and also demonstrate the consistency of our method across different metrics.

W3. Insufficient discussion of causal discovery more broadly.

We chose linear non-Gaussian and non-linear Gaussian additive noise settings, primarily to study DAG recovery within a class of identifiable models, thus providing a clear theoretical basis for the main supervised learning objective. For the linear setting, the identifiability of the complete DAG is theoretically supported by LiNGAM under conditions of no loops, no unobserved confounding, independent non-Gaussian noise, and non-degenerate relations. Uniform noise is one of the non-Gaussian noise settings we employed.

Q1. Some heuristic designs lack intuition or justificaiton.

(1) Keeping the last K out of N candidate graphs: The search steps N control the exploration budget, and K controls the number of graph states ultimately used to construct the training task. Using the last K graphs aims to concentrate the training set on candidates after multiple rounds of score-guided search, and to limit the cost of subsequent generation and training.

(2) The entire data generation pieces(SIM): For each candidate graph, we fit a node-by-node mechanism from the current observation data according to its parent node set, and then generate a new dataset along the topological order of the graph. The candidate graph provides graph labels for the constructed data generation, while the mechanism fitting enables the simulated data to reflect some of the distribution information in the current observation data. This allows us to eliminate some of the OOD effects when training SCL, enabling SCL to focus on learning the mapping rules from the data to the causal graph. Appendix C already formalizes mechanism fitting and topological sampling; Appendix E, Figure 3 illustrates the workflow, and Appendix A.3 specifies the sampling and training settings.

(3) Score derived probability: our acceptance probability is

=\min\{1,\exp[\operatorname{Score}(G_{\mathrm{cand}},D_{\mathrm{test}})-\operatorname{Score}(G_{\mathrm{curr}},D_{\mathrm{test}})]\}$$ The acceptance probability is designed for several reasons: candidate graphs with improved scores are always accepted, while candidate graphs with decreased scores still have a certain probability of being accepted, with the probability decreasing as the score decreases. This provides a bias towards higher-scoring candidates while accepting lower-scoring ones with a certain probability, thus allowing for the exploration of both local and global optima. Using score differences also allows the rule to handle negative scores while remaining invariant to additive shifts in the scores.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.