- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
The paper proposes GLL-ADMG, a likelihood-based nonparametric causal discovery method that models direct causal relationships among observed variables using nonlinear structural equations and implicitly represents latent confounding through correlated Gaussian noise, thereby jointly learning directed and bidirected edges in an acyclic directed mixed graph (ADMG). Specifically, it combines nonlinear regularization with differentiable constraints enforcing acyclicity, ancestrality, and bow-freeness, yielding a continuous optimization framework for causal structure learning. It introduces the residual faithfulness assumption and establishes structural identifiability for bow-free graphs and parameter identifiability for ancestral graphs.
Critical strengths
- The paper proposes a structure learning algorithm by systematically investigating structural and parameter identifiability, and providing theoretical justification for the generic validity of the residual faithfulness assumption.
- The proposed method accommodates nonlinear causal mechanisms and models latent confounding through correlated noise, without explicitly introducing latent variables. It can also recover both directed and bidirected edges.
Critical weaknesses
-
The paper establishes causal structure identifiability under the residual faithfulness assumption and discusses the theoretical justification for this assumption holding generically. However, generic validity does not necessarily imply that the assumption is readily satisfied in real-world data, nor does it guarantee reliable causal structure recovery in finite samples. In particular, when the data are close to exceptional cases that violate this assumption, even small estimation errors may potentially lead to substantial changes in the recovered structure. It would be better to further discuss the practical applicability of this assumption and provide finite-sample stability analyses or additional experiments to clarify the relationship between theoretical identifiability and reliable structure recovery in practice.
-
Although the paper adopts a joint optimization framework to learn the causal structure and model parameters, the residuals still depend on neural networks fitted to the causal mechanisms and therefore do not coincide with the true structural errors in finite samples. In particular, overfitting or underfitting by the nonlinear MLPs may distort the dependence structure of the estimated residuals relative to that of the true errors. For example, overfitting may absorb variations that should remain in the residuals, weakening the residual correlations induced by genuine latent confounding and consequently leading to missed bidirected edges. Conversely, underfitting may introduce additional residual correlations, resulting in spurious bidirected edges.
Questions for authors
-
The paper introduces a nonlinearity regularization term into the objective function, penalizing the second derivatives of the causal mechanism functions to encourage the learning of nonlinear causal relationships. However, although the structural equation model allows for general, potentially nonlinear mechanisms, this does not imply that the true causal mechanisms must be nonlinear. Why is it necessary to explicitly encourage nonlinearity in the objective function? When the true causal relationships are linear or approximately linear over the main range of the observed data, could this regularization term introduce unnecessary model bias and consequently impair causal structure recovery?
-
The paper addresses causal structure discovery in the presence of both nonlinear mechanisms and latent confounders. However, the related work appears to provide insufficient coverage of existing methods for nonlinear causal discovery with latent confounding, such as "Score Matching Through the Roof: Linear, Nonlinear, and Latent Variables Causal Discovery". It would be better to discuss relevant approaches more comprehensively and clarify how the proposed method differs from existing work in terms of problem formulation, identifiability assumptions, theoretical guarantees, and empirical performance. Such a comparison would help better establish the novelty and scope of applicability of the proposed method.
-
The proposed method incorporates a nonlinearity regularization term into the objective function, using the Hessian of the causal mechanism functions to construct the penalty. This design may introduce additional computational and memory overhead during optimization, particularly when the number of variables is large or complex neural networks are used to model the causal mechanisms.
-
The synthetic experiments primarily consider graphs with , while the real-world evaluation is also relatively limited in scale and scope. Consequently, the current experiments may not be sufficient to establish the effectiveness of the proposed method for high-dimensional data and complex causal structures. In particular, on the Sachs dataset, the score of the proposed method is lower than the scores obtained by both FCI and DCD-A. This result suggests that, although the proposed method may offer advantages in learning structures involving latent confounding, its ability to recover directed edges requires further validation. Further analysis of the relatively poor directed-edge recovery performance would also be valuable.
-
The paper represents bidirected edges induced by latent confounding through nonzero off-diagonal entries of the error covariance matrix , and recovers the corresponding graph structure by thresholding the estimated entries. However, this procedure may be sensitive to finite-sample covariance estimation errors and the choice of threshold. When the true error correlation is weak, the estimated covariance may fall below the threshold even when latent confounding is present, resulting in missed bidirected edges. Conversely, estimation errors may produce spurious nonzero correlations and lead to false positives. Furthermore, when multiple error terms are strongly correlated, covariance estimation errors may affect the identification of multiple bidirected edges simultaneously.
Q1&Q3: Nonlinearity regularization term
Thank you for the comments. Our analysis considers a nonlinear structural equation model. Linear models with Gaussian noise are generally not identifiable. As in other causal discovery approaches, the theoretical results are formulated under model assumptions, whose applicability depends on the setting. The computational cost of the nonlinearity regularization is discussed in the manuscript.
Q2: Prior work
Yes, we'd be happy to expand the discussion of related literature.
Q4: Computation cost
We note that the space of ADMGs is substantially larger than that of DAGs, and there is a very large number of models to select from already for the graph sizes we consider. The setting is in line with those in prior work.
Q5: Estimation error and thresholding
This point seems more a critique of all of statistical model selection than a specific comment related to our paper.