acceptodds
Withdrawn from ICLR 2027

Likelihood-Based Nonparametric Causal Discovery under Latent Confounding

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Learning causal structure from observational data in the presence of latent confounding remains a fundamental challenge in causal discovery. Acyclic directed mixed graphs (ADMGs) provide a natural representation of this problem by indicating causal effects and latent confounding via directed and bidirected edges. A line of recent work has focused on score-based approaches that formulate ADMG learning as continuous optimization problems with acyclicity and further structural constraints enforced through differentiable functions. However, the methods that were developed either focus on linear structural equations or lack computational and statistical tractability due to structural equations involving explicitly introduced latent confounders with general non-linear effects. In this work, we seek to trade-off the advantages of these two perspectives and propose causal models that relate the observed variables through non-linear structural equations but summarize confounding in mere noise correlations. For the proposed model class, we establish structural identifiability of bow-free graphs under a residual faithfulness assumption. Moreover, given a known ancestral graph, we clarify that the model parameters are identifiable. With this insight, we develop a novel likelihood-based ADMG learning framework. Importantly, our approach does not introduce explicit latent variables and, thus, necessitates disentangling correlations among noise terms and linear parts of causal effects, which we address using a non-linearity regularizer. Experiments show that our method outperforms existing state-of-the-art baselines on synthetic datasets while remaining competitive on real-world datasets.

Primary area:
causal reasoning
Keywords:
Causal discovery, Latent confounding, Acyclic directed mixed graphs, Structural identifiability, nonlinear structural equation models

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_hYR6OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper proposes GLL-ADMG, a likelihood-based nonparametric causal discovery method that models direct causal relationships among observed variables using nonlinear structural equations and implicitly represents latent confounding through correlated Gaussian noise, thereby jointly learning directed and bidirected edges in an acyclic directed mixed graph (ADMG). Specifically, it combines nonlinear regularization with differentiable constraints enforcing acyclicity, ancestrality, and bow-freeness, yielding a continuous optimization framework for causal structure learning. It introduces the residual faithfulness assumption and establishes structural identifiability for bow-free graphs and parameter identifiability for ancestral graphs.

Critical strengths
  1. The paper proposes a structure learning algorithm by systematically investigating structural and parameter identifiability, and providing theoretical justification for the generic validity of the residual faithfulness assumption.
  2. The proposed method accommodates nonlinear causal mechanisms and models latent confounding through correlated noise, without explicitly introducing latent variables. It can also recover both directed and bidirected edges.
Critical weaknesses
  1. The paper establishes causal structure identifiability under the residual faithfulness assumption and discusses the theoretical justification for this assumption holding generically. However, generic validity does not necessarily imply that the assumption is readily satisfied in real-world data, nor does it guarantee reliable causal structure recovery in finite samples. In particular, when the data are close to exceptional cases that violate this assumption, even small estimation errors may potentially lead to substantial changes in the recovered structure. It would be better to further discuss the practical applicability of this assumption and provide finite-sample stability analyses or additional experiments to clarify the relationship between theoretical identifiability and reliable structure recovery in practice.

  2. Although the paper adopts a joint optimization framework to learn the causal structure and model parameters, the residuals still depend on neural networks fitted to the causal mechanisms and therefore do not coincide with the true structural errors in finite samples. In particular, overfitting or underfitting by the nonlinear MLPs may distort the dependence structure of the estimated residuals relative to that of the true errors. For example, overfitting may absorb variations that should remain in the residuals, weakening the residual correlations induced by genuine latent confounding and consequently leading to missed bidirected edges. Conversely, underfitting may introduce additional residual correlations, resulting in spurious bidirected edges.

Questions for authors
  1. The paper introduces a nonlinearity regularization term into the objective function, penalizing the second derivatives of the causal mechanism functions to encourage the learning of nonlinear causal relationships. However, although the structural equation model allows for general, potentially nonlinear mechanisms, this does not imply that the true causal mechanisms must be nonlinear. Why is it necessary to explicitly encourage nonlinearity in the objective function? When the true causal relationships are linear or approximately linear over the main range of the observed data, could this regularization term introduce unnecessary model bias and consequently impair causal structure recovery?

  2. The paper addresses causal structure discovery in the presence of both nonlinear mechanisms and latent confounders. However, the related work appears to provide insufficient coverage of existing methods for nonlinear causal discovery with latent confounding, such as "Score Matching Through the Roof: Linear, Nonlinear, and Latent Variables Causal Discovery". It would be better to discuss relevant approaches more comprehensively and clarify how the proposed method differs from existing work in terms of problem formulation, identifiability assumptions, theoretical guarantees, and empirical performance. Such a comparison would help better establish the novelty and scope of applicability of the proposed method.

  3. The proposed method incorporates a nonlinearity regularization term into the objective function, using the Hessian of the causal mechanism functions to construct the penalty. This design may introduce additional computational and memory overhead during optimization, particularly when the number of variables is large or complex neural networks are used to model the causal mechanisms.

  4. The synthetic experiments primarily consider graphs with d≤12d\leq 12, while the real-world evaluation is also relatively limited in scale and scope. Consequently, the current experiments may not be sufficient to establish the effectiveness of the proposed method for high-dimensional data and complex causal structures. In particular, on the Sachs dataset, the score of the proposed method is lower than the scores obtained by both FCI and DCD-A. This result suggests that, although the proposed method may offer advantages in learning structures involving latent confounding, its ability to recover directed edges requires further validation. Further analysis of the relatively poor directed-edge recovery performance would also be valuable.

  5. The paper represents bidirected edges induced by latent confounding through nonzero off-diagonal entries of the error covariance matrix Σ\Sigma, and recovers the corresponding graph structure by thresholding the estimated entries. However, this procedure may be sensitive to finite-sample covariance estimation errors and the choice of threshold. When the true error correlation is weak, the estimated covariance may fall below the threshold even when latent confounding is present, resulting in missed bidirected edges. Conversely, estimation errors may produce spurious nonzero correlations and lead to false positives. Furthermore, when multiple error terms are strongly correlated, covariance estimation errors may affect the identification of multiple bidirected edges simultaneously.

1 comment

Q1&Q3: Nonlinearity regularization term

Thank you for the comments. Our analysis considers a nonlinear structural equation model. Linear models with Gaussian noise are generally not identifiable. As in other causal discovery approaches, the theoretical results are formulated under model assumptions, whose applicability depends on the setting. The computational cost of the nonlinearity regularization is discussed in the manuscript.

Q2: Prior work

Yes, we'd be happy to expand the discussion of related literature.

Q4: Computation cost

We note that the space of ADMGs is substantially larger than that of DAGs, and there is a very large number of models to select from already for the graph sizes we consider. The setting is in line with those in prior work.

Q5: Estimation error and thresholding

This point seems more a critique of all of statistical model selection than a specific comment related to our paper.

Reviewer_cLZoOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This manuscript studies nonparametric causal discovery under latent confounding. The authors model direct causal relationships using nonlinear functions and latent confounding using correlated Gaussian noise. They establish structural identifiability for bow-free graphs under a residual faithfulness assumption and parameter identifiability for ancestral graphs. Based on these results, they propose GLL-ADMG, a likelihood-based method for learning ADMGs. Experiments show effectiveness on synthetic datasets.

Critical strengths
  • The proposed model family captures nonlinear causal relationships and latent confounding without explicitly introducing latent variables.

  • The manuscript provides theoretical identifiability results and evaluates the proposed method under different graph structures and data-generating mechanisms.

Critical weaknesses
  • The main identifiability result relies heavily on the bow-free assumption, while the manuscript does not discuss its practical relevance. It can be invalid in real-world causal systems. Moreover, it remains unclear whether bow-freeness is necessary for structural identifiability under the proposed model or merely required by the current proof. The real-world dataset used in the experiments does not satisfy this assumption.

  • The residual faithfulness assumption appears somewhat restrictive. If I understand correctly, Definition 3.1 requires the condition to hold for all invertible Gaussian representations of the observed distribution. Yet, consider the following case: two independent Gaussian variables can be represented as an orthogonal transformation of independent Gaussian noises. In this case, both variables depend on the same noise while remaining independent, which seems to violate the assumption. The authors would be better to clarify whether this assumption can hold even for such simple cases.

Questions for authors

Is the bow-free assumption necessary for structural identifiability, or is it only required by the current proof? Can the authors provide a counterexample showing that allowing bows leads to non-identifiability?

1 comment

Q1: Bow-free assumption

Thank you for the questions. Bow-freeness is necessary for structure identifiability. Indeed, in a bivariate model containing a bow, the linear dependence corresponding to the bidirected edge can always be absorbed into the nonlinear dependence corresponding to the directed edge. Therefore, we cannot differentiate a graph with a bow from the same graph where we removed the bidirected edge from the bow.

W2: Counterexample

The proposed counterexample transforms two independent Gaussian variables into an orthogonal transformation of independent Gaussian noise error terms. This is similar to the counterexample of reviewer 11. In this case, each structural equation for both X1X_1 and X2X_2 contain multiple error terms Z1Z_1 and Z2Z_2. This differs from the structural equation formulation considered in our work. In the proof, the function covariance pairs in H(PX)\mathcal{H}(\mathbb{P}^{\mathbf{X}}) are considered within this structural model class. We will make this restriction explicit in the definition.

Reviewer_PMdbOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes GLL-ADMG for learning acyclic directed mixed graphs with latent confounding. The theory contributions include structural identifiability of bow-free ADMGs under a residual faithfulness condition and identifiability given a known ancestral graph. The estimation method uses neural networks for the structural functions, a Gaussian residual likelihood, sparsity penalties, a Hessian-based nonlinearity regularizer, and differentiable acyclicity together with ancestral, bow-free constraints.

Critical strengths

The research problem is interesting and important.

Critical weaknesses
  1. Schultheiss & Bühlmann (2023, "On the pitfalls of Gaussian likelihood scoring for causal discovery ") report a negative result for Gaussian likelihood scoring combined with nonparametric regression under non-Gaussian errors. To my understanding, the proposed approach suffers from the issue mentioned there.
  2. The experiments are very limited. All synthetic data have at most d=12 nodes, and use sparse graphs and n=2000. A major concern is the scalability of the method.
Questions for authors
  1. Consider X1=e1X_1 = e_1 and X2=X13+e2X_2 = X_1^3 + e_2 with e1,e2e_1, e_2 being iid standard normal. Now let Z1=(e1+e2)/2Z_1 = (e_1 + e_2)/\sqrt{2} and Z2=(e1−e2)/2Z_2 = (e_1-e_2)/\sqrt{2}. Then X1=(Z1+Z2)/2X_1 = (Z_1 + Z_2)/\sqrt{2} and X2=((Z1+Z2)/2)3+(Z1−Z2)/2X_2 = ((Z_1 + Z_2)/\sqrt{2})^3 + (Z_1 - Z_2)/\sqrt{2}. By Definition 3.1, M~(G)=∅\widetilde{\mathcal M}(G) = \emptyset. As currently defined, the true mechanism never satisfy residual faithfulness.
  2. Can you discuss how the results in Schultheiss & Bühlmann (2023) impact the proposed method?
  3. How does the method scale computationally? Why is evaluation limited to small graphs?
1 comment

Q1: Counterexample

Thank you for the counterexample. The proposed counterexample has the form X1=(Z1+Z2)/2X_1 = (Z_1+Z_2)/\sqrt{2}, X2=X13+(Z1−Z2)/2X_2 = X_1^3 + (Z_1-Z_2)/\sqrt{2}. In each structural equation for both X1X_1 and X2X_2, there are multiple error terms Z1Z_1 and Z2Z_2 . This differs from the structural equation formulation considered in our work. In the proof, the function covariance pairs in H(PX)\mathcal{H}(\mathbb{P}^{\mathbf{X}}) are considered within this structural model class. We will make this restriction explicit in the definition.

Q2: Prior work

We will discuss the relationship to Schultheiss & Bühlmann (2023) in the revised manuscript.

Q3: scalability

We note that the space of ADMGs is substantially larger than that of DAGs, and there is a very large number of models to select from already for the graph sizes we consider. The setting is in line with those in prior work.

Comments on the paper

1 comment

We thank the reviewers for their time and constructive feedback. We provide brief clarifications on the model formulation, assumptions, and experiments below. That said, we have decided to withdraw the submission and will incorporate the feedback into a revised manuscript.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.