acceptodds
Under review as a conference paper at ICLR 2027

Differentiable Structure Learning for Cyclic Linear Gaussian Models with Latent Confounders

Abstract

We study causal structure learning from observational data in linear Gaussian structural causal models in the presence of directed cycles and an unknown number of exogenous latent confounders, bounded by a given maximum. We derive the covariance of the observed variables and introduce marginal quasi-equivalence, which characterizes when different causal models share a full-dimensional subset of the observational distributions they can generate. We formulate structure learning as minimization of the Gaussian negative log-likelihood with a logarithmically scaled complexity penalty that counts directed edges and latent variables. For a fixed number of observed variables and a fixed upper bound on latent variables, we establish consistency of global score minimizers up to marginal quasi-equivalence under algebraic faithfulness, structural minimality, and model-overlap assumptions. We parameterize the inclusion of directed edges and candidate latent variables using Bernoulli gates, whose continuous probabilities are optimized jointly with the structural coefficients. Averaging the penalized negative log-likelihood over these gates yields an objective with a closed-form differentiable complexity penalty. We prove that this expected objective has the same global infimum as the corresponding discrete structure-learning objective. Experimental results show that our approach achieves lower recovery error than previous methods in several experimental settings.

Primary area:
causal reasoning
Keywords:
Causal discovery, cyclic causal models, latent confounding, differentiable structure learning, linear Gaussian models, distribution equivalence

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_GraQOpenReview ↗
Rating
3
Confidence
2
Informativeness
3
Presentation
2
Soundness
3
Read review
Paper summary

This paper investigates causal structure learning from observational data in the presence of both directed cycles and latent confounders. Previous approaches typically either allow directed cycles while assuming the absence of latent confounders, account for latent confounders while assuming that the causal graph is a DAG, or address both but require interventional data or additional restrictions on graph structures.

The paper first introduces Marginal Quasi-Equivalence as a notion of equivalence between causal graphs based on the covariance matrices of observed variables. It then establishes that, under several assumptions, particularly structural minimality, the estimated graph is guaranteed to be marginally quasi-equivalent to the true graph. The authors further propose an algorithm for causal structure learning by parameterizing discrete graph structures using Bernoulli probabilities. Experiments on synthetic and biological data demonstrate that the proposed method achieves superior performance in terms of bidirectional covariance-family compatibility error.

Critical strengths

The main strength of this paper is that it clearly identifies the conditions under which recovery up to marginal quasi-equivalence, defined in terms of observational covariance matrices, can be theoretically guaranteed in the presence of both directed cycles and latent confounders, a setting in which theoretical guarantees have traditionally been difficult to establish. Although the proposed framework does not guarantee unique identification of the true causal graph, it rigorously establishes conditions under which the estimated graph is marginally quasi-equivalent to the true graph.

Since causal structures involving both directed cycles and latent confounders are highly relevant to real-world applications, demonstrating the theoretical feasibility of structure learning under such general conditions represents a significant contribution to the research field.

Critical weaknesses

The most critical weakness of this paper is that the gap between its theoretical guarantees and practical applicability is not sufficiently addressed. Although the theoretical consistency guarantee is limited to marginal quasi-equivalence, the paper proposes an algorithm that outputs a specific causal graph. The algorithm selects a single graph by imposing a complexity penalty, but there is no theoretical guarantee that the selected graph corresponds to the true causal graph. Despite this limitation, the paper provides little discussion of how and to what extent the estimated causal graph may differ from the true causal graph, particularly in terms of specific structural differences. Consequently, users have no clear basis for determining how much confidence they should place in the estimated causal structure. While I understand that the primary contribution of this paper lies in its theoretical guarantees, I believe that addressing this issue would substantially improve the practical applicability of the proposed method.

A related concern is that the experimental evaluation focuses exclusively on bidirectional covariance-family compatibility error. I understand that metrics such as Structural Hamming Distance (SHD) do not fully align with the objective of the proposed method, since structurally different causal graphs may be observationally equivalent. Nevertheless, given that the proposed algorithm outputs a specific causal graph, it is important to quantitatively evaluate how closely the estimated graph matches the true causal graph. Even if the results are unfavorable, such an evaluation would provide valuable insights into the characteristics and limitations of the proposed method and could therefore make a positive contribution to the paper.

Questions for authors

Could the authors discuss how users should interpret the causal graph produced by the proposed method in relation to the true causal graph? Alternatively, could the authors provide a more comprehensive evaluation, including metrics such as Structural Hamming Distance (SHD), to quantitatively and qualitatively clarify how the estimated causal graph differs from the true graph?

3 comments

We thank the reviewer for their valuable comments and address them below:

Regarding the interpretation of marginal quasi-equivalence and the estimated graph

Our guarantee concerns recovery up to marginal quasi-equivalence, not exact recovery of the true underlying causal graph, which is not identifiable from observational data in linear Gaussian models (with or without cycles). Under the stated assumptions (Assumptions 7–9), Theorem 4 and Corollary 6 (Section 4.1; proofs in Appendices A.9–A.10) guarantee that every global score minimizer is marginally quasi-equivalent to the true graph with probability tending to one as the sample size goes to infinity. Such a graph may differ from the true graph in individual edges and therefore have a nonzero structural Hamming distance (SHD). SHD compares one particular graph representation and can penalize differences permitted by our recovery criterion. A nonzero SHD therefore does not, by itself, contradict our guarantee.


Relationship between the estimated and true graphs

To clarify the structural interpretation, in the rebuttal phase, we additionally establish that marginal quasi-equivalence preserves the partition of observed variables into weakly connected components, including connections through latent variables. Two observed variables belong to the same component if a path connects them when arrow directions are ignored; the path may pass through latent variables. Therefore, under our consistency assumptions, every global minimizer of our proposed score assigns the observed variables to the same weakly connected components as the true graph, with probability tending to one as the sample size increases. We have added the definitions, and proof to Appendix F of the paper in the rebuttal phase.

To show this empirically, we evaluate component recovery using the adjusted Rand index (ARI) between the observed-variable component partitions of the true graph and the graph recovered by each method, where 1 means identical partitions. With 20% latent variables and maximum observed degree four, our method achieves ARI 1 on all completed trials. The following table reports mean ARI at the three largest evaluated graph sizes:

Observed variablesOursDGlearnAmendolaDCCD-CONF
321.001.000.900.90
641.000.900.800.80
1281.000.24——

Amendola and DCCD-CONF were evaluated through 64 variables only. At 128 variables, three additional trials reached the six-hour limit during DGlearn’s search and are excluded from the matched comparison. The revised manuscript, uploaded to OpenReview, includes the experimental details and comparison plots in Appendix F.

We hope the above explanation addresses your concern, and we sincerely appreciate your time and consideration.

Thank you for your careful and detailed response. Your explanation has helped me better understand where the main contribution of this paper lies. While I understand that a nonzero SHD does not contradict the theoretical guarantee, my concern was mainly about how users should interpret the differences between the estimated and true graphs in practice. Nevertheless, I am inclined to support acceptance and will therefore keep my rating unchanged.

Thank you for supporting our work. We agree that understanding the structural differences between estimated and true graphs is important in practice. Our weakly connected component analysis provides one indicator for structural interpretation. Identifying additional graphical properties and evaluation metrics that clarify these differences is a valuable research direction, which we will highlight in our discussion of future work.

Reviewer_B7FTOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

Causal discovery for linear Gaussian models with latent confounders and cycles, using a differentiable and consistent loss function.

Critical strengths

The paper provides a scalable (i.e. differentiable) structure learning algorithm that is provably correct in global optimum.

There is a good number of explanatory examples, although I feel they are not optimized for learning and more useful as a reference document.

Critical weaknesses

The authors kept the main paper as clean as possible; nevertheless, certain tools used look alien to people unaccustomed to them (including me), and their exposition was not quite pedagogical/explanatory enough for me. Consequently -- both direct and indirect -- I was not able to verify large amounts of the proofs in the appendix.

Interpretation of the equivalence class is generally the most difficult part of this type of papers. I can't say I was able to understand the implications of the class this paper proposes at all. I do not think this is a weakness of this paper in particular, however, because I struggle with such results in general.

Questions for authors

None.

2 comments

We thank the reviewer for their valuable comments and address them below:

Introductory section and readers guideline

Thank you for this feedback. We will add introductory sections to the appendix that guide readers through the concepts needed to understand the paper and its proofs. We will also provide a proof roadmap explaining what each result establishes and where it is used.


*Why marginal quasi-equivalence? *

Our distributional perspective is motivated by a limitation of conditional-independence (CI) comparisons. Two graphs are Markov equivalent if they imply the same CI relations. In the standard fully observed linear Gaussian DAG (i.e., without having cycles) model, all Markov-equivalent graphs generate the same family of observed distributions. With directed cycles, however, graphs with the same observed CI relations can impose different additional constraints on the observed distributions (Ghassami et al., 2020; Améndola et al., 2020, Lacerda et al., 2008). CI relations therefore need not capture all information relevant to distinguishing these models. For our zero-mean Gaussian models, the covariance matrix determines the observed distribution completely. We consequently compare the covariance families generated by admissible parameters of each graph, rather than only their CI relations.

Marginal quasi-equivalence means that two graphs share a set of observed distributions whose dimension equals that of each graph’s distribution family. As a geometric illustration, consider two filled squares in a plane. Their intersection may be a region with positive area (dimension two), a line segment (dimension one), or a single point (dimension zero). Our definition requires the first type of overlap: the intersection has the same dimension as both squares. Sharing only an edge or a point is insufficient, while the squares need not coincide completely. The same principle applies to the graphs’ observed covariance families, with dimension measured relative to each family.

We hope the above explanation addresses your concern, and we sincerely appreciate your time and consideration.

Dear Reviewer,

Thank you again for taking the time to review our paper. As the discussion period is approaching its end, we wanted to follow up and check whether our response above has addressed your questions and concerns. If anything remains unclear or if any further clarification would be helpful, we would be very happy to provide it. We greatly appreciate your time and consideration.

Reviewer_F6AYOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper investigates causal discovery in linear Gaussian structural causal models with latent confounders and directed cycles. To address the challenges of complex distributional equivalence classes and the non-identifiability of unique causal directions in this setting, the paper introduces marginal quasi-equivalence as a relaxed identification target. It then develops a continuous optimization objective based on Bernoulli-gate relaxation and proves that the resulting expected objective shares the same global infimum as the discrete structure-learning objective. Building on this result, the paper proposes an algorithm to learn causal graphs belonging to the same marginal quasi-equivalence class.

Critical strengths
  • The authors introduce marginal quasi-equivalence for linear Gaussian models, deliberately relaxing the objective of causal structure recovery.

  • The authors employ Bernoulli-gate relaxation to continuously relax discrete structure learning and prove that the expected objective after relaxation shares the same global infimum as the original discrete objective, providing a gradient-optimizable alternative for discrete causal structure search.

Critical weaknesses
  • The methodological novelty and advantages over existing approaches remain insufficiently established. The theoretical framework and algorithmic design may be perceived as a combination of existing continuous relaxation techniques for optimization and a newly defined, weaker equivalence class. Compared with established notions such as Markov equivalence, distributional equivalence, and other equivalence concepts for cyclic structures, what distinctive structural invariants does marginal quasi-equivalence capture? Furthermore, compared with existing continuous-relaxation methods for causal structure search, what substantive and non-redundant theoretical or algorithmic advantages are provided by the proposed global-infimum equivalence theorem and the method's adaptation to settings involving both latent confounding and directed cycles? These distinctions and advantages are not sufficiently articulated in the current manuscript, leaving the methodological novelty insufficiently substantiated.
  • The global-infimum equivalence between the relaxed and discrete objectives does not guarantee recovery of the true graph's marginal quasi-equivalence class. The paper lacks clear identifiability conditions and recovery guarantees, as well as an analysis of discretization errors, leaving a significant gap between the theoretical results and reliable causal structure recovery.
Questions for authors
  • The paper introduces the concept of marginal quasi-equivalence, but its scope and criteria for determining equivalence remain insufficiently explained. Is this concept merely an artificially defined, weaker objective for causal structure recovery? The paper does not clearly specify under what conditions different graph topologies, edge orientations, and latent-variable connections are grouped into the same marginal quasi-equivalence class.
  • Is the upper bound on the number of latent variables specified a priori or determined using a data-driven or alternative strategy? If it must be specified in advance, could the authors provide an adaptive selection procedure based on data characteristics or model complexity? Furthermore, how do excessively large or small upper bounds affect computational efficiency, structural identification accuracy, and the final recovery results?
  • The experimental evaluation lacks comparisons with causal discovery methods capable of handling both latent variables and directed cycles, making it difficult to adequately assess the advantages of the proposed approach over existing methods, like CCI.
4 comments

We thank the reviewer for their valuable comments and address them below:

The methodological novelty and advantages over existing approaches

We thank the reviewer and agree that the manuscript should state the novelty more clearly. We will revise Sections 1, 3.1 and App. E accordingly.

Marginal quasi-equivalence. Ghassami et al. (2020) define distribution equivalence and quasi-equivalence (Defs. 3 and 9), and prove consistency, only for latent-free models. Our notion extends theirs to models with latent confounders. In the latent-free case, it is a precise version of their quasi-equivalence, and our marginal distribution equivalence coincides exactly with their distribution equivalence, since without latents M(G)={Θ−1:Θ∈Θ(G)}\mathcal{M}(G) = \{\Theta^{-1} : \Theta \in \Theta(G)\}. Their Def. 9 measures sets of "linearly independent parameters". Since the map from structural parameters to distributions is nonlinear and generally not injective (e.g., a single latent parent of two observed variables has five parameters, but its covariance family is all of S++2\mathbb{S}^2_{++}, which has dimension three), it is unclear which parameters this refers to, and with latent variables the candidates' parameter spaces differ altogether. Our Eq. (7) instead compares the observed covariance families directly by their semialgebraic dimension; it requires no choice of parameters and requires the shared part to be full-dimensional in both families.

This formulation yields a concrete invariant: if G1+≅OG2+G_1^+ \cong_O G_2^+, then V(G1+)=V(G2+)V(G_1^+) = V(G_2^+). This follows from Prop. 2 and Lemma 16, and we will state it as a lemma. Quasi-equivalent graphs therefore satisfy exactly the same polynomial equality constraints on ΣO\Sigma_O. These include all CI constraints implied by the graph as well as cycle- and latent-induced (e.g., tetrad) constraints. This makes precise the remark of Ghassami et al. that quasi-equivalent graphs share the same "hard constraints", which they define as distributional constraints whose solution set has Lebesgue measure zero in the parameters involved. Our results thus give the chain: distribution equivalence ⇒\Rightarrow quasi-equivalence ⇒\Rightarrow equal Zariski closures ⇒\Rightarrow same CI constraints. Quasi-equivalence is strictly finer than CI-based equivalence, since cyclic graphs with the same CI relations can still be distinguished from observational data (Lacerda et al., 2008). The middle step cannot be reversed in general. Equal Zariski closures do not guarantee a full-dimensional intersection of semialgebraic families (App. A.6). This is why we need Assumption 9. In Ghassami et al., Assumption 1 only yields equal hard constraints, and the proof of their Theorem 3 concludes quasi-equivalence from this "by definition", i.e., it uses the converse of their remark, which does not hold in general. Moreover, their condition that constraint-relaxing graphs need more edges (Assumption 1, second part) is justified only by the remark that it "is clearly the case for DAGs". Our Prop. 22 proves the analogous condition (our Assumption 8) for cyclic targets without 2-cycles, against competitors with latents and reciprocal edges. Prop. 22 also covers targets with latent variables, as long as their covariance family can be generated by such a latent-free graph. App. B.2 shows that the condition can fail under our edge-plus-latent penalty when the true covariance family cannot be generated by any latent-free graph.

Continuous formulation. We agree that using Bernoulli gates are established tools and not new (App. E). In DCCD-CONF (Sethuraman and Fekri, 2025), Theorem 2 pertains to characterization of exact maximizers of a discrete population score with a fixed, suitably small λ\lambda and interventional data. No result there relates their relaxation (Eq. 10) to that score. Our Prop. 10 shows that the Bernoulli objective, which also gates latent activation and latent-to-observed edges, has the same global infimum and the same minimizing graphs as the discrete penalized likelihood, with a binary minimizer whenever the infimum is attained. This is the link that allows us to apply Theorem 4 and Corollary 6 to the formulation we optimize, yielding a sample-based consistency guarantee with λn=12log⁡n\lambda_n = \tfrac12\log n from observational data alone.


Identification conditions and recovery guarantees...

The statement that the paper lacks identification conditions and recovery guarantees is incorrect. Section 4.1 explicitly states assumptions of algebraic faithfulness, structural minimality, and full-dimensional overlap (Assumptions 7–9). Lemma 17 in Appendix A.7 shows that these imply the identification condition required by Corollary 6. Together with Theorem 4, this establishes that, under the stated assumptions, every global score minimizer is marginally quasi-equivalent to the data-generating graph with probability tending to one. The consistency proofs appear in Appendices A.9–A.10.

We thank the reviewer for their valuable comments and address them below:

The paper introduces the concept of marginal quasi-equivalence, but its scope and criteria for determining equivalence remain insufficiently explained...under what conditions different graph topologies, edge orientations, and latent-variable connections are grouped into the same marginal quasi-equivalence class

Motivation and interpretation of marginal quasi-equivalence Our distributional perspective is motivated by a limitation of conditional independencies in distinguishing linear Gaussian models with cycles. In particular, with directed cycles, graphs with the same observed CI relations can impose different additional constraints on the observed distributions (Ghassami et al., 2020; Améndola et al., 2020; Lacerda et al., 2008, Section 5.1). CI relations therefore need not capture all information relevant to distinguishing these models. Marginal quasi-equivalence addresses an observational ambiguity where, when two graphs induce the same observed distribution, even unlimited observational data cannot distinguish those graphs at that distribution. Our criterion requires this ambiguity to extend over a shared set with the full dimension of both covariance families, rather than arise only at exceptional parameter choices. It follows the positive-overlap motivation of Ghassami et al. (2020), Definition 9, extended to graphs with different latent variables. For intuition, two filled rectangles satisfy the analogous criterion when their intersection has positive area. Sharing only an edge or a point is insufficient, while the rectangles need not coincide completely. Likewise, our definition requires more than fitting one covariance, but less than equality of the entire covariance families.

Properties preserved by marginal quasi-equivalence

1-We additionally establish that marginal quasi-equivalence preserves the partition of observed variables into weakly connected components, including connections through latent variables. Two observed variables belong to the same component if a path connects them when arrow directions are ignored; the path may pass through latent variables. Therefore, under our consistency assumptions, every global minimizer of our proposed score assigns the observed variables to the same weakly connected components as the true graph, with probability tending to one as the sample size increases. We have added the definitions and proof to Appendix F. Marginally quasi-equivalent graphs produce exactly the same groups of observed variables. Thus, two observed variables belong to the same weakly connected component in one graph if and only if they do in the other. Individual edges, orientations, and latent structures within these components may nevertheless differ. For example, Appendix B.3 shows that a graph with one latent parent of two observed variables and no observed edge generates the same covariance family as the latent-free graph 1→21\to2. The reverse orientation 1←21\leftarrow2 also generates this family. All three graphs generate every positive-definite 2×22\times2 covariance matrix and therefore satisfy even the stronger condition of distribution equivalence. Their observed variables belong to the same weakly connected component despite these structural differences. Our response to Reviewer 3 evaluates component recovery using ARI, where 1 indicates identical component partitions. At 64 observed variables, with a latent count approximately 20% of the observed count and maximum observed-to-observed total degree four, mean ARI across ten matched datasets is 1.0 for our method, 0.9 for DGlearn, and 0.8 for both Améndola et al. and DCCD-CONF. The revised manuscript, uploaded to OpenReview, includes the experimental details and comparison plots in Appendix F.

2-As explained in our earlier response concerning methodological novelty, marginal quasi-equivalence also preserves polynomial equality constraints on the observed covariance. Specifically, if an equation involving covariance entries holds for every covariance matrix generated by one graph, it must also hold for every covariance matrix generated by the other, provided the equation is polynomial. We refer to that earlier response for more details.

The global-infimum equivalence between the relaxed and discrete objectives does not guarantee recovery of the true graph's marginal quasi-equivalence class

The global-infimum equivalence result (Proposition 10) establishes a separate result: under its stated admissibility conditions, the exact Bernoulli formulation preserves the discrete objective’s global infimum. As stated in Proposition 10, if this infimum is attained, a global minimizer exists with all gate probabilities equal to zero or one, so the gates select one fixed graph. Under the stated sampling and identification assumptions, Corollary 6 then guarantees that such globally minimizing graphs are marginally quasi-equivalent to the data-generating graph with probability tending to one as the sample size increases.

If “discretization error” refers to thresholding the gate probabilities obtained by numerical optimization or to using the Binary Concrete approximation, these are limitations of practical optimization, not an absence of explicit identification conditions (Assumptions 7–9, Section 4.1) or of the conditional recovery guarantee established in Corollary 6


Is the upper bound on the number of latent variables and adaptive selection procedure Lmax⁡L_{\max} is specified before optimization, as stated in Sections 2.1 and 3.2. The activation gates in Section 4.2 learn the number of active latent variables within this bound. The complexity penalty discourages unnecessary latent variables and edges.

Appendices D.1--D.2 report the experimental choice: Lmax⁡=2⌊pr+1/2⌋+1L_{\max}=2\lfloor pr+1/2\rfloor+1 for generating latent ratio r>0r>0, and Lmax⁡=0L_{\max}=0 for no-latent experiments. This uses the generating ratio and therefore does not constitute data-driven bound selection.

A bound that is too small may exclude every graph marginally quasi-equivalent to the data-generating graph, making our recovery target unattainable within the candidate class. A smaller bound may nevertheless suffice if an equivalent representation uses fewer latent variables. A larger bound adds parameters and increases computational demands. For an enlarged fixed bound, the quasi-equivalence guarantee still requires the identification assumptions to hold for the enlarged candidate class (Section 4.1, Corollary 6).

A practical adaptive extension is to fit models using several prespecified bounds and select the bound with the highest mean log-likelihood on held-out data. However, our way of activation gates modeling already selects the number of active latent variables within Lmax⁡L_{\max}, and in our experiments, optimizing the activation gates within the prescribed bounds was sufficient to obtain the reported performance, without an additional search over (Lmax⁡L_{\max}), so we did not require a separate bound-selection step in these experimental settings.


The experimental evaluation lacks comparisons with causal...

Our experiments already include more related and recent papers DCCD-CONF, and Am'endola et al., both of which handle directed cycles and latent confounding, albeit under their respective assumptions. We additionally compare against DGlearn, a closely related method for cyclic linear Gaussian models without latent confounders. Thus, comparisons with methods addressing both challenges are present.

CCI is also relevant. However, it returns a partially oriented maximal almost ancestral graph over observed variables. Its endpoints encode ancestral information, with circles indicating unresolved orientations; its edges should not simply be interpreted as direct structural coefficients.

Our bidirectional compatibility metric requires a specified covariance family. We generate covariances by varying a graph’s admissible parameters and fit those parameters to covariances generated from another graph. CCI’s output does not directly provide the cyclic Gaussian parameterization required for these operations. From the output of CCI, we cannot compute our metrics and compare with other methods directly.

We hope the above explanation addresses your concern, and we sincerely appreciate your time and consideration.

Dear Reviewer,

Thank you again for taking the time to review our paper. As the discussion period is approaching its end, we wanted to follow up and check whether our response above has addressed your questions and concerns. If anything remains unclear or if any further clarification would be helpful, we would be very happy to provide it. We greatly appreciate your time and consideration.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.