- Rating
- 3
- Confidence
- 2
- Informativeness
- 3
- Presentation
- 2
- Soundness
- 3
Read review
Paper summary
This paper investigates causal structure learning from observational data in the presence of both directed cycles and latent confounders. Previous approaches typically either allow directed cycles while assuming the absence of latent confounders, account for latent confounders while assuming that the causal graph is a DAG, or address both but require interventional data or additional restrictions on graph structures.
The paper first introduces Marginal Quasi-Equivalence as a notion of equivalence between causal graphs based on the covariance matrices of observed variables. It then establishes that, under several assumptions, particularly structural minimality, the estimated graph is guaranteed to be marginally quasi-equivalent to the true graph. The authors further propose an algorithm for causal structure learning by parameterizing discrete graph structures using Bernoulli probabilities. Experiments on synthetic and biological data demonstrate that the proposed method achieves superior performance in terms of bidirectional covariance-family compatibility error.
Critical strengths
The main strength of this paper is that it clearly identifies the conditions under which recovery up to marginal quasi-equivalence, defined in terms of observational covariance matrices, can be theoretically guaranteed in the presence of both directed cycles and latent confounders, a setting in which theoretical guarantees have traditionally been difficult to establish. Although the proposed framework does not guarantee unique identification of the true causal graph, it rigorously establishes conditions under which the estimated graph is marginally quasi-equivalent to the true graph.
Since causal structures involving both directed cycles and latent confounders are highly relevant to real-world applications, demonstrating the theoretical feasibility of structure learning under such general conditions represents a significant contribution to the research field.
Critical weaknesses
The most critical weakness of this paper is that the gap between its theoretical guarantees and practical applicability is not sufficiently addressed. Although the theoretical consistency guarantee is limited to marginal quasi-equivalence, the paper proposes an algorithm that outputs a specific causal graph. The algorithm selects a single graph by imposing a complexity penalty, but there is no theoretical guarantee that the selected graph corresponds to the true causal graph. Despite this limitation, the paper provides little discussion of how and to what extent the estimated causal graph may differ from the true causal graph, particularly in terms of specific structural differences. Consequently, users have no clear basis for determining how much confidence they should place in the estimated causal structure. While I understand that the primary contribution of this paper lies in its theoretical guarantees, I believe that addressing this issue would substantially improve the practical applicability of the proposed method.
A related concern is that the experimental evaluation focuses exclusively on bidirectional covariance-family compatibility error. I understand that metrics such as Structural Hamming Distance (SHD) do not fully align with the objective of the proposed method, since structurally different causal graphs may be observationally equivalent. Nevertheless, given that the proposed algorithm outputs a specific causal graph, it is important to quantitatively evaluate how closely the estimated graph matches the true causal graph. Even if the results are unfavorable, such an evaluation would provide valuable insights into the characteristics and limitations of the proposed method and could therefore make a positive contribution to the paper.
Questions for authors
Could the authors discuss how users should interpret the causal graph produced by the proposed method in relation to the true causal graph? Alternatively, could the authors provide a more comprehensive evaluation, including metrics such as Structural Hamming Distance (SHD), to quantitatively and qualitatively clarify how the estimated causal graph differs from the true graph?
3 comments
Thank you for your careful and detailed response. Your explanation has helped me better understand where the main contribution of this paper lies. While I understand that a nonzero SHD does not contradict the theoretical guarantee, my concern was mainly about how users should interpret the differences between the estimated and true graphs in practice. Nevertheless, I am inclined to support acceptance and will therefore keep my rating unchanged.
Thank you for supporting our work. We agree that understanding the structural differences between estimated and true graphs is important in practice. Our weakly connected component analysis provides one indicator for structural interpretation. Identifying additional graphical properties and evaluation metrics that clarify these differences is a valuable research direction, which we will highlight in our discussion of future work.
We thank the reviewer for their valuable comments and address them below:
Our guarantee concerns recovery up to marginal quasi-equivalence, not exact recovery of the true underlying causal graph, which is not identifiable from observational data in linear Gaussian models (with or without cycles). Under the stated assumptions (Assumptions 7–9), Theorem 4 and Corollary 6 (Section 4.1; proofs in Appendices A.9–A.10) guarantee that every global score minimizer is marginally quasi-equivalent to the true graph with probability tending to one as the sample size goes to infinity. Such a graph may differ from the true graph in individual edges and therefore have a nonzero structural Hamming distance (SHD). SHD compares one particular graph representation and can penalize differences permitted by our recovery criterion. A nonzero SHD therefore does not, by itself, contradict our guarantee.
To clarify the structural interpretation, in the rebuttal phase, we additionally establish that marginal quasi-equivalence preserves the partition of observed variables into weakly connected components, including connections through latent variables. Two observed variables belong to the same component if a path connects them when arrow directions are ignored; the path may pass through latent variables. Therefore, under our consistency assumptions, every global minimizer of our proposed score assigns the observed variables to the same weakly connected components as the true graph, with probability tending to one as the sample size increases. We have added the definitions, and proof to Appendix F of the paper in the rebuttal phase.
To show this empirically, we evaluate component recovery using the adjusted Rand index (ARI) between the observed-variable component partitions of the true graph and the graph recovered by each method, where 1 means identical partitions. With 20% latent variables and maximum observed degree four, our method achieves ARI 1 on all completed trials. The following table reports mean ARI at the three largest evaluated graph sizes:
Amendola and DCCD-CONF were evaluated through 64 variables only. At 128 variables, three additional trials reached the six-hour limit during DGlearn’s search and are excluded from the matched comparison. The revised manuscript, uploaded to OpenReview, includes the experimental details and comparison plots in Appendix F.
We hope the above explanation addresses your concern, and we sincerely appreciate your time and consideration.