acceptodds
Under review as a conference paper at ICLR 2027

Provable Guarantees for Learning Pure Latent High-dimensional Structural Equation Models

Abstract

Learning Directed Acyclic Graphs (DAGs) from observational data is a fundamental challenge in causal inference, exacerbated when the true causal mechanisms occur among unobserved latent variables. In this paper, we study the identifiability and learning of pure latent high-dimensional Structural Equation Models (SEMs). While traditional causal discovery algorithms only recovering Markov equivalence classes (CPDAGs) or succumbing to latent confounding. We propose a generic solver framework to recover the exact, fully-directed latent DAG in pure latent high-dimensional SEMs. Our two-phase approach first reconstructs the latent covariance from observational data, and then iteratively identifying and marginalizing terminal vertices to recover the latent DAG structure. Under this framework, we relaxes the faithfulness assumption in standard causal discovery algorithm, and provides theoretical guarantees for topological ordering, bounded parameter estimation error, and exact support recovery via thresholding. We establishing that exact latent DAG recovery is achievable with logarithmic sample complexity of where is the number of latent variable. Our experiments on synthetic data and a real-world fMRI brain connectivity dataset demonstrates that our method successfully recovers exact directed causal structures where standard observed or equivalence-class baselines fail.

Primary area:
learning theory
Keywords:
Learning Theory, Causal inference, DAG, SEM, Provable Guarantee

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_eWYAOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper investigates causal discovery among unobserved latent variables. Existing methods for recovering causal structures among latent variables may fail to identify the true causal graph uniquely, and some approaches also suffer from high computational cost. Under the strong assumption of a pure latent model, this paper first reconstructs the latent covariance and precision matrices from the observed data. It then estimates the causal structure among the latent variables using a peeling procedure that sequentially removes terminal vertices based on the diagonal entries of the precision matrix.

The paper further introduces an identifiability condition requiring, at every peeling step, the precision diagonal of each terminal vertex to be smaller than those of all non-terminal vertices, together with a minimum signal strength condition. Under these conditions, the authors theoretically show that the method can recover the fully directed true latent DAG itself, rather than only its Markov equivalence class. The paper also derives the sample complexity required when CLIME or Graphical Lasso is used to estimate the precision matrix. Experiments on synthetic data demonstrate that the empirical recovery performance is broadly consistent with the theoretical predictions.

Critical strengths

The critical strength of this paper is that it proposes a new method for causal structure learning among latent variables by combining latent covariance/precision matrix reconstruction from observed data with a peeling procedure that sequentially removes terminal vertices based on the diagonal entries of the precision matrix. This results in a computationally efficient framework with theoretical guarantees for latent causal discovery.

Regarding the theoretical guarantees, the paper combines the theoretical properties of the individual components with identifiability and minimum signal strength conditions, and shows that the overall method can recover the fully directed true latent DAG itself, rather than only its Markov equivalence class.

Another strength is that the paper derives explicit sample complexity results when CLIME or Graphical Lasso is used for precision matrix estimation.

Critical weaknesses

The paper is generally well organized, clearly written, and theoretically sound. However, its most significant drawback is the lack of any discussion on how to determine the number of latent variables. The paper provides neither a practical criterion for selecting the latent dimension nor an analysis of how the estimated causal graph may change depending on this choice. This issue could become a major obstacle to the practical use of the proposed method. A discussion of this problem, together with some principled criterion for selecting the number of latent variables, could substantially broaden the applicability of the proposed framework.

In addition, the paper does not sufficiently discuss how likely the diagonal-entry condition required by the peeling procedure is to hold in real-world data. Even for observed variables, this condition is not particularly intuitive to assess in practice, and for latent variables it is even less obvious how a user could determine whether the condition is plausibly satisfied. Discussing how restrictive this assumption is in realistic settings, how often it may be expected to hold, or whether there are simple sufficient or diagnostic criteria for assessing it would further improve the practical applicability of the proposed method.

Questions for authors
  • Could you discuss how the number of latent variables should be determined, or how the choice of the latent dimension affects the recovered causal structure?
  • Could you discuss how likely the precision-matrix diagonal condition required for the peeling procedure is to hold in real-world data, or whether there is a simple practical criterion for assessing whether this condition is satisfied?
2 comments

We sincerely thank the reviewer for their very kind and encouraging assessment of our manuscript, as well as for raising two thoughtful questions regarding practical senario. We appreciate these constructive suggestions. We address both questions in detail and outline how we will incorporate these discussions into the revised manuscript.

1. Real-World Plausibility of the Identifiability Condition

We thank the reviewer for raising this important practical question about how likely Assumption 1 is to hold in real-world data.

Our diagonal-gap identifiability condition is directly adapted from the foundational precision-matrix DAG learning framework of Ghoshal & Honorio (2018) ("Learning linear structural equation models in polynomial time and sample complexity", AISTATS 2018). Notably, in their work, Ghoshal & Honorio specifically examined the applicability of this exact condition on 7 real-world gene-expression datasets, demonstrating that precision-matrix diagonal ordering serves as an effective and reliable structural criterion in real-world biological networks. In Addtion we do have a real world example experiment included in the appenddix.

In the revised manuscript, we will explicitly cite Ghoshal & Honorio's real-world empirical findings alongside this variance-ratio intuition in Section 3.3 so practitioners have clear context on when Assumption 1 is expected to hold.

2. Determining the Number of Latent Variables (pp)

We agree with the reviewer that understanding how the latent dimension pp is chosen and how its choice impacts recovery is a very important aspect of applying latent variable models in practice.

To the best of our knowledge, our work is the first to establish provable, finite-sample guarantees for exact, fully-directed DAG recovery in pure latent high-dimensional SEMs (including Gaussian and sub-Gaussian regimes). Prior methods either recover only Markov equivalence classes/CPDAGs (e.g., Silva et al., 2006), require strict non-Gaussianity and higher-order cumulants without finite-sample O(log⁡p)\mathcal{O}(\log p) guarantees (e.g., GIN / Cai et al., 2025), or assume all variables are directly observed (e.g., Ghoshal & Honorio, 2018). Because no existing baseline solves this exact problem with comparable theoretical guarantees, we made our best effort in the experiments to construct reasonable comparison baselines (such as two-stage combinations of latent covariance/cluster estimation with observed-data DAG/CPDAG solvers, as well as oracle baselines) and focused our theoretical analysis on establishing exact recovery given pp.

In many applied settings, pp is determined a priori by domain structure (for example, in our fMRI experiment, voxel measurements are naturally partitioned into pp anatomical Regions of Interest). When pp is not known in advance, practitioners can estimate it in a preliminary step using existing pure-cluster/tetrad tests (e.g., BPC or FOFC) or spectral eigenvalue thresholding on Σ^XX\widehat{\Sigma}_{XX}.

We fully agree with the reviewer that estimating the unknown number of latent variables pp alongside exact finite-sample latent DAG recovery is a very valuable and natural extension of our framework.

Thank you for your thorough response. I understand that the diagonal-entry condition has been empirically examined in prior work. Regarding the number of latent variables, I understand that the authors primarily rely on prior domain knowledge in practical settings, while estimating the number of latent variables is considered an important direction for future work. Overall, I believe my concerns have been addressed to a reasonable extent. Since my original evaluation was already in favor of acceptance, I will keep my rating unchanged.

Reviewer_cda9OpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
1
Soundness
2
Read review
Paper summary

Latent covariance reconstruction:

  • in the pure measurement model with orthogonal/linear relationship between latents and observed, the authors reconstruct the covariance matrix of the latent variables from observed data

Identifiability & theoretical guarantees

  • under an identifiability condition about terminal vertices, the authors give some theoretical guarantees for their approach.

Experiments

  • the approach is evaluated on synthetic data matching their identifiability assumption.
Critical strengths

Problem setting:

  • this is an important problem: recovering latent causal relationships from potentially high-dimensional observations. Causality problems are not usually presented as a simple collection of 10 variables that are all easily observable, so causal discovery on latent variables is clearly important for real-world problems.
Critical weaknesses

Writing/exposition

  • The algorithm is not stated anywhere in the paper, and only vaguely described. The paper is poorly written and was not checked for spelling or grammar mistakes, and a variety of formatting mistakes occur with references and theorem/proposition names.

Comparison to CPDAGs/Markov equivalence classes:

  • the paper claims to improve over methods, as it can identify DAGs, but others can only produce CPDAGs. The key information source to make this claim is not the method or theory, but Assumption 1. This is not made clear in the abstract or introduction, and makes the experiments questionable by comparing to methods that target different problems.

Theoretical guarantees:

  • the sample complexity and some of the theory does not appear to account for the fact that estimates of the latent covariance mechanism also have a sample complexity.
Questions for authors

None

1 comment

We sincerely thank the reviewer for their feedback and for taking the time to evaluate our manuscript. We are committed to ensuring the revised version is as clear, polished, and self-contained as possible. While we are glad that other reviewers found the manuscript well-organized and clearly written, we appreciate your perspective on how we can further improve the exposition and presentation. Below, we directly address your three main concerns.

1. Algorithm Specification and Formatting

The mathematical steps of our two-phase procedure are formally detailed in Section 3.4 of the submitted manuscript. However, we agree that adding a dedicated, self-contained pseudocode block will make the algorithmic flow immediately accessible to all readers. In the revised manuscript, we will place Algorithm 1 directly before Section 3.4, so that Section 3.4 serves as a step-by-step explanation and discussion of the pseudocode.

We are conducting a thorough, line-by-line proofreading pass to polish grammar, unify all theorem/proposition cross-references, and ensure formatting consistency across the main text and appendix. If there are specific spelling, grammatical, or reference/formatting mistakes you noticed during your review, we would be very grateful if you could point them out so we can ensure every single one is addressed in the camera-ready version.

2. Role of Assumption 1 and Fairness of Experimental Comparisons

  • Clarifying the Role of Assumption 1 vs. Our Theoretical Contribution: We agree that Assumption 1 (the diagonal-gap condition) is the population-level symmetry breaker that makes unique DAG identification possible beyond the Markov equivalence class (CPDAG)—analogous to equal error variances in observed-variable SEMs (Peters & Bühlmann, 2014; Ghoshal & Honorio, 2018) or non-Gaussianity in LiNGAM/GIN.

At the same time, Assumption 1 alone only establishes identifiability if the latent variables ZZ were directly observed. In our pure latent setting, Phase 1 reconstructs Σ^\widehat{\Sigma} from noisy proxies XX only up to an diagonal sign-flip ambiguity. Our core algorithmic and theoretical contribution is proving that generic sparse precision solvers and sequential Schur complement marginalization commute with F^\widehat{F} across all peeling steps, guaranteeing exact support recovery S(D^)=S(D∗)\mathcal{S}(\widehat{D}) = \mathcal{S}(D^*) without ever knowing the true latent signs.

  • Context on Literature & Baseline Comparisons: To the best of our knowledge, our work is the first to establish provable, finite-sample guarantees for exact, fully-directed DAG recovery in pure latent high-dimensional SEMs (spanning both Gaussian and sub-Gaussian regimes). Prior methods either recover only Markov equivalence classes/CPDAGs (e.g., Silva et al., 2006), require strict non-Gaussianity and higher-order cumulants without finite-sample O(log⁡p)\mathcal{O}(\log p) guarantees (e.g., GIN / Cai et al., 2025), or assume all variables are directly observed (e.g., Ghoshal & Honorio, 2018).

    Because no existing baseline solves this exact problem with comparable theoretical guarantees, we made our best effort in our experiments to construct reasonable comparison baselines (including unique-DAG non-Gaussian methods, two-stage CPDAG solvers, and oracle baselines). To ensure complete fairness when comparing against CPDAG methods, we will also report undirected skeleton recovery metrics alongside directed DAG metrics in the revised manuscript, showing that our regularized second-order estimator outperforms baselines even on skeleton recovery in the high-dimensional (n≪Qn \ll Q) regime.

3. Sample Complexity Accounting for Latent Covariance

We would like to clarify that the estimation error and sample complexity of recovering the latent covariance matrix are already incorporated into our theoretical framework through the modular structure of Assumption 3 and the Main Theorem:

  1. Modular Error Abstraction (Assumption 3): Rather than tying our structural recovery theory to a single covariance estimation method, Assumption 3 explicitly models the entry-wise estimation error ε\varepsilon of the latent covariance/precision reconstruction. This allows any general solver or Phase 1 procedure satisfying standard ℓ∞\ell_\infty concentration bounds to be plugged into our framework.
  2. Propagation into the Final Theorem: Our Main Theorem (and subsequent Corollaries) explicitly carries this estimation error ε\varepsilon through every step of the sequential peeling procedure, establishing the exact sample size nn required so that AεA\varepsilon remains strictly below the diagonal gap Δgap\Delta_{\text{gap}} and minimum edge signal Dmin⁡D_{\min}.
Reviewer_XFQCOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper considers the problem of recovering the causal structure among latent variables when we may only observe the proxy variables (i.e., the pure child). Existing methods under this setting either use higher-order cumulants and suffer from extreme sample complexity or rely on rank faithfulness assumptions and offer only asymptotic consistency. The authors propose a two-stage approach for uniquely recovering the underlying DAG. Specifically, they first use existing methods to recover the underlying covariance matrix among the latent variables from observational data. Subsequently, they show that the terminal (leaf) nodes in the underlying graph have strictly smaller diagonal precision entries than non-terminal nodes, and this can be used to sequentially recover the terminal nodes and hence the causal structure. They show that the DAG can be uniquely recovered under the proposed assumptions, and provide a finite-sample recovery guarantee, where the exact latent DAG recovery is achievable with logarithmic sample complexity of O(log p), where p is the number of latent variables.

Critical strengths
  1. The authors provide a finite-sample recovery guarantee of the proposed method, which has been rarely considered in the literature.
  2. The overall presentation of the proposed method is easy to follow. The notations, assumptions, identification algorithm, and the corresponding identifiability results are all clearly presented.
Critical weaknesses
  1. The restrictiveness of Assumption 1 is merely discussed. As the author mentioned, this assumption is necessary for unique identifiability. Therefore, it would be better if the authors could provide a toy example explaining this assumption and what would happen if this assumption is violated.
  2. The paper lacks a clear comparison with the existing methods, especially with similar methods for unique identification. For the theoretical results, it is not clear to me if there are any connections between the conditions in a non-Gaussian setup. For the simulation results, the authors only show that the proposed method performs better when pp is small, which is expected as they are based on higher-order cumulants.
Questions for authors
  1. Does the proposed method require that each terminal variable have at least two pure children (i.e., x-variable)? If yes, how do the theoretical results compare with existing methods that rely on similar assumptions (plus non-Gaussianity)? If not, for a terminal variable that has only one child, how to distinguish the exogenous noise of this variable and the exogenous noise of its pure child?
  2. Can any type of equivalence class be identified if Assumption 1 is violated?
  3. How do the existing methods perform in the same setting as in Figure 1, especially the ones that only provide asymptotic consistency?
1 comment

We sincerely thank the reviewer for their insightful comments and constructive questions. Your feedback highlights important connections between our second-order framework and existing latent variable methods, and provides a great opportunity to clarify the intuition behind our identifiability conditions. Below, we address each of your weaknesses and questions in detail.

1. Intuition and Restrictiveness for Assumption 1

Far from being overly restrictive, Assumption 1 (the diagonal-gap condition) is a generalization and relaxation of the "equal variance" assumption widely used to achieve unique DAG identifiability in continuous SEMs (e.g., Peters & Bühlmann, 2014; Ghoshal & Honorio, 2018).

We actually provide a formal counterexample construction in the proof of Lemma 2 in the manuscript. Specifically, Lemma 2 demonstrates that when Assumption 1 is violated, there exist exponentially many distinct DAG structures that induce the exact same latent covariance and precision matrix, making unique causal identification mathematically impossible from second-order statistics. In the revised manuscript, we will add a clearer, dedicated comment directly under Lemma 2 to highlight this counterexample and provide intuitive discussion for the reader.

For Question 2, if Assumption 1 is violated globally, we lose the ability to uniquely orient edges via precision diagonals; however, the support of the precision matrix still encodes the moralized graph and conditional independencies. Thus, under standard faithfulness, one can still recover the classical Markov Equivalence Class (CPDAG). Furthermore, if Assumption 1 is only violated locally (e.g., a tie Ωi,i=Ωj,j\Omega_{i,i} = \Omega_{j,j} between a specific pair while all other nodes satisfy the gap), the ambiguity is strictly localized: we recover a restricted equivalence class where all other edges in the DAG are uniquely directed, and only the orientation between (i,j)(i, j) remains unidentifiable.

2. Pure Children Requirement (qi>1q_i > 1)

For Question 1, we do require each latent variable to have at least two pure observed children (qi>1q_i > 1), which is explicitly stated in line 142 of our manuscript. This condition is taken directly from Long et al. (2026), Definition 1.

Basically, if qi=1q_i = 1, ΣXi\Sigma_{X_i} reduces to a single scalar variance λ1=Var(Zi)+σxi2\lambda_1 = \text{Var}(Z_i) + \sigma_{x_i}^2 with no second eigenvalue. Without a second dimension to pin down σxi2\sigma_{x_i}^2, there are infinitely many ways to decompose the observed variance into latent variance Var(Zi)\text{Var}(Z_i) and exogenous measurement noise σxi2\sigma_{x_i}^2.

3. Comparison with Existing Methods and Non-Gaussian Setups.

For Question 3, to the best of our knowledge, our work is the first to establish provable, finite-sample guarantees for exact, fully-directed DAG recovery in pure latent high-dimensional SEMs (spanning both Gaussian and sub-Gaussian regimes). Prior methods either recover only Markov equivalence classes/CPDAGs (e.g., Silva et al., 2006), require strict non-Gaussianity and higher-order cumulants without finite-sample O(log⁡p)\mathcal{O}(\log p) guarantees (e.g., GIN / Cai et al., 2025), or assume all variables are directly observed (e.g., Ghoshal & Honorio, 2018). Because no existing baseline solves this exact problem with comparable theoretical guarantees, we made our best effort in our experiments to select and construct reasonable comparison baselines.

Furthermore, our second-order framework differs fundamentally from non-Gaussian methods in two key ways:

  1. Gaussian vs. Non-Gaussian Distributional Assumptions: Existing unique-identification methods for latent DAGs (e.g., GIN, Latent LiNGAM) strictly require non-Gaussian latent noises ϵZ\epsilon_Z and collapse to an equivalence class under Gaussianity. Because our method operates purely on second-order statistics (covariance and precision matrices) under Assumption 1, it achieves unique DAG identifiability for both Gaussian and sub-Gaussian distributions.
  2. Second-Order Moments vs. Higher-Order Cumulants (Why small-nn matters): Non-Gaussian methods rely on 3rd- or 4th-order cumulants to orient edges, which suffer from severe finite-sample variance and require very large sample sizes (n≫pn \gg p). By relying strictly on 2nd-order statistics, our framework leverages sparse precision estimators (CLIME, Graphical Lasso) that concentrate exponentially fast, yielding provable finite-sample recovery in the high-dimensional regime (n≪Qn \ll Q, with sample complexity n=O(log⁡p)n = \mathcal{O}(\log p)).

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.