acceptodds
Under review as a conference paper at ICLR 2027

Direct Linear Latent Causal Structure Learning via Triplet Difference Constraint

Abstract

Learning causal structures among unobserved latent variables is fundamental yet challenging, especially when latent variables are causally related. Existing non-Gaussian methods rely heavily on repeated, pairwise or subset-wise independence tests, suffering from error accumulation and heavy computational overheads. To tackle this challenge, we propose a direct framework for latent causal discovery based on higher-order statistical information. Specifically, we introduce the the Triplet Difference (TD), a novel algebraic criterion formulated in terms of second- and third-order cumulants, which enables the characterization of latent causal relationships through observable data. Building on this criterion, we establish identifiability conditions for latent causal clusters and causal structures among latent variables. Rather than relying on repeated independence testing, our framework exploits the constraint-satisfaction patterns of TD across variable triplets to distinguish latent causal topologies, thereby enabling latent cluster identification and causal ordering. Complexity analysis confirms that our algorithm avoids exponential evaluation costs over observed variables. Extensive experiments on both synthetic and real-world datasets demonstrate the effectiveness of the proposed approach in recovering latent causal structures, corroborating the theoretical findings and practical utility of the framework.

Primary area:
causal reasoning
Keywords:
Causal Discovery, Latent Variables, Latent Causal Structure Learning

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_68vcOpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper presents an algorithm for causal structure learning applicable to causal structures in which all dependencies between observable variables are caused by latent parents.

Critical strengths

The strength of the paper is that it relies on a simple statistic (Eq 8) to infer common influences.

Critical weaknesses

The main limitation is that the novelty is limited and it is unjustified why not further statistics that can play a similar role are combined (questions 8-10).

Questions for authors

1-While cumulant tensors are described in Section 2.2, the results of Section 4 are not fully linked to cumulant properties, namely to the multilinear factorization of tensors: for Xi=∑j=1paijUjX_i = \sum_{j=1}^p a_{ij} U_j, i=1,...mi=1,...m, with UjU_j, j=1...pj=1...p independent, K(Xi1,...,Xin)=∑j=1p(∏r=1nairj)Kn(Uj)K(X_{i_1},...,X_{i_n})= \sum_{j=1}^p (\prod_{r= 1}^{n} a_{i_r j}) K_n(U_j). This property underlies the form of Eq 8 and 9 and clarifies the scope of the Triplet Difference approach: Once the linear latent causal model is adopted, no new set of cumulant properties needs to be derived. The novelty lies in the identification of the specific cancelation of Eq 8 and 9 for certain latent causal structures, and hence its utility for structure learning. Clearly separating standard cumulant properties from the novelty in their use would sharpen the scope of the contribution.

2-Related to the above, the proofs in Appendix A are very unnecessarily convoluted. They just apply the multilinear factorization, but the fact that the factorizing factors that cancel are not identified with auxiliary variables makes the formulas long and less transparent. Eq 12 to Eq 17 derive from a common principle that can be express in a more concise and transparent form.

3-Related to the above, the lack of additional transparency in the proof of Th 5.1 renders less visible an additional concern: lines 695-696 state, but do not prove, the desire case in which τij∣k(n)≠0\tau^{(n)}_{ij|k}\neq 0. In fact, the cancellation may still occur, but it would require some fine tuning of the mechanisms and hence of the coefficients and statistical properties. The statement would need to be punctuated by some assumption of stability to infinitesimal changes (faithfulness-like assumption), if a full proof without such an additional assumption cannot be fully shown.

4- In Figure 1, b1,2b_{1,2} should be b3,2b_{3,2}; b2,2b_{2,2} should be b4,2b_{4,2}.

5- After eq 7, b1,2b_{1,2} should be b2,1b_{2,1}.

6- Eq 5 is not consistent with the notation of Eq 3. You write C(L1,L1)C(L_1,L_1) and C(L13)C(L_1^3). For consistency with Eq 3 you will need to write C(L12)C(L_1^2) instead of C(L1,L1)C(L_1,L_1).

7-The novelty of the paper is not well circumscribed: the criteria of Eq 9 and 10 are not new, they are a concrete instantiation of a standard well-known cumulants property. The core of the Latent Causal structure learning Algorithm 2 is not new either, it is shared and preceded by several previous works in causal learning with latent parents. It goes back to Silva et al 2006, and is substantially analogous to Cai et al. 2019. The novelty is hence the implementation of the TD algebraic invariant as a cheap statistic to apply the Latent Causal structure learning Algorithm.

8-It is not justified why all the focus is on the statistic τ(i,j∣k)\tau(i,j|k) of Eq 9. There are alternatives to exploit the same sort of causal structure of Fig 1. For example, consider C(X12,X2)=b1,12b2,1K3(L1)C(X_1^2,X_2)= b_{1,1}^2 b_{2,1}K_3(L_1), C(X1,X2,X3)=b1,1b2,1a2,1b3,2K3(L1)C(X_1,X_2,X_3) = b_{1,1} b_{2,1} a_{2,1} b_{3,2} K_3(L_1), $C(X_2,X_3^2) = b_{2,1} a_{2,1}^2 b_{3,2}^2 ,K_3(L_1)$. Therefore, Fig 1 leads to C(X12,X2)C(X2,X32)−C(X1,X2,X3)2=0C(X_1^2,X_2)C(X_2,X_3^2) - C(X_1,X_2,X_3)^2= 0. So this $\tau_2(i,j|k) \equiv C(X_1^2,X_2)C(X_2,X_3^2) - C(X_1,X_2,X_3)^2 $ could have been used. But this is only one additional case. You also have, for Kiij=C(Xi,Xi,Xj)K_{iij}= C(X_i,X_i,X_j), that KiijKjkk=KiikKjjk=KijjKikk=Kijk 2K_{iij}K_{jkk} = K_{iik}K_{jjk} = K_{ijj}K_{ikk} = K_{ijk}^{\,2}, for different i,j,ki,j,k. Or you have KiijKikk=KiikKijkK_{iij}K_{ikk}=K_{iik}K_{ijk}, and KiijKjjk=KijjKijkK_{iij}K_{jjk}=K_{ijj}K_{ijk}. So the corresponding differences work as signatures of the same causal structure as well.

9-Note that in fact τ(i,j∣k)\tau(i,j|k) of Eq 9 could equally be used simply as τ′(i,j∣k)≡C(Xi,Xk2)C(Xi,Xk)C(Xj2,Xk)−C(Xj,Xk2)C(Xj,Xk)C(Xi2,Xk)\tau'(i,j|k) \equiv C(X_i, X_k^2)C(X_i,X_k)C(X_j^2, X_k)- C(X_j, X_k^2)C(X_j,X_k)C(X_i^2,X_k). This would have the advantage of avoiding numerical instabilities when C(Xj2,Xk)C(X_j^2, X_k) or C(Xi2,Xk)C(X_i^2,X_k) are close to zero. On the other hand, for any of the statistics τ′(i,j∣k)\tau'(i,j|k) or the ones described in point 8 above, which have a form A−B=0A-B =0, the actual test can be performed on (A−B)/(∣A∣+∣B∣+ε)(A-B)/(|A|+|B|+\varepsilon), being ε\varepsilon a very small constant. This would render the estimation of the difference proportional to their magnitude, but would avoid the product in the denominator.

10-The authors may argue that all these alternatives are valid, but that they prove that the use of Eq 9 is enough for completeness. However, two lines of arguments can be raised against that claim. First, the issue raised in point 3 above. Since some assumption of faithfulness is required, the combination of several statistics could be helpful to unmask cases in which a zero value is not associated with the causal structure, but a consequence of some parameter tuning. Second, and in practice perhaps more importantly, even with total faithfulness, in the regime of estimation from finite data, it is expected that performance could potentially improve substantially if not relying on a single statistic.

11-It would have been more general to present in Section 4 the general derivation using conditional cumulants, that is, terms like C(Xi,Xk,Xk∣W=w)C(X_i,X_k,X_k |\mathbf{W}=\mathbf{w}), where W\mathbf{W} are observable latent ancestors that inactivate paths such that the same procedure described in the paper is only applicable after conditioning on them. Nothing qualitatively changes, but the formulation is more general.

12- The restriction of no direct connections between observed nodes is quite restrictive. This is tested in Case 5, but this is only a single fixed scenario. It would have been more informative to present simulations with progressively stronger direct connections, or progressively more existing direct connections, and quantify the degradation of the latent structure recovery performance.

11 comments
AuthorsOpenReview ↗

Response to Reviewer 68vc (1/9) — Contribution and cumulant factorization

Thank you for your detailed review and constructive suggestions. We respond to your comments below.

W1 — Reviewer comment

The main limitation is that the novelty is limited and it is unjustified why not further statistics that can play a similar role are combined (questions 8-10).

Response to W1

We appreciate this critical comment. The core contribution of this work lies in constructing the TD constraints from an algebraic perspective to eliminate the effects of latent variables. We further leverage the constraint patterns across different observed triplets to identify latent causal clusters and recover the causal order over latent variables. This yields a unified algebraic criterion for both tasks without relying on independence tests. We emphasize that our novelty is not merely the computational form of this statistic itself. More importantly, our contribution establishes the mapping between these observable algebraic constraints and the target latent causal structures, together with a systematic structural recovery framework built upon this mapping.

For this reason, we deliberately focus on TD in isolation in the current theoretical development: combining additional statistics would entangle these carefully constructed algebraic constraints and break the clean cancellation mechanism that removes effects of latent variables, making it much harder to establish the identifiability guarantees for cluster grouping and causal ordering. We agree that integrating complementary statistics is a promising avenue for improving robustness, which we have discussed as future work in the revised manuscript.

We provide our responses to each question point by point.

Q1 — Reviewer comment

While cumulant tensors are described in Section 2.2, the results of Section 4 are not fully linked to cumulant properties, namely to the multilinear factorization of tensors: for Xi=∑j=1paijUjX_i=\sum_{j=1}^p a_{ij}U_j, i=1,…,mi=1,\dots,m, with UjU_j, j=1,…,pj=1,\dots,p independent, K(Xi1,…,Xin)=∑j=1p(∏r=1nairj)Kn(Uj)K(X_{i_1},\dots,X_{i_n})=\sum_{j=1}^p \left(\prod_{r=1}^n a_{i_r j}\right) K_n(U_j). This property underlies the form of Eq 8 and 9 and clarifies the scope of the Triplet Difference approach: Once the linear latent causal model is adopted, no new set of cumulant properties needs to be derived. The novelty lies in the identification of the specific cancellation of Eq 8 and 9 for certain latent causal structures, and hence its utility for structure learning. Clearly separating standard cumulant properties from the novelty in their use would sharpen the scope of the contribution.

Response to Q1

Your understanding of the contributions of Equations (8) and (9) in this paper is correct. The key finding of this work is that specific latent variable structures induce equality relations among the cross-cumulants of observed variables, which can be exploited for structure identification. By constructing Equations (8) and (9), we eliminate the shared influence of the same latent variable on observed variables, such that the latent relationships that are not directly observable can be characterized via constraints among observable statistics. Furthermore, we leverage the satisfaction patterns of these constraints across different triples to identify causal clusters and infer the causal order among latent variables.

Therefore, the contributions of this paper lie in the construction of these concrete algebraic constraints and the establishment of the connection between such constraints and latent variable structures. The multilinear decomposition property of cumulants serves as the theoretical foundation for this construction. Nevertheless, this decomposition alone does not directly yield the TD constraints and the corresponding structural identification criteria proposed in our work.

AuthorsOpenReview ↗

Response to Reviewer 68vc (2/9) — Proof revision and complementary constraints

Q2 — Reviewer comment

Related to the above, the proofs in Appendix A are very unnecessarily convoluted. They just apply the multilinear factorization, but the fact that the factorizing factors that cancel are not identified with auxiliary variables makes the formulas long and less transparent. Eq 12 to Eq 17 derive from a common principle that can be express in a more concise and transparent form.

Response to Q2

The proof in the original version adopted term-by-term expansion to explicitly demonstrate how observed variables within the same causal cluster share latent contributions and how the causal coefficients of distinct latent variables cancel out in the constructed statistic. Beyond merely presenting the multilinear decomposition of cumulants, we intended to help readers directly grasp why variables from the same cluster yield identical statistics given the same auxiliary variable, thereby satisfying the zero-TD equality constraint. This step is central to connecting cumulant properties with causal cluster identification.

Following your suggestion, we have revised the proof in the manuscript. We retain the representative derivation for the same-cluster case, explicitly factor out the causal coefficient to illustrate its cancellation, and adopt unified notation to streamline redundant calculations for other cases. This revision preserves the intended link between structural assumptions and equality constraints in the original proof, while making the shared principle behind Equations (12)–(17) much clearer.

Q3 — Reviewer comment

Related to the above, the lack of additional transparency in the proof of Th 5.1 renders less visible an additional concern: lines 695-696 state, but do not prove, the desire case in which τij∣k(n)≠0\tau_{ij|k}^{(n)} \neq 0. In fact, the cancellation may still occur, but it would require some fine tuning of the mechanisms and hence of the coefficients and statistical properties. The statement would need to be punctuated by some assumption of stability to infinitesimal changes (faithfulness-like assumption), if a full proof without such an additional assumption cannot be fully shown.

Response to Q3

We thank the reviewer for identifying a gap in the original proof. We agree that comparing the contributions of a single noise source leaves open the possibility of parameter cancellation in an individual TD constraint. The proof can be completed by using two auxiliary pure indicators and complementary cumulant configurations.

Suppose the two pure indicators under consideration belong to distinct latent variables LaL_a and LbL_b. Since each latent variable has at least two pure indicators, we can choose one additional pure indicator of each latent variable as an auxiliary variable. Let

va=Var⁡(La),vb=Var⁡(Lb),c=Cov⁡(La,Lb).v_a=\operatorname{Var}(L_a),\qquad v_b=\operatorname{Var}(L_b),\qquad c=\operatorname{Cov}(L_a,L_b).

For n≥3n\geq 3 and 1≤m≤n−21\leq m\leq n-2, define

U=cum⁡(La[m],Lb[n−m]),V=cum⁡(La[m+1],Lb[n−m−1]),U=\operatorname{cum}(L_a^{[m]},L_b^{[n-m]}),\qquad V=\operatorname{cum}(L_a^{[m+1]},L_b^{[n-m-1]}),

where the bracketed superscript denotes the number of times a variable appears as an argument of the cumulant.

We use the additional pure indicator of LbL_b as the auxiliary variable for configuration mm and the additional pure indicator of LaL_a for the complementary configuration n−m−1n-m-1. If the loadings and the nnth-order marginal cumulants of both latent variables are nonzero, then simultaneous vanishing of the two cross-product constraints yields, after canceling the loading factors,

cU−vbV=0,vaU−cV=0.cU-v_bV=0,\qquad v_aU-cV=0.

In the latent structural model, independent disturbances with strictly positive variances ensure that the latent covariance matrix is positive definite. Hence

vavb−c2>0.v_av_b-c^2>0.

The linear system therefore has only the solution U=V=0U=V=0. Under the nondegeneracy condition (U,V)≠(0,0)(U,V)\ne(0,0), at least one of the two constraints must be nonzero and thus distinguish the indicators as belonging to different clusters. This argument allows parameter cancellation in an individual TD constraint.

For third-order cumulants, m=1m=1 is self-complementary, and the existing procedure already evaluates both required constraints by checking all auxiliary variables. For fourth-order cumulants, the guarantee applies to the joint use of configurations m=1m=1 and m=2m=2. The third-order result does not provide a guarantee for either fourth-order configuration on its own.

We will state these identification conditions explicitly in the main text and provide the details of the loading cancellation and the proof for general cumulant orders in the appendix. We will also update the higher-order implementation to follow the revised criterion and re-evaluate the experimental results affected by this change.

AuthorsOpenReview ↗

Response to Reviewer 68vc (3/9) — Notation corrections and relation to prior work

Q4 — Reviewer comment

In Figure 1, b1,2b_{1,2} should be b3,2b_{3,2}; b2,2b_{2,2} should be b4,2b_{4,2}.

Response to Q4

We appreciate the reviewer for pointing out the labeling issue in Figure 1. We have corrected the corresponding symbols and cross-checked the annotations in the diagram, structural equations and main text to ensure consistent mapping between variable indices and causal edges.

Q5 — Reviewer comment

After Eq. 7, b1,2b_{1,2} should be b2,1b_{2,1}.

Response to Q5

You are correct, and we have fixed this issue in the revised manuscript.

Q6 — Reviewer comment

Eq. 5 is not consistent with the notation of Eq. 3. You write C(L1,L1)C(L_1, L_1) and C(L12)C(L_1^2). For consistency with Eq. 3 you will need to write C(L12)C(L_1^2) instead of C(L1,L1)C(L_1, L_1).

Response to Q6

We align Equation (5) with Equation (3) and standardize notations for repeated cumulant arguments in the revised manuscript.

Q7 — Reviewer comment

The novelty of the paper is not well circumscribed: the criteria of Eq. 9 and 10 are not new, they are a concrete instantiation of a standard well-known cumulants property. The core of the Latent Causal structure learning Algorithm 2 is not new either, it is shared and preceded by several previous works in causal learning with latent parents. It goes back to Silva et al. 2006, and is substantially analogous to Cai et al. 2019. The novelty is hence the implementation of the TD algebraic invariant as a cheap statistic to apply the Latent Causal structure learning Algorithm.

Response to Q7

We agree that the multilinear expansion of cumulants is a standard result and that the two-stage framework of first recovering measurement clusters and then inferring causal relationships among latent variables has precedents.

Our contribution is to establish an explicit algebraic correspondence between TD constraints on joint cumulants of observed variables and the underlying latent causal structure. By combining cross-cumulants of different orders, the TD constraints cancel terms arising from the effects of shared latent causes on the observed variables. The constraints serve different purposes in the two stages. To recover measurement clusters, we examine which TD constraints vanish for a pair of observed variables across multiple auxiliary variables. Under the stated nondegeneracy conditions, this pattern distinguishes indicators in the same cluster from indicators in different clusters. To infer the causal ordering of the latent variables, we use the observed variables in one cluster as references and infer the ordering from how the pattern of cross-cluster constraints changes with the choice of reference cluster.

The Triad criterion of Cai et al. (2019) obtains structural information by testing whether a pseudo-residual is independent of an auxiliary variable. TCD directly evaluates TD constraints on cross-cumulants of observed variables and specifies the conditions under which satisfaction of these constraints supports causal structure identification. This gives both cluster recovery and latent causal ordering a common algebraic criterion, removing the need to construct pseudo-residuals and perform independence tests at each stage. We have revised the manuscript to explain how the TD equalities are constructed, what structural distinctions they capture, and under what conditions those distinctions are identifiable.

Reviewer_W9gQOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes Triplet Difference (TD), a cumulant-based algebraic criterion for identifying latent causal clusters and relationships among latent variables. TD cancels indicator-specific loadings through cumulant ratios, and the resulting TCD algorithm combines triplet constraint patterns, causal ordering, and rank-based edge pruning. The algebraic intuition is appealing, but the statistical assumptions, global recovery guarantees, and empirical support for efficiency and robustness require further clarification.

Critical strengths
  1. Canceling measurement loadings through cumulant ratios provides an interpretable approach to latent structure identification.

  2. Cached cumulants avoid repeated kernel-based independence tests and their associated computational costs.

  3. Reporting direction recovery with both true and estimated clusters helps distinguish clustering errors from orientation errors.

Critical weaknesses
  1. The proof of Theorem 5.1 appears to assume that each observed variable under consideration has exactly one parent which is a latent variable. If I'm correct, please make this assumption explicit. This assumption cannot by implied by Assumptions 2.1 and 2.2, .

  2. Assumptions 2.1 and 2.2 are rather restrictive. For linear non-Gaussian latent causal model, [1,2] make neither Assumption 2.1 nor 2.2.

  3. Non-Gaussianity alone does not guarantee that the third-order TD ratios are well-defined. If all independent noise sources are symmetric, such as Laplace or uniform noise, the third-order cross-cumulants vanish, resulting in undefined ratios. While higher-order cumulants may address this degeneracy, in my experience, the variance of their estimators becomes very high, making the estimates highly inaccurate and less reliable than independence tests.

  4. The finite-sample implementation of the population-level triplet constraints is insufficiently specified. Algorithm 1 uses exact equalities, Algorithm 2 introduces a threshold δ\delta, and Section 7.2 mentions bootstrap confidence intervals, but the paper does not clearly explain how these decision rules are calibrated or how near-zero denominators are handled.

[1] Causal structure recovery with latent variables under milder distributional and graphical assumptions. ICLR 2024.
[2] Recovery of causal graph involving latent variables via homologous surrogates. ICLR 2025.

Questions for authors

Please see weaknesses.

6 comments
AuthorsOpenReview ↗

Response to Reviewer W9gQ (1/3) — Pure indicators and structural assumptions

Thank you for your detailed review and constructive suggestions. We respond to your comments below.

W1 — Reviewer comment

The proof of Theorem 5.1 appears to assume that each observed variable under consideration has exactly one parent which is a latent variable. If I'm correct, please make this assumption explicit. This assumption cannot by implied by Assumptions 2.1 and 2.2.

Response to W1

Yes, you are correct. The conditions for Theorem 5.1 imply that xix_i and xjx_j are the only pure children of the same latent variable. We have added the word own to Assumption 2.2 (Two-Pure-Child Assumption: Each latent variable has at least two pure observed variables as its direct own children) to make this implication clearer.

W2 — Reviewer comment

Assumptions 2.1 and 2.2 are rather restrictive. For linear non-Gaussian latent causal model, [1,2] make neither Assumption 2.1 nor 2.2.

Response to W2

We agree that both [1] and [2] allow direct relationships among observed variables and relax the requirement that each latent variable has at least two pure indicators. These two works achieve identifiability via conditions such as generalized pure indicator pairs and homologous proxies, respectively. We have added explicit comparisons of their structural assumptions and identification results in the revised manuscript.

This paper targets a more specific question: when each latent variable is measured by a dedicated set of observed variables with no direct causal edges among these observed variables, can we identify pure clusters and infer the latent causal order using a unified observable statistical criterion? Our contribution is a new identification rule, TD, derived from algebraic constraints that exploits information encoded in (joint) high-order cumulants. Under suitable identification conditions, TD cancels the effect of latent variables through ratios of cross-cumulants, and characterizes cluster membership and ancestral relations among latent variables by examining the zero-constraint patterns across different triples. The algorithm reuses pre-estimated cross-cumulants for both stages, which avoids repeatedly constructing pseudo-residuals and running independence tests for every candidate triple.

This setting is fundamental and widely adopted in latent causal discovery. In many cognitive, neuroimaging and multi-omics studies, latent quantities (e.g., latent cognitive factors, functional microbial modules, hidden disease phenotypes) are not directly measurable and are only reflected by a dedicated battery of observed variables, with no direct causal edges among observed variables — all association is mediated by the underlying latent variables. This is the standard setup of latent measurement models. Developing identification tools for this setting complements existing works that allow direct edges among indicators. Leveraging the algebraic redundancy induced by the measurement structure, our TD criterion unifies cluster assignment and causal ordering, yielding better computational efficiency and stability compared with repeated residual-based independence tests.

[1] Causal structure recovery with latent variables under milder distributional and graphical assumptions. ICLR 2024.

[2] Recovery of causal graph involving latent variables via homologous surrogates. ICLR 2025.

AuthorsOpenReview ↗

Response to Reviewer W9gQ (2/3) — Symmetric noise and fourth-order cumulants

W3 — Reviewer comment

Non-Gaussianity alone does not guarantee that the third-order TD ratios are well-defined. If all independent noise sources are symmetric, such as Laplace or uniform noise, the third-order cross-cumulants vanish, resulting in undefined ratios. While higher-order cumulants may address this degeneracy, in my experience, the variance of their estimators becomes very high, making the estimates highly inaccurate and less reliable than independence tests.

Response to W3

We agree that non-Gaussianity alone does not guarantee non-zero third-order cross-cumulants. When all independent noise sources follow symmetric distributions, the third-order criterion may degenerate at the population level. The ratio form in the paper is introduced to illustrate the cancellation of loading coefficients, while our experiments actually compute the cross-product difference to avoid division by near-zero sample cumulants. It should be noted that if the relevant cumulants degenerate in population, the cross-product difference cannot provide informative signals for structural discrimination either. Meanwhile, Definition 4.2 in our paper provides an alternative: when third-order cumulants vanish, we may resort to the 4-th Order or more than 4-th Cumulant-based Triplet Difference for inference.

For the structures considered in this work, the number of latent variables shared by each three observed variables is small. As a result, we do not need cumulants of very high orders in practical inference, which keeps the estimation error relatively mild. We have supplemented additional experiments to demonstrate the performance of the fourth-order cumulant variant. Specifically, the supplementary simulations adopt fourth-order cumulants for three noise settings: Gamma3\mathrm{Gamma}^3, Gumbel3\mathrm{Gumbel}^3, and Laplace3\mathrm{Laplace}^3. This combined scheme employs two complementary fourth-order constraints only in the cluster discovery stage, and latent variable ordering still relies on a single fourth-order configuration. The identifiability guarantee of the fourth-order criterion is subject to its corresponding non-degeneracy conditions. We construct percentile intervals via 500 bootstrap resamples to account for sampling variability in finite-sample decisions. The table below reports results under symmetric Laplace3\mathrm{Laplace}^{3} noise, and the full experimental results are provided in the appendix.

TCD under cubic Laplace noise

Complementary denotes TCD with complementary constraints. F1e and F1t use estimated and ground-truth clusters, respectively. Bold marks the higher F1 within each pair, including ties.

CaseNComplementaryOriginal TCD
LO ↓LC ↓MM ↓F1e ↑F1t ↑LO ↓LC ↓MM ↓F1e ↑F1t ↑
15000.150.000.030.701.000.150.000.030.701.00
10000.150.000.000.701.000.050.000.000.901.00
20000.200.000.000.701.000.200.000.000.701.00
25000.100.000.080.801.000.150.000.080.701.00
10000.050.000.000.901.000.050.000.000.901.00
20000.000.000.001.001.000.050.000.040.901.00
35000.100.000.030.780.950.170.000.030.771.00
10000.130.000.100.871.000.130.000.070.801.00
20000.130.000.000.771.000.170.000.000.701.00
45000.200.000.200.771.000.130.000.130.831.00
10000.030.000.030.971.000.070.000.060.871.00
20000.030.000.030.971.000.030.000.030.971.00
55000.350.000.180.201.000.350.000.200.301.00
10000.300.000.070.301.000.400.000.120.301.00
20000.500.000.120.201.000.350.000.030.501.00
65000.170.000.030.801.000.170.000.020.771.00
10000.200.000.080.671.000.170.000.000.671.00
20000.170.000.100.680.980.100.000.030.770.98
AuthorsOpenReview ↗

Response to Reviewer W9gQ (3/3) — Finite-sample implementation

W4 — Reviewer comment

The finite-sample implementation of the population-level triplet constraints is insufficiently specified. Algorithm 1 uses exact equalities, Algorithm 2 introduces a threshold , and Section 7.2 mentions bootstrap confidence intervals, but the paper does not clearly explain how these decision rules are calibrated or how near-zero denominators are handled.

Response to W4

Algorithms 1 and 2 are identification procedures derived from our theory, while Section 7.2 presents a practical, heuristic implementation. In experiments, all observed variables are first centered and standardized. For a dataset with N samples, we perform 500 rounds of bootstrap resampling with replacement, drawing N samples in each round. For the triplet difference (TD) of the form A/B−C/DA/B - C/D, our implementation directly computes AD−CBAD-CB on the original sample and every bootstrap replicate. Given a tail probability parameter aa, we construct two-sided percentile intervals using the a/2a/2 and 1−a/21-a/2 quantiles of the bootstrap statistics. If the original sample statistic is finite and the interval contains zero, the constraint is marked compatible with zero, meaning we fail to reject the null constraint under the adopted rule.

For Algorithm 1 with the combined configuration: for each candidate pair, we compute the fourth-order triplet differences with m=1m=1 and m=2m=2 over the remaining p−2p-2 reference variables. The baseline significance level α\alpha in supplementary experiments is set to 0.20. Bonferroni correction is applied within each candidate pair for every TD interval, i.e., a=α/[2(p−2)]a = \alpha/\big[2(p-2)\big]. A candidate pair passes the same-cluster screening only if all these intervals contain zero.

For Algorithm 2 in the revised version, the fixed absolute threshold δ\delta originally used for ordering in the pseudocode is replaced by the interval-based criterion described above. The ordering step adopts fourth-order TD with m=1m=1 and the uncorrected baseline α\alpha. We separately count the number of triples compatible with zero for both causal directions, and the difference between the two counts determines the directional vote; no vote is cast when the counts are tied. After obtaining the preliminary ordering, covariance rank constraints are used for pruning if more than two latent variables are identified. The absolute standardized cross-covariance is used for the empty conditioning set, and the ratio of the largest to the smallest singular value is used for non-empty conditioning sets, both with a numerical tolerance of 0.05. We have explicitly stated the interval rule for ordering and the tolerance for pruning in the revised manuscript.

Reviewer_TDaJOpenReview ↗
Rating
2
Confidence
3
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

This paper focuses on latent causal variable discovery from linear non-Gaussian data. Compared with previous works, which construct proxy variables to perform independence tests and rely on complex kernel computations and hyperparameter settings, this work presents a high-order cumulant criterion that completely avoids independence tests and provides identifiability guarantees for the causal clusters of the observed variables and the causal order of the latent variables. The paper also clearly presents practical algorithms.

Critical strengths
  1. The paper is very well structured and easy to follow, from the problem and intuition to the motivation, proposal, and algorithms. The formulas are also very clear.
  2. Avoiding independence tests in latent causal variable discovery is an important goal, and this work achieves it completely.
  3. The synthetic results show that it performs among the best of the baselines.
Critical weaknesses
  1. Steps 14–25 of Algorithm 2 are not introduced in the paper and are mentioned only very briefly in lines 321–323.
  2. Although you compare with several baselines, including ETCD and RLTD, you do not compare with Triad, GIN, or TIN. Please discuss the theoretical and methodological differences from these methods in greater detail.
  3. Why is TIN not compared empirically, even though you discuss it in the motivation section?
  4. Table 1 seems to show that LaHiCaSl performs equally well or even better in the more challenging cases 5 and 6, so the results do not fully align with the discussion in lines 425–427. Please discuss the baselines’ better performance in more detail and address W2.
Questions for authors
  1. [Minor] What does p mean in line 79?
  2. What is line 219 supposed to say?
  3. What if Xi is generated by multiple Ls? According to Theorem 5.4, would you discover only one latent variable or cluster?
  4. Apparently, there is accumulated error in this two-stage algorithm, as shown in Table 2. Is it possible to provide some indicators from the results of Algorithm 2? Or do you have any thoughts?
  5. Regarding complexity, please report running-time comparisons across different settings and compare them with the baselines. Also, please report memory costs if possible.
11 comments
AuthorsOpenReview ↗

Response to Reviewer TDaJ (1/8) — Algorithm 2 and baseline methodology

Thank you for your thorough review and valuable suggestions. We respond to your comments below.

W1 — Reviewer comment

Steps 14–25 of Algorithm 2 are not introduced in the paper and are mentioned only very briefly in lines 321–323.

Response to W1

Steps 14–25 in Algorithm 2 serve to remove redundant edges from the candidate graph constructed based on the latent causal order. For a candidate edge La→LbL_a\to L_b, we delete it if there exists a subset Q⊆Pred(Lb)∖LaQ\subseteq\mathrm{Pred}(L_b)\setminus\\{L_a\\} such that La⊥Lb∣QL_a\perp L_b\mid Q.

This conditional independence is verified via the observational rank constraint from Silva et al. (2006). Specifically, we select one pure indicator for LaL_a and one for LbL_b, together with two distinct pure indicators for each latent variable in Q. These indicators are assigned to the two sides of the covariance matrix to form Ma,b∣Q=Cov⁡!((Xa,XQ(1)),(Xb,XQ(2))).M_{a,b\mid Q} =\operatorname{Cov}\\!\left( (X_a,X_Q^{(1)}),(X_b,X_Q^{(2)}) \right). Under the corresponding identifiability assumptions, the conditional independence holds if and only if rank⁡(Ma,b∣Q)≤∣Q∣\operatorname{rank}(M_{a,b\mid Q})\leq |Q|. The algorithm searches over subsets with increasing ∣Q∣|Q| and removes the candidate edge once such a valid set Q is found.

W2 — Reviewer comment

Although you compare with several baselines, including ETCD and RLTD, you do not compare with Triad, GIN, or TIN. Please discuss the theoretical and methodological differences from these methods in greater detail.

Response to W2

Tables 1 and 2 in the original submission already include the algorithm corresponding to Triad under its algorithm's name LSTC, and GIN is also listed. Discussions on the distinctions among these baselines are provided in Appendix D. Appendix D.1 introduces BPC, FOFC, RLCD and ETCD: BPC and FOFC identify measurement clusters via tetrad constraints, RLCD recovers latent structures using covariance rank information, and ETCD handles more intricate relationships between observed and latent variables under the pure-child assumption. Appendix D.2 examines the independence criteria adopted by Triad, GIN and their hierarchical extensions. Appendix D.3 discusses TIN and other higher-order cumulant-based methods. Table 7 in the appendix further compares the computational time and space complexity of all approaches.

The key distinction of TCD lies in the following: from an algebraic perspective, we leverage higher-order cumulants to eliminate the influence of latent variables on observed variables and construct the TD criterion. We identify pure clusters and latent causal orders from the zero-constraint patterns over different observed triples, followed by edge pruning using covariance rank constraints. Unlike Triad, GIN and LaHiCaSl, our proposed method avoids repeatedly constructing pseudo-residuals or linear combinations for iterative independence tests. Compared with BPC, FOFC and RLCD, which mainly rely on second-order covariance constraints, TCD additionally exploits non-Gaussian information encoded in higher-order cumulants to determine latent causal directions.

AuthorsOpenReview ↗

Response to Reviewer TDaJ (2/8) — TIN–GIN comparison

W3 — Reviewer comment

Why is TIN not compared empirically, even though you discuss it in the motivation section?

Response to W3

We discuss TIN in the motivation section because it demonstrates how to extract causal structural information from data with measurement errors via linear transformations and independence tests. The main focus of Dai et al. (2022) is ordered group recovery under the single-indicator measurement error model. In contrast, this work aims to identify unknown measurement clusters from observed data and further recover causal relationships among latent variables. Due to these differences in task objectives and measurement assumptions, TIN was not included in the empirical comparisons of our initial submission.

Following the reviewer's suggestion, since the original authors have not released the code for latent causal structure learning, we reproduce the structure-learning pipeline for the two-pure-indicator model described in Appendix B of Dai et al. We denote this reproduced baseline as TIN–GIN (Appendix B) and compare it with TCD on identical datasets using the same evaluation metrics.

The experimental results are presented in the table below. When comparing TIN–GIN against TCD under the same experimental settings in Table 2 of the main paper, TCD achieves a higher F1 score for orientation recovery across all 18 configurations, for both estimated clusters and ground-truth clusters as input. This observation indicates that TCD delivers better orientation recovery performance for the end-to-end task of learning latent causal structures starting from unknown measurement clusters.

TIN–GIN (Appendix B) under cubic Gumbel noise

LO = latent omission, LC = latent commission, and MM = mismeasurement. The two F1 columns use estimated and ground-truth clusters, respectively.

MetricCase 1Case 2Case 3
Sample size (N)500100020005001000200050010002000
Latent omission ↓0.400.650.700.400.350.550.530.470.60
Latent commission ↓0.000.000.000.000.000.000.000.000.00
Mismeasurement ↓0.000.000.000.000.000.000.000.000.00
F1 (estimated clusters) ↑0.200.100.000.400.300.100.130.070.20
F1 (true clusters) ↑0.600.300.300.700.500.200.260.240.13
MetricCase 4Case 5Case 6
Sample size (N)500100020005001000200050010002000
Latent omission ↓0.030.230.130.500.650.650.000.000.00
Latent commission ↓0.130.070.030.000.000.000.000.000.00
Mismeasurement ↓0.000.000.000.000.000.000.000.000.00
F1 (estimated clusters) ↑0.340.130.300.000.000.000.170.270.07
F1 (true clusters) ↑0.360.070.290.600.500.300.460.480.18
AuthorsOpenReview ↗

Response to Reviewer TDaJ (3/8) — LaHiCaSl and manuscript clarifications

W4 — Reviewer comment

Table 1 seems to show that LaHiCaSl performs equally well or even better in the more challenging cases 5 and 6, so the results do not fully align with the discussion in lines 425–427. Please discuss the baselines' better performance in more detail and address W2.

Response to W4

Latent omission (LO), latent commission (LC) and measurement misclassification (MM) must be interpreted jointly. A low LO only indicates more true latent variables are matched, while the clusters may still contain misassigned or spurious indicators.

Taking cubic-Gumbel results in Table 1: for Case 5 (N=1000N=1000), LaHiCaSl has better LO and MM but higher LC (0.15 vs. 0.00). For Case 6 (N=1000N=1000), LaHiCaSl achieves lower LO (0.07 vs. 0.17) yet higher MM (0.03 vs. 0.00). To understand how clustering errors degrade orientation recovery, we inspected LaHiCaSl runs: indicators from distinct latent parents were mixed in one cluster across 13 replicates, and duplicate indicator assignment occurred once.

  • Case 5 violates Assumption 2.1: in 7 out of 30 replicates, X1X_1 and X4X_4 from separate true clusters were incorrectly grouped; they were discarded in the other 23 runs. In the 4th replicate (N=1000N=1000), estimated clusters produced a wrong edge, whereas ground-truth clusters recovered the correct orientation.
  • Case 6 satisfies Assumptions 2.1 and 2.2: in the 6th replicate (N=1000N=1000), LaHiCaSl merged two true clusters X1,X2\\{X_1,X_2\\} and X3,X4\\{X_3,X_4\\}. No edges were returned with estimated clusters; the collider structure was recovered given ground-truth clusters.
  • Case 3 shows duplicate assignment: in the 3rd replicate (N=2000N=2000), both X1,X2,X3\\{X_1,X_2,X_3\\} and X1,X2,X3,X4\\{X_1,X_2,X_3,X_4\\} were output. No edges were learned from this clustering, while the two true edges were recovered using ground-truth clusters.

In the terms of F1 score, TCD yields an average F1 of 0.723 over all 18 settings in Table 2, versus 0.583 for LaHiCaSl. For Case 6 (N=2000N=2000), TCD still outperforms LaHiCaSl with estimated clusters despite LaHiCaSl's lower LO. For Case 5 (N=1000N=1000), their orientation F1 scores are 0.50 and 0.00, respectively.

We have revised Lines 425–427 to fairly acknowledge LaHiCaSl's strengths on several cluster metrics. Combined with the structural assumptions in W2, we explain that a smaller LO does not guarantee better end-to-end orientation recovery.

Q1 — Reviewer comment

[Minor] What does p mean in line 79?

Response to Q1

pp denotes the number of observed variables.

Q2 — Reviewer comment

What is line 219 supposed to say?

Response to Q2

Line 219 repeated the name of the proposed algorithm. We removed the redundant line in the revised manuscript.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.