acceptodds
Under review as a conference paper at ICLR 2027

The Blessing of Linear Non-Gaussian Models: Powerful Independent Subspace Inference for Latent-Variable Causal Discovery

Abstract

The linear non-Gaussian model is a classical parametric model for causal discovery, known for its strong identifiability. A key ingredient underlying such identifiability is independence induced by linear transformations. For example, given two random vectors and , one may seek a transformation such that , where may arise from linear regression or more general constructions. In the presence of latent confounders, two established tools, generalized independent noise (GIN) and transformed independent noise (TIN), exploit such constraints to characterize independent linear transformations. Despite their success in causal identification, existing implementations do not fully exploit the statistical structure of linear non-Gaussian models, which may compromise the reliability of causal identification. To address this issue, building on the characterization of independence through constant conditional means and variances, we construct a fixed linear operator whose kernel exactly characterizes all independent directions, enabling unified estimation and testing using only low-order information. We develop a paired-bootstrap GIN test with pointwise asymptotic Type I error control and a jointly calibrated TIN rank estimator. Synthetic and real-world experiments across several settings show gains in GIN testing power, TIN dimension recovery, and downstream latent-variable causal discovery.

Primary area:
causal reasoning
Keywords:
Linear Non-Gaussian Models, Independent Subspace Inference, Causal Discovery

Reviews

Among:
Shape:
1234this paper, Top 1%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.67 over 3 reviews, above 99% of the others.
Reviewer_KoUUOpenReview ↗
Rating
3
Confidence
2
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

This paper addresses the problem of identifying independent subspaces within linear non-Gaussian models. The core approach involves identifying all projection directions that satisfy independence conditions by examining the null space of a specific linear operator; based on this, the authors develop the GIN test and a method for estimating the dimension (TIN). The paper provides theoretical results and validates the proposed methods using both simulated and real-world data.

Critical strengths
  1. Characterizing all independent directions via the null space—rather than searching for them one by one—is an intriguing approach. The identification results, derived under finite-feature assumptions, also enhance the practical utility of this method.

  2. The testing procedure accounts for errors introduced during the estimation of projection directions, which is a noteworthy detail. The experiments also include useful ablation analyses.

Critical weaknesses

I have concerns regarding the choice of baselines in the multidimensional GIN experiments. The proposed method utilizes the entire null space of the covariance operator, whereas HSIC and LiNGIC are applied using only a single direction. This discrepancy may exaggerate the relative performance of the proposed method. While ablation studies are provided, they do not fully clarify the specific source of the observed performance gains.

Questions for authors
  1. If HSIC and LiNGIC were applied using the same directional space—while controlling for the Type I error rate—to what extent would the proposed method still demonstrate an advantage?

  2. In Appendix G, only 499 bootstrap iterations are performed, resulting in a minimum p-value of 0.002. Given this, how should the clustering results on the real-world data be interpreted when using a significance level of α=0.001\alpha = 0.001? Furthermore, would the conclusions change if the number of bootstrap iterations were increased?

3 comments

We thank the reviewer for careful reading and suggesting these comparisons.


W1&Q1: We added three baselines based on the estimated covariance-null space. Let Y=(Y1,…,Yd)Y=(Y_1,\ldots,Y_d) denote the projections of XX onto a basis of that space.

  • Full-subspace HSIC (FS-HSIC) tests all dd basis projections of the fitted covariance-null space jointly.
  • Full-subspace LiNGIC (FS-LiNGIC) tests, for each reference variable, the conditional means of all YjY_j and all products YjYkY_jY_k (j≤kj\leq k), which together determine the first two conditional moments along every direction in the fitted subspace.
  • Random-projection HSIC (RP-HSIC) samples 50 directions from the same fitted subspace.

All the tests used the same variance-normalized basis fitted to the test data and 499 draws per method. Our bootstrap refitted the basis in each paired resample. The baselines used random-sign multipliers with a first-order correction for basis estimation, holding rank and bandwidths fixed. FS-HSIC tested YY against the full vector ZZ. RP-HSIC shared signs across 50 projections to calibrate the maximum standardized HSIC. FS-LiNGIC corrected both the Gaussian and quadratic kernel terms in YY and shared signs across ZjZ_j to calibrate Fisher’s combination of rank-based pp-values. For each method and condition, we used 999 null datasets to select a cutoff targeting a 5% Type I error rate. We then evaluated all methods on the same 1,000 held-out null datasets and 500 alternative datasets. Each entry reports power at θ=0.8\theta=0.8, with the held-out Type I error rate in parentheses. Both are percentages averaged over the five source distributions.

SettingSample size nnFS-HSICRP-HSICFS-LiNGICOurs
I20027.68 (5.18)27.68 (5.18)28.44 (4.50)60.76 (4.84)
I50049.64 (5.46)49.64 (5.46)67.44 (4.92)88.92 (4.72)
I80063.68 (4.70)63.68 (4.70)85.64 (5.58)96.20 (4.74)
I100071.88 (5.32)71.88 (5.32)90.84 (4.74)98.20 (4.42)
IMean53.22 (5.17)53.22 (5.17)68.09 (4.94)86.02 (4.68)
II20017.40 (4.74)16.04 (5.40)11.20 (5.38)40.40 (5.06)
II50031.88 (4.34)29.68 (4.94)29.56 (4.38)77.04 (5.20)
II80043.08 (4.64)40.16 (4.82)50.96 (5.52)89.52 (5.14)
II100049.24 (4.60)47.16 (4.88)56.72 (4.46)93.24 (5.22)
IIMean35.40 (4.58)33.26 (5.01)37.11 (4.94)75.05 (5.16)
III20015.12 (4.60)13.60 (4.58)10.68 (5.22)51.32 (4.94)
III50032.84 (5.02)33.28 (4.90)27.56 (4.56)83.36 (4.88)
III80042.40 (5.54)43.88 (5.62)44.36 (4.94)93.80 (5.38)
III100046.64 (5.12)51.28 (5.70)50.92 (4.64)96.36 (4.94)
IIIMean34.25 (5.07)35.51 (5.20)33.38 (4.84)81.21 (5.04)

In Setting II, our test exceeded the strongest baseline, FS-LiNGIC, by 37.94 percentage points in mean power (75.05% vs. 37.11%), with held-out Type I error rates of 5.16% and 4.94%, respectively. In Setting III, the gain over the strongest baseline, RP-HSIC, was 45.70 percentage points (81.21% vs. 35.51%), with corresponding Type I error rates of 5.04% and 5.20%.


Q2: Thank you for raising this point. The clustering search retains a pair of indicators when it passes the rank checks and its GIN test does not reject. At α=0.001\alpha=0.001, all three pairs in the speed group and all three in the error group passed the rank checks. All six GIN queries had nonempty candidate spaces, so with 499 bootstrap draws their smallest attainable pp-value was 0.0020.002. Our procedure therefore retained all six pairs and recovered both groups (F1=0.80F_1=0.80). Using Fisher-combined pp-values, the HSIC baseline rejected two of the three error-group queries and the LiNGIC baseline rejected all three. Both recovered only the speed group (F1=0.50F_1=0.50). The higher clustering F1F_1 reflects recovery of the additional reference group and cannot by itself establish greater GIN rejection power.

We then increased the budget to 49,999 paired-bootstrap draws per query, lowering the minimum attainable pp-value to 0.000020.00002 and allowing rejection at α=0.001\alpha=0.001. All 20 bootstrap runs on the same dataset again recovered both groups (F1=0.80F_1=0.80).

Thank you for the detailed response and additional experiments. I have no further questions.

Thank you for taking the time to review our response and the additional experiments. We are glad we could address your questions.

Reviewer_mriuOpenReview ↗
Rating
4
Confidence
3
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

Can we just not use off-the-shelf CI tests for causal discovery? Apparently so for the LiNGAM model.

Critical strengths

Nobody enjoys using (conditional) independence tests. The recent result by Li et al. 2026 pose to make our lives easier going forward.

Specifically about this paper's efforts: Direct RHKS linear operator construction and finite eval surrogate correctness results are clean and look correct. The uniform treatment for recovering two existing methods is nice.

Critical weaknesses

None.

Questions for authors

None.

2 comments

We thank the reviewer for the thoughtful and encouraging assessment and for highlighting how our work builds on LiNGIC. We particularly appreciate the comments on the operator construction and the finite-evaluation result. The operator characterizes the entire independent transformation subspace as its kernel, and the finite-evaluation result establishes when a finite matrix has the same kernel at the population level. A central aim of the paper was to show how this single characterization provides a common foundation for GIN testing and TIN dimension estimation, and we are glad that the reviewer highlighted this connection.

Thank you. I do not have any outstanding concerns.

Reviewer_dScpOpenReview ↗
Rating
4(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies estimating and testing independent linear subspaces in linear non-Gaussian models. The main contribution is a full characterization of the independent transformation subspace as the null space of a linear moment operator, together with conditions under which finitely many kernel evaluations preserve this characterization. The authors develop GIN and TIN inference procedures that account for estimated projection directions.

Critical strengths

The exact linear-operator characterization recovers the entire independent subspace without searching over projections.

Critical weaknesses

The largest improvements are over single-direction baselines that may miss dependence. It is unclear whether the improvements persist against randomized-direction or full-subspace tests.

Questions for authors
  • Do the improvements persist against randomized-direction and full-subspace HSIC tests at matched Type I error?
  • Do the power improvments persist in larger dimensions, particularly when dependence is confined to a few directions? The current experiments do not establish whether retaining all mixed moments remains effective.
4 comments

We thank the reviewer for highlighting our characterization of the independent transformation subspace and suggesting these comparisons.


W1&Q1: We added three baselines. Let Y=(Y1,…,Yd)Y=(Y_1, \ldots, Y_d) denote the projections of XX onto a basis of the estimated covariance-null space. We introduce:

  • Full-subspace HSIC (FS-HSIC) tests all dd basis projections of the fitted covariance-null space jointly.
  • Full-subspace LiNGIC (FS-LiNGIC) tests, for each reference variable, the conditional means of all YjY_j and all products YjYkY_jY_k (j≤kj\leq k), which together determine the first two conditional moments along every direction in the fitted subspace.
  • Random-projection HSIC (RP-HSIC) samples 50 directions from the same fitted subspace.

For multivariate ZZ, all three baselines combine pp-values across its coordinates using Fisher’s method. RP-HSIC does this within each direction before applying a Bonferroni correction across directions. The following tables report mean power (%) across five source distributions at θ=0.8\theta=0.8 and a matched empirical Type I error rate of 5%.

SettingSample size nnHSICLiNGICFS-HSICRP-HSICFS-LiNGICOurs
I20029.445.029.425.945.060.7
I50052.077.452.040.377.487.7
I80068.890.868.856.590.896.3
I100072.295.572.260.395.599.0
IMean55.677.255.645.777.285.9
II2005.45.217.817.117.241.3
II5005.03.437.034.342.278.2
II8004.45.457.048.965.590.6
II10005.45.657.254.868.091.9
IIMean5.14.942.338.848.275.5
III2006.07.024.019.621.351.1
III5004.04.442.438.639.483.8
III8004.03.954.446.852.194.6
III10006.25.661.854.461.098.1
IIIMean5.15.245.739.943.481.9

Across Settings I–III, mean power was 81.1% for our test and 56.3% for FS-LiNGIC, the strongest baseline on average. Broader coverage helps the baselines detect dependence that a single projection can miss. The remaining advantage at full coverage points to a benefit of our mixed-moment statistic, which pairs each candidate projection with all observed coordinates of XX.


Q2: We introduce Setting IV that extends Setting II (d=2d=2) by adding independent noise coordinates to XX. We vary dd over {2,5,10,20}, with fixed sample size n=1000n=1000.

ddHSICLiNGICFS-HSICRP-HSICFS-LiNGICOurs
25.45.657.254.868.091.9
55.04.046.642.039.086.2
106.63.825.624.626.272.3
204.04.216.211.616.845.9
Mean5.34.436.433.337.574.1

Our mean power fell from 91.9% at d=2d=2 to 45.9% at d=20d=20, where it still exceeded that of the strongest baseline (FS-LiNGIC) by 29.1 percentage points. Even as additional noise directions made the single dependent contrast harder to detect at a fixed sample size, our test retained the highest mean power at every dimension studied.

Furthermore, for each candidate direction ww, we compared the complete second-order response X(w⊤X)X(w^\top X) with its projection onto the fitted candidate space. Both variants used the same kernel features and first-order response. Each tested all candidate directions and refitted them in every paired-bootstrap draw. At θ=0.8\theta=0.8 in Setting III (d=3d=3, n=500n=500, uniform sources, 3.2% target level), power rose from 82.3% with the within-space response to 98.8% with the complete response (Appendix F, Table 6). In a separate experiment reported here at the same signal strength (d=20d=20, n=1000n=1000, 5% target level), the corresponding increase in mean power was from 16.5% to 45.2% across uniform and Laplace sources. These results show that even in the largest candidate space tested, retaining mixed moments involving the component of XX orthogonal to that space helped reveal the single dependent contrast.

Dear Reviewer dScp,

Thank you again for your constructive feedback. We have responded to your questions in our rebuttal and updated the manuscript with additional clarifications and experimental results. We hope these results will be helpful in your final assessment.

As the discussion period draws to a close, we would be grateful if you could review our responses and let us know whether they address your concerns. We would be happy to clarify any remaining points.

Thank you again for your time and thoughtful consideration.

Reviewer_dScpOpenReview ↗

Official comment

Thank you for the additional experiments. I have no additional questions.

Thank you for reviewing the additional experiments and for your thoughtful feedback throughout the process. Your questions about the baselines and sparse dependence in higher dimensions helped us strengthen our empirical evaluation.

Best regards,
The Authors

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary of Revisions

We sincerely thank the area chair for coordinating the review and all three reviewers for their careful reading and constructive feedback. In response, we have revised the manuscript and expanded our empirical evaluation in four main points (changes to the manuscript are shown in blue):

  • Full covariance-null space baseline. We added full-subspace versions of HSIC and LiNGIC, plus HSIC over 50 random projection directions in the same fitted space. At θ=0.8\theta=0.8 and a matched 5% empirical false-positive rate, mean power in the two- and three-dimensional settings was 78.7% for our test and 45.8% for the strongest baseline, full-subspace LiNGIC (Section 6.1 and Appendix E.1, Table 2).
  • Sparse dependence in higher dimensions. The new Setting IV embeds one dependent direction in spaces of dimension 2 to 20 (Section 6.1, Figure 4). A separate ablation tests the contribution of mixed moments using the same fitted directions for both variants (Appendix F, Table 6).
  • Latent-variable causal discovery. We expanded the comparison from three to six generalized independent noise (GIN) tests and evaluated recovery of latent edges and clusters at α=0.01\alpha=0.01 (Section 6.2, Table 1, and Appendix E.3).
  • Real-data clustering. We increased the bootstrap budget so the test could reject at α=0.001\alpha=0.001. All 20 bootstrap runs on the same data recovered the speed and error groups (F1 = 0.80). (Appendix G).

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.