acceptodds
Under review as a conference paper at ICLR 2027

Spectral Deconfounding for Causal Inference from High-dimensional Perturbed Model

Abstract

High-dimensional causal inference from observational studies is challenging due to unobserved confounders. To ensure identifiability, previous works assume the dense confounding condition, which means that each latent confounder is correlated with most covariates. This dense confounding introduces strong correlations among covariates, leading to large singular values along certain directions of the design matrix. To remove the confounding effect, spectral deconfounding methods apply spectral transformations to the design matrix, attenuating the dominant singular components caused by latent confounders. In sparse linear regression, it has been established that such deconfounding can improve the estimation for Lasso. However, a fundamental question remains open: Can spectral deconfounding identify which features are causally relevant to the response? This is known as the model selection problem and is important in many applications like scientific discovery. In this paper, we show that by weakening confounding-induced spurious correlations, spectral deconfounding enables the irrepresentable condition holds asymptotically, therefore enabling Lasso to identify and optimize within the causal features set. This perspective in turn motivates us to reduce the bias produced by the penalty, and thus improving the estimation. Specifically, we employ the Linearized Bregman Iteration, which was proposed as an optimization algorithm and was recently found to enjoy the unbiased property from a differential-inclusion perspective in the fixed design setting. We extend this property to random design setting with latent confounders, and show that, our estimator achieves the oracle estimation rate under the beta-min condition, matching the minimax-optimal rate for unconfounded setting with fixed design up to a logarithmic factor.

Primary area:
causal reasoning
Keywords:
causal inference, spectral deconfounding, model selection

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_wMhaOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This work investigates the deconfounding problem in high-dimensional causal inference and proposes Spec-LBI, which integrates Trim-based spectral deconfounding with linearized Bregman iteration (LBI). Theoretically, it establishes that the Trim transformation can asymptotically restore the irrepresentable condition, extending LBI's selection and estimation consistency guarantees to random-design settings with latent confounding. The method is evaluated through simulations and two real-world applications.

Critical strengths
  1. The paper addresses the spectral confounding problem and proves that Trim-based spectral preprocessing can make the irrepresentable (IRR) condition hold asymptotically.
  2. The paper extends the existing statistical theory of linearized Bregman iteration (LBI). While the theoretical guarantees of the original LBI are established under fixed-design assumptions without latent confounding, this work extends feature selection consistency and ℓ2\ell_2-estimation consistency to random-design settings with latent confounding, establishing a convergence rate for the proposed Spec-LBI estimator.
Critical weaknesses
  1. In Section 4.1, the intuitive statement “Intuitively, the IRR condition requires that causal features are weakly correlated with others” is conceptually problematic. The irrepresentable (IRR) condition is purely a design‑matrix‑level assumption for sparse regression, where the set S denotes the active set of non‑zero regression coefficients rather than true causal features. By directly equating regression‑level active features to causal features in this intuitive explanation, the paper implicitly assumes that non‑zero regression coefficients correspond exactly to true causal variables, which conflates correlational regression quantities with causal quantities without explicit justification.
  2. Spec‑LBI acts solely as an estimation solver and provides no causal identification guarantees. However, the manuscript does not clearly articulate under which exact set of assumptions causal quantities become identifiable within the proposed pipeline. This exposition creates potential risk of misinterpretation, where readers may mistakenly treat Spec‑LBI as a general‑purpose tool for discovering causal features directly from confounded observational data, rather than an estimator operating conditional on pre‑established causal identifiability.
Questions for authors
  1. As an iterative algorithm, the proposed method lacks sufficient algorithmic discussion in the method section. The manuscript does not clarify whether the iterative sequence is guaranteed to converge, what fixed‑point solution it converges to, practical rules for stopping criteria such as coefficient change threshold or maximum iteration number, whether multiple local solutions exist, and whether initialization choices can alter the final selected feature set. These practical implementation details are critical for reproducibility and real‑world deployment, yet they remain underexamined.

  2. When deployed for causal backdoor adjustment, one needs to select confounders satisfying the backdoor criterion, instead of merely selecting variables predictive of outcome Y. Spec‑LBI and Lasso are optimised for predictive performance. Predictive variables do not necessarily qualify as valid confounders; they could be colliders (descendants of the outcome variable Y). Without additional constraints, Spec‑LBI may select such collider variables into the adjustment set and introduce collider bias that invalidates causal estimation. This important risk of using prediction‑oriented sparse regression for confounder selection is not acknowledged in the method section.

  3. The presented theory hinges on multiple stringent asymptotic assumptions, including the beta‑min condition, prescribed sparsity level of β∗\beta^*, and specific scaling requirements for the Trim transformation parameters. None of these assumptions can be empirically verified given real‑world observational datasets. The manuscript lacks systematic discussion regarding performance degradation when these assumptions are violated in practice.

  4. The practical performance of Spec‑LBI heavily depends on critical hyper‑parameters: damping factor κ\kappa, step‑size, and early‑stopping time τ\tau. While cross‑validation is suggested as a practical recipe for hyper‑parameter tuning, the paper provides no sensitivity analysis for these hyper‑parameters. Consequently, it remains unclear how hyper‑parameter misspecification in finite samples would harm feature‑selection consistency and estimation error.

  5. This work falls into the family of spectral‑deconfounding methods built on the dense‑confounding factor‑model framework. Although it cites prior Trim‑based spectral deconfounding literature, the paper lacks systematic conceptual positioning and empirical benchmarking against closely‑related competing approaches for high‑dimensional latent confounding causal inference.

5 comments
AuthorsOpenReview ↗

Causal interpretation and identification

We thank the reviewer for the helpful feedback and thoughtful questions. We address the questions below in three parts, including causal interpretation and identification, contributions, and others.

1.Causal interpretation and identification

We sincerely thank the reviewer for raising these concerns. Here, our main claim is that the identifiability relies on the widely used dense-confounding assumption, and our causal interpretation is based on the structural equation model (SEM) formulation of model (1) in Line 138 (Line 144 in the revised version). Below, we address the related concerns.

Weakness 1 :About the causal interpretation of non‑zero regression coefficients and IRR condition.

Response: We thank the reviewer for pointing out this concern. We would like to claim that our causal interpretation of the nonzero regression coefficients follows from the structural equation model (SEM) formulation of model (1), as discussed in the spectral deconfounding literature [1, 5]. Specifically, under this SEM formulation of the linear model, it can be easily shown that each nonzero coefficient in βi∗\beta^*_i indeed represents the causal effect of XiX_i on the response YY. That means, under the SEM formulation of model (1), those active features with nonzero regression coefficients are exactly causal features. Consequently, the active features with nonzero regression coefficients correspond precisely to the causal features. In this sense, the irrepresentable (IRR) condition is imposed on the design matrix with respect to the set of causal features. We have incorporated these explainations into the revised manuscript (Lines 140–159 and 255–256).

Weakness 2 :About the causal identification guarantees.

Response: We thank the reviewer for pointing this out. We would like to clarify that our estimation result relies on the dense confounding assumption, as we have already stated on lines 160-172 ( lines 168-179 in the revised version). The dense confounding assumption has been widely adopted to ensure the identifiability of the causal effect [1,5,6,7]. On the basis of this assumption, we show that Spec-LBI can mitigate confounding bias while reducing the shrinkage bias relative to existing spectral deconfounding methods, and even achieve the oracle minimax estimation rate.

Questions 2: About the backdoor adjustment.

Response: We thank the reviewer for raising this concern. We would like to clarify that Spec-LBI actually does not use backdoor adjustment for causal estimation under the structural equation model (SEM) formulation of model (1). Instead, our debiasing approach builds on the dense confounding assumption, under which latent confounders induce a few spiked singular values in the covariate matrix. Under this assumption, spectral deconfunding shrinks these singular values and consequently removes the confound effect.

AuthorsOpenReview ↗

Contributions

2.Contributions

We thank the reviewer for raising these concerns. Here, our main claim is that both our experimental design and algorithm are aligned with our two main contributions built upon the dense-confounding setting: (1) explaining theoretically how spectral deconfounding improves variable selection and (2) building on this insight to propose Spec-LBI for bias reduction and more accurate estimation. Below, we address the related concerns.

Questions 5: About contribution and related work.

Response: We thank the reviewer for raising this concern. We would like to clarify that the comparisons conducted in our experiments are sufficient to support our claims and contributions. Specifically, as stated in Lines 65 and 76 of the manuscript, our work has two main contributions. First, we provide a theoretical understanding of why spectral deconfounding can improve estimation. Second, we propose the Spec-LBI estimator as a bias-reduction approach to Lasso, aiming to substantially reduce estimation bias and improve estimation accuracy.

Regarding the first contribution, our goal is to understand the underlying mechanism of spectral deconfounding rather than to propose a new method. Specifically, we theoretically establish that spectral deconfounding can drive the irrepresentable (IRR) condition toward zero, and further validate this theoretical finding through numerical experiments. Regarding the second contribution, we demonstrate that the proposed Spec-LBI estimator can substantially reduce the bias of Lasso and thereby improve estimation precision.

Questions 1: About the the coverage of Spec-LBI iterations.

Response: We thank the reviewers for raising this question. We would like to clarify that the Spec-LBI iterations are used to select variables by early stopping, rather than to solve the associated optimization problem to convergence. The key is therefore to choose an appropriate early stopping time. We have provided the theoretical stopping time in lines 1207–1211 (1261–1265 in the revised version), while in practice we use cross-validation following Osher et al. (2016)[8], as described in lines 319–321 ( lines 323–329 in the revised version). For initialization, we follow Osher et al. (2016)[8] and set ρ0=β0=0\rho_0=\beta_0=0, as specified in lines 188–191 (196–199 in the revised version). Furthermore, LBI indeed converges to the unique solution of its associated constrained optimization problem established in [9].

3. Others

Here, we discuss some commonly used assumptions underlying our theoretical results and Spec-LBI’s sensitivity to hyperparameter choices.

Questions 3: About asymptotic assumptions in real world.

Response: We thank the reviewer for raising this concern. We would like to emphasize that the beta-min condition and the sparsity formulation are standard and foundamental in the high-dimension statistics [2-4]. In particular, the sparsity constraint is to ensure that the true coefficient is identifiable (even without the latent confounder) in high dimensional setting. For the beta-min condition, even without this condition, we can also achieve the ℓ2\ell_2 convergence rate of slogp/n.\sqrt{slog p/n}.

For the trim parameter mm, Theorem 4.1 remains valid over a broad range of values of the scaling parameter. Roughly speaking, mm can be chosen flexibly above the number of latent factors, as long as it satisfies m=o(n1−ϵ/s)m=o(n^{1-\epsilon}/s) . For example, for fixed ss,we can choose m≍n0.7m\asymp n^{0.7}.

Furthermore, we conducted additional experiments to demonstrate the robustness of our method when these assumptions are violated. For the sparsity constraint, we vary the sparsity level. Figures 10 (b)-(c) show that the ℓ2\ell_2-error scales approximately as slog⁡s\sqrt{s\log s} with the sparsity level ss, while remaining nearly unchanged as the trim parameter mm increases.

For beta-min, we vary the coefficient magnitude δ\delta from 11 to 0.050.05 and report the estimation error. As shown, our method achieves a lower ℓ2\ell_2-error than all baselines, even when the beta-min condition is violated (δ=0.05\delta=0.05).

Methodδℓ₂ error ↓
Lasso1.002.034
Trim+Lasso1.000.590
Ours1.000.149
Ours0.500.177
Ours0.200.312
Ours0.050.137

Questions 4: About hyperparameters.

Response: We thank the reviewer for this question. We would like to clarify that we have already discussed the hyperparameters and how their values affect the model in lines 315–321 (lines 323–329 in the revised version). To verify these claims empirically, we have also conducted additional experiments. Let SS and S^\widehat S denote the true and estimated supports, respectively. We report the F1 score, FP=∣S^∖S∣\mathrm{FP}=|\widehat S\setminus S|, and FN=∣S∖S^∣\mathrm{FN}=|S\setminus\widehat S| for support recovery and the ℓ2\ell_2-error for estimation accuracy, all averaged over 20 repetitions:

The damping parameter κ\kappa, the step size α\alpha: Since these parameters must jointly satisfy the κα≤1/∣∣X~∗X~∣∣2\kappa\alpha \leq 1/||\tilde{X}^{*}\tilde{X}||_2 condition , we evaluate their effects jointly. To show these effects more clearly, we use a larger support size of s=15s=15. The results show that a larger κ\kappa increases the path time but can reduce the ℓ2\ell_2 estimation error.

κ\kappaStep size α\alphaF1 ↑FP ↓FN ↓ℓ₂ error ↓Path time (s) ↓
21/(2∥X~∗X~∥2)1/(2\lVert\tilde{X}^{\ast}\tilde{X}\rVert_2)0.64±0.1218.60±9.380.00±0.000.57±0.140.05±0.01
101/(10∥X~∗X~∥2)1/(10\lVert\tilde{X}^{\ast}\tilde{X}\rVert_2)0.95±0.041.55±1.320.00±0.000.31±0.060.21±0.04
201/(20∥X~∗X~∥2)1/(20\lVert\tilde{X}^{\ast}\tilde{X}\rVert_2)0.96±0.031.15±1.090.00±0.000.30±0.060.42±0.06
501/(50∥X~∗X~∥2)1/(50\lVert\tilde{X}^{\ast}\tilde{X}\rVert_2)0.98±0.020.65±0.670.00±0.000.29±0.061.02±0.14

The stopping time: We vary the stopping time relative to the cross-validated choice tCVt_{CV} to assess sensitivity. The results show that stopping too early misses true variables and increases the estimation error.

Stopping timeF1 ↑FP ↓FN ↓ℓ₂ error ↓
tCVt_{\mathrm{CV}}1.00±0.000.00±0.000.00±0.000.15±0.04
0.50tCV0.50t_{\mathrm{CV}}0.19±0.340.00±0.004.20±1.581.95±0.64
2.00tCV2.00t_{\mathrm{CV}}0.95±0.130.85±2.130.00±0.000.19±0.12
Reviewer_5TZaOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies variable selection and effect estimation in high-dimensional linear models with latent confounding. It shows that Trim can satisfy the irrepresentable condition asymptotically under certain assumptions. It then combines Trim with Linearized Bregman Iteration to reduce confounding and shrinkage bias. The paper proves consistency for variable selection and effect estimation. It also tests the method on synthetic data, LLM-generated text, and gene expression data.

Critical strengths
  1. The paper asks whether spectral deconfounding can identify causally relevant variables, rather than just estimate their effects. This is a useful question with practical value.

  2. The paper studies the irrepresentable condition after Trim. It also extends LBI theory to random designs with latent confounding. Both are useful theoretical contributions.

  3. The experiments test variable selection and estimation error and include two applications. They also test the method under heavy-tailed distributions, correlated noise, and partial confounding.

Critical weaknesses
  1. Linear models are common in causal inference, but their use still needs to be justified for the specific application. Why is the linear additive model in Equation (1) a reasonable choice for the text and gene expression data? It is also unclear to me whether variable selection and effect estimation remain reliable when this assumption does not hold.

  2. It is unclear to me whether the stopping time chosen by cross-validation preserves the theoretical guarantees for variable selection. AUC measures the order in which variables enter the path, so it does not show whether the final selected set is correct.

  3. The partial-confounding experiments still link the confounder to most variables. This leaves me uncertain about how the method performs when confounding affects only a few variables and does not create strong singular directions. Such settings would help show how much the method depends on the key confounding assumption.

Questions for authors
  1. What supports the linear additive assumption in the two applications? How does the method perform when the data include nonlinear effects or interactions?

  2. Is there any guarantee for the stopping time chosen by cross-validation? At this stopping time, how many irrelevant variables are selected, and how many relevant variables are missed?

  3. What happens when confounding affects only a few variables and the loadings are too weak to create strong singular directions? Can the method still recover relevant variables in these cases?

4 comments
AuthorsOpenReview ↗

Evaluation Metrics

We thank the reviewer for the thoughtful feedback and helpful questions. We organize our responses into two parts: evaluation metrics and research setting.

1. Evaluation metrics

Weakness 2 & Questions 2:About the guarantee for cross-validated stopping and AUC evaluation.

Response: We thank the reviewer for this insightful comment. Cross-validation is indeed the empirical selection method we adopt following [8]. Since the theoretical stopping time depends on the unknown parameter η\eta in Eq. (52), and thus there is no theoretical guarantee for the empirically selected stopping time. For this reason, we can only use AUC to evaluate feature selection along the solution path rather than only focusing on feature selection at the stopping time.

Nevertheless, we also conduct additional experiments to directly evaluate feature selection at the cross-validation stopping time. As reported in our response to Weakness 3, Table 1 (row 3, “baseline; Ours”) shows an F1 score of 1, with no false positives (FP =0=0) or false negatives (FN =0=0), for the simulation experiments in Section 5. At this empirically selected stopping time, our method also recovers all causally relevant features without selecting any irrelevant variables.

AuthorsOpenReview ↗

2. Research setting

2.Research setting

We sincerely thank the reviewer for these questions. Our main claim here is that our contributions are built upon the widely used dense-confounding setting. Below, we address the concerns about our setting.

Weakness 3 & Questions 3:About the weak and sparse confounding.

Response: We appreciate the reviewer's comment. We address this concern about our assumption from two perspectives below.

First, we would like to point out that dense confounding is a common and widely used assumption in the literature. It has been adopted not only by Ćevid et al. (2020) [9], but also by Guo et al. (2022) [10], Sun et al. (2024) [11], and Wang and Shah (2025) [12]. Such structural assumptions are necessary because causal effects are generally unidentifiable under unobserved confounding without additional assumptions. Moreover, an advantage of this assumption is that it does not require additional auxiliary variables, which are difficult to obtain in high-dimensional settings. Furthermore, this assumption is natural in a lot of real-world applications. For example, a few genes may influence most other genes [4–5], while writing style can affect an entire text [13]. These are important problems, and our work is built upon this setting. Our main contribution is to analyze how spectral deconfounding improves estimation through causal feature selection under this assumption (line 65), and not to relax the dense-confounding assumption.

Second, the dense-confounding assumption can still hold when loadings are either weak and dense or strong and sparse. However, when loadings are both sparse and weak, the assumption may no longer hold. This is because dense confounding asssumption only requires λmin⁡(Ψ)=Ω(p)\lambda_{\min}(\Psi)=\Omega(\sqrt p) (Assumption 3.1), without imposing any specific loading pattern. To illustrate this, we further tested sparse confounding with weak loadings affecting only 10 of 400 features, following the setting in Section 5 of our paper with (n,p,q,s)=(200,400,3,5)(n,p,q,s)=(200,400,3,5). Let SS and S^\widehat S denote the true and estimated supports, respectively. We report averages over 20 repetitions: F1 score, FP=∣S^∖S∣\mathrm{FP}=|\widehat S\setminus S|, FN=∣S∖S^∣\mathrm{FN}=|S\setminus\widehat S| for support recovery and ℓ2\ell_2 error for estimation accuracy.

Table 1

Nonzero loadingsAffected featuresF1 ↑FP ↓FN ↓ℓ₂ error ↓
Ours
N(0,0.5²) (weak)400/4000.9920.1000.164
N(0,1²) (weak)400/4001.0000.0000.151
N(0,4²) (baseline;Ours)400/4001.0000.0000.149
N(0,0.5²) (weak)10/4000.8103.0000.505
N(0,1²) (weak)10/4000.8292.1500.387
N(0,4²) (strong)10/4000.9910.1000.159
Baseline
N(0,4²) (Lasso)400/4000.3816.652.202.034
N(0,4²) (Trim+Lasso)400/4000.9191.100.000.590

As shown in rows 1–3 and 6, our method achieves high F1 scores and small ℓ2\ell_2 estimation errors with dense loadings or sparse but strong loadings. When loadings are both sparse and weak (rows 4–5), the dense-confounding assumption is violated and performance deteriorates. Interestingly, even in the worst case, the false positives are still less for our method than for the Lasso (3.00 vs. 6.65). Moreover, it also achieves a smaller ℓ2\ell_2 error than both the Lasso and Trim+Lasso baselines. These results further demonstrate the robustness of our method.

Weakness 2 & Questions 2:About Linear model assumptions in real-world experiments.

Response: We thank the reviewer for raising this concern. As the reviewer states, linear models are widely used in causal inference. They are also well established in real-world applications and have yielded interesting findings. For gene expression, linear models are widely used [1–5]. For LLM-based evaluation, X is a representation that encodes high-level concepts such as curiosity, individualism, and satiety, thus motivating a linear relationship between it and the response. Such a linear model is widely adopted [6,7] in the literature. Extending our analysis to nonlinear effects or interactions is also of interest but beyond the scope of this paper.

Reference:

[1] Shabalin A A. Matrix eQTL: ultra fast eQTL analysis via large matrix operations. Bioinformatics, 2012, 28(10): 1353–1358.

[2] Magwene P M, Kim J. Estimating genomic coexpression networks using first-order conditional independence. Genome Biology, 2004, 5: R100.

[3] Haury A C, Mordelet F, Vera-Licona P, Vert J P. TIGRESS: Trustful inference of gene regulation using stability selection. BMC Systems Biology, 2012, 6: 145.

[4] Li Z, Liu Y, Wei K, et al. Deconfounded and debiased estimation for high-dimensional linear regression under hidden confounding with application to omics data. Bioinformatics, 2025, 41(7): btaf400.

[5] Jablonski K P, Pirkl M, Ćevid D, Bühlmann P, Beerenwinkel N. Identifying cancer pathway dysregulations using differential causal effects. Bioinformatics, 2022, 38(6): 1550–1559.

[6] Kim J, Evans J, Schein A. Linear representations of political perspective emerge in large language models. The Thirteenth International Conference on Learning Representations (ICLR), 2025.

[7] Gurnee W, Tegmark M. Language models represent space and time. The Twelfth International Conference on Learning Representations (ICLR), 2024.

[8] Osher, Stanley, et al. "Sparse recovery via differential inclusions." Applied and Computational Harmonic Analysis 41.2 (2016): 436-469.

[9] Ćevid, D., Bühlmann, P., and Meinshausen, N. Spectral deconfounding via perturbed sparse linear models. Journal of Machine Learning Research, 21(232):1–41, 2020.

[10] Guo, Z., Ćevid, D., and Bühlmann, P. Doubly debiased lasso: High-dimensional inference under hidden confounding. Annals of Statistics, 50(3):1320, 2022.

[11] Sun, Y., Ma, L., and Xia, Y. A decorrelating and debiasing approach to simultaneous inference for high-dimensional confounded models. Journal of the American Statistical Association, 119(548):2857–2868, 2024.

[12] Wang, Y. and Shah, R. Latent confounding in high-dimensional nonlinear models. arXiv, 2025.

[13] Liu, S., Ye, H., Xing, L., and Zou, J. Y. (2024). In-context Vectors: Making In Context Learning More Effective and Controllable Through Latent Space Steering. In Proceedings of the 41st International Conference on Machine Learning (ICML), PMLR 235:32287–32307.

AuthorsOpenReview ↗

Follow-up Before the Discussion Deadline

Dear Reviewer 5TZa,

With less than a day remaining in the discussion period, we would like to kindly follow up on our responses. We have addressed your concerns in two parts:

Evaluation metrics. We have explained our use of cross-validation to select the stopping time and our choice of AUC as the evaluation metric. We have also provided additional experimental results on feature selection at the cross-validation stopping time, showing an F1 score of 1 with zero false positives and zero false negatives.

Research setting. We have clarified that our contributions build on the widely used dense-confounding setting, which allows both strong sparse loadings and weak dense loadings. We have provided additional experimental results for settings where sparse and weak loadings violate the dense-confounding assumption. We have also provided references showing that linear models are widely used in both applications.

We would greatly appreciate your feedback on whether these responses address your concerns and welcome any remaining questions before the discussion closes.

Thank you again for your time and consideration.

Reviewer_JNbeOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper investigates whether spectral deconfounding can achieve true model selection (identifying causally relevant features) in high-dimensional observational data with latent, dense confounders. The authors prove that spectral deconfounding mitigates spurious correlations to satisfy the irrepresentable condition asymptotically, allowing the Lasso to correctly isolate the causal feature set. To eliminate standard regularization bias, they incorporate the Linearized Bregman Iteration, extending its unbiased properties to random designs with latent confounders. Ultimately, their estimator achieves the oracle L∞L_{\infty } estimation rate under the beta-min condition, matching the minimax-optimal rate of unconfounded fixed designs up to a logarithmic factor.

Critical strengths
  • the paper shifts the focus of high-dimensional spectral deconfounding from mere estimation accuracy to true model selection. By proving that attenuating dominant singular values allows the irrepresentable condition to hold asymptotically, it provides a foundational answer to how deconfounding isolates causally relevant features in the presence of latent, dense confounders.
  • the authors replaced biased penalization methods with LBI, extending its feature selection and unbiasedness guarantees from traditional fixed designs to complex random designs containing hidden confounding. This theoretical leap yields an ℓ2\ell _{2} convergence rate of slog⁡s/ns \log s / n under the beta-min condition, matching the minimax-optimal rate of completely unconfounded settings up to a logarithmic factor.
  • the proposed framework addresses a pressing real-world issue in modern AI, specifically demonstrating how dense confounding manifests within LLM hidden representations (due to shared styles or prompt formatting).
Critical weaknesses
  • the theoretical validity of the spectral transformation hinges heavily on the dense confounding assumption (where latent factors affect almost all covariates). If the confounding is sparse or only targets a small subpopulation of features, the spectral transformation could accidentally attenuate true causal signals, severely undermining both model selection and estimation.
  • chieving the minimax-optimal oracle estimation rates requires satisfying a strict beta-min condition. In practical observational studies, causal effects can be weak or decay smoothly, meaning real-world datasets may frequently violate this threshold and degrade selection performance.
  • while Linearized Bregman Iteration avoids the optimization bias of Lasso, tracking its dynamic differential-inclusion solution path can be computationally expensive. The paper lacks a detailed discussion on the algorithmic scalability and tuning overhead when scaling to high-dimensional matrices.
Questions for authors

What happens to the model selection guarantees if the true causal features have effects that fall below the threshold mandated by the beta-min condition? Does the method suffer from high false-negative rates in weak-signal regimes?

3 comments
AuthorsOpenReview ↗

Research setting

We thank the reviewer for the positive feedback and thoughtful comments. We organize our responses into two parts: research setting, and computational complexity.

1.Research setting

We sincerely thank the reviewer for these questions. Our main claim here is that our contributions are built upon the widely used dense-confounding setting. Below, we address the concerns about this setting and discuss some commonly used assumptions.

Weakness 1:About the dense confounding assumption.

Response 1: We appreciate the reviewer's comment. We address this concern about our assumption from two perspectives below.

First, we would like to clarify that dense confounding is a common and widely used assumption in the literature. It has been adopted not only by Ćevid et al. (2020) [1], but also by Guo et al. (2022) [2], Sun et al. (2024) [3], and Wang and Shah (2025) [4]. Such structural assumptions are necessary because causal effects are generally unidentifiable under unobserved confounding without additional assumptions. Moreover, an advantage of this assumption is that it does not require additional auxiliary variables, which are difficult to obtain in high-dimensional settings. Furthermore, this assumption is natural in a lot of real-world applications. For example, a few genes may influence most other genes [5–6], while writing style can affect an entire text [7]. These are important problems, and our work is built upon this setting. Our main contribution is to analyze how spectral deconfounding improves estimation through causal feature selection under this assumption (line 65), and the relaxation of the dense-confounding assumption is beyond the scope of this paper.

Second, the dense-confounding assumption does not require latent factors to affect almost all covariates. It allows sparse loadings of sufficient strength, as long as the loading matrix satisfies λmin⁡(Ψ)=Ω(p)\lambda_{\min}(\Psi)=\Omega(\sqrt p) (Assumption 3.1). In Appendix E.4 (Table 1), we also report a setting in which confounding is restricted to a subset of features by setting some loadings to zero. Our method achieves a variable-selection AUC of 0.99 in this setting, showing that it remains effective when confounding affects only a subset of features.

Weakness 2 & Questions :About the beta-min condition.

Response 2: We would like to emphasize that, even without beta-min, our estimator still achieves the ℓ2\ell_2 estimation error rate Op(slog⁡p/n)O_p(\sqrt{s\log p/n}), matching the minimax-optimal rate in the unconfounded fixed-design setting where no beta-min condition is imposed [8].Moreover, Table 1 provides empirical evidence of estimation accuracy with weak signals: even with very weak signals, our estimator still achieves a lower ℓ2\ell_2 estimation error than the baselines evaluated under strong signals (see below for details).

As for model selection, we would like to clarify that beta-min is a common sufficient condition for model selection[8,10,11,12]. Thus, to show the perfomance without beta-min condition, we conduct additional experiments by varying δ\delta from strong to weak in βS∗=(1,1,1,δ,δ)\beta_S^*=(1,1,1,\delta,\delta), following the setting in Section 5 of our paper, with (n,p,q,s)=(200,400,3,5)(n,p,q,s)=(200,400,3,5). Let SS and S^\widehat S denote the true and estimated supports, respectively. We report averages over 20 repetitions: the F1 score, FP=∣S^∖S∣\mathrm{FP}=|\widehat S\setminus S|, and FN=∣S∖S^∣\mathrm{FN}=|S\setminus\widehat S| for model selection, and the ℓ2\ell_2 error for estimation accuracy.

Table 1

MethodδF1 ↑FP ↓FN ↓ℓ₂ error ↓
Lasso1.000.3816.652.202.034
Trim+Lasso1.000.9191.100.000.590
Ours1.001.0000.000.000.149
Ours0.500.9810.150.050.177
Ours0.200.7610.201.800.312
Ours0.050.7500.002.000.137

As shown, when the signals are strong (δ=1\delta = 1, row 3), our method achieves the best F1 score of 1, indicating support recovery. As the signals δ\delta gradually weaken (row 4-6), false negatives increase but still remain lower than those of the Lasso baseline evaluated under strong signals, while false positives remain low compared with both baselines.

AuthorsOpenReview ↗

Computational complexity

2.Computational complexity

Weakness 3:About the computationally efficiency and scalability .

Response 3: We thank the reviewer for raising this concern. We would like to clarify that Spec-LBI does not increase the computational burden. On the contrary, it can improve computational efficiency. Intuitively, Spec-LBI requires only a single iterative run to generate a regularization path [8], whereas computing a Spec-Lasso path using ISTA or FISTA [9] requires iterative optimization at multiple regularization parameter values. More specifically, for X∈Rn×pX\in\mathbb{R}^{n\times p} with p≥np\ge n, let mλm_\lambda denote the number of regularization parameter values. By the KK-th iteration, Spec-LBI only costs O(n2p+Knp)O(n^2p+Knp), whereas Spec-Lasso costs O(n2p+mλKnp)O(n^2p+m_\lambda Knp). This highlights the computational efficiency of our method.

Reference:

[1] Ćevid, D., Bühlmann, P., and Meinshausen, N. Spectral deconfounding via perturbed sparse linear models. Journal of Machine Learning Research, 21(232):1–41, 2020.

[2] Guo, Z., Ćevid, D., and Bühlmann, P. Doubly debiased lasso: High-dimensional inference under hidden confounding. Annals of Statistics, 50(3):1320, 2022.

[3] Sun, Y., Ma, L., and Xia, Y. A decorrelating and debiasing approach to simultaneous inference for high-dimensional confounded models. Journal of the American Statistical Association, 119(548):2857–2868, 2024.

[4] Wang, Y. and Shah, R. Latent confounding in high-dimensional nonlinear models. arXiv, 2025.

[5] Li Z, Liu Y, Wei K, et al. Deconfounded and debiased estimation for high-dimensional linear regression under hidden confounding with application to omics data. Bioinformatics, 2025, 41(7): btaf400.

[6] Jablonski K P, Pirkl M, Ćevid D, Bühlmann P, Beerenwinkel N. Identifying cancer pathway dysregulations using differential causal effects. Bioinformatics, 2022, 38(6): 1550–1559.

[7] Liu, S., Ye, H., Xing, L., and Zou, J. Y. (2024). In-context Vectors: Making In Context Learning More Effective and Controllable Through Latent Space Steering. In Proceedings of the 41st International Conference on Machine Learning (ICML), PMLR 235:32287–32307.

[8] Osher, Stanley, et al. "Sparse recovery via differential inclusions." Applied and Computational Harmonic Analysis 41.2 (2016): 436-469.

[9] Beck A, Teboulle M. A fast iterative shrinkage-thresholding algorithm for linear inverse problems. SIAM Journal on Imaging Sciences, 2009, 2(1): 183–202.

[10] Zhao, Peng, and Bin Yu. "On model selection consistency of Lasso." The Journal of Machine Learning Research 7 (2006): 2541-2563.

[11] Zhang, C.-H. (2010). Nearly unbiased variable selection under minimax concave penalty. Annals of Statistics, 38(2), 894–942.

[12]Fan, J., Xue, L., & Zou, H. (2014). Strong oracle optimality of folded concave penalized estimation. The Annals of Statistics, 42(3), 819–849.

AuthorsOpenReview ↗

Follow-up Before the Discussion Deadline

Dear Reviewer JNbe,

With less than a day remaining in the discussion period, we would like to kindly follow up on our responses to your comments. We have addressed your concerns in two parts:

Research setting. We have clarified that our contributions build on the widely used dense-confounding assumption, which allows sparse loadings of sufficient strength. Within this setting, we have further clarified the role of the standard beta-min condition: even without it, our estimator matches the minimax-optimal ℓ2\ell_2 rate for unconfounded fixed designs. We have also provided weak-signal experiments assessing selection and estimation performance.

Computational complexity. We have provided evidence of our method’s scalability and computational efficiency through a computational complexity analysis.

We would greatly appreciate your feedback on whether these clarifications and results address your main concerns. If any issues remain unresolved or require further clarification, we would be happy to address them before the discussion closes.

Thank you again for your time and consideration.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary of Revisions and Additional Evidence

We thank the Area Chair and reviewers for their thoughtful feedback. We have addressed the concerns through manuscript revisions, clarifications, and additional experiments. We summarize our responses below.

Research setting and assumptions (Reviewers JNbe, 5TZa, wMha). Our contributions build on the widely used dense-confounding setting, which allows strong sparse and weak dense loadings. Within this setting, even without the standard beta-min condition, Spec-LBI also matches the minimax-optimal ℓ2\ell_2 rate for unconfounded fixed designs. We have also provided additional experiments showing low ℓ2\ell_2 errors under violations of beta-min (0.137–0.312 vs. 0.590 for Trim+Lasso and 2.034 for Lasso) and dense confounding (0.387–0.505 vs. 0.590 and 2.034, respectively), with the baselines evaluated in the strong-signal and dense-confounding setting.

Causal interpretation and identification (Reviewer wMha). We have clarified that our causal interpretation follows from the structural equation model (SEM) formulation of our model, while identifiability relies on the dense-confounding assumption.

Contributions and experimental benchmarks (Reviewer wMha). We have provided sufficient experimental benchmarks designed around two contributions: theoretically explaining how spectral deconfounding improves causal feature selection, and building on this insight to develop Spec-LBI for bias reduction and more accurate estimation. Our work is therefore positioned within the spectral-deconfounding methods under the dense-confounding assumption.

Computational efficiency, algorithmic details, and hyperparameter sensitivity (Reviewer JNbe, wMha, 5TZa). We have provided a computational complexity analysis and clarified algorithmic details, including early stopping, cross-validation, AUC, initialization, and convergence. Although the manuscript already explains how the hyperparameters affect the model, we have also conducted additional sensitivity experiments.

The additional experiments have been included in the appendix E. Revisions are marked as follows:

  • 🟥 Red — Reviewer JNbe.
  • 🟩 Green — Reviewer 5TZa.
  • 🟦 Blue — Reviewer wMha.
  • ⬛ Black — Content shared across reviewers.

We have provided detailed point-by-point responses and additional experimental evidence addressing the concerns raised. We also followed up to invite further discussion; however, we have not yet received further feedback or clarification requests from the reviewers. We would welcome any additional questions or suggestions during the remaining discussion period and hope the revised manuscript and supplementary evidence adequately address the concerns raised.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.