- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
The paper proposes a scaled gradient descent (ScaledGD) algorithm to estimate latent-variable Gaussian graphical models via a sparse–low-rank decomposition of the precision matrix. The algorithm combines Gram-matrix preconditioning, symmetric hard thresholding, and a joint feasibility-preserving line search. Its main theoretical contribution is a local linear convergence guarantee for both sparse and low-rank components, with a contraction factor independent of the latent condition number under appropriate identifiability and initialization assumptions. The paper also establishes sufficient conditions for spectral initialization and derives finite-sample estimation bounds. Synthetic experiments demonstrate improved convergence under latent ill-conditioning and compare ScaledGD with existing optimization methods.
Critical strengths
Meaningful theoretical contribution. Theorem 1 establishes local linear convergence for the joint sparse and low-rank estimation problem, with a contraction factor independent of the latent condition number. The analysis explicitly handles hard thresholding, support and rank preservation, positive definiteness, and line-search acceptance, extending existing scaled-gradient results to the coupled estimation setting.
Technically well-developed analysis. The fixed-rank geometric arguments in Lemmas 6–9 provide a principled explanation for the condition-number-independent convergence rate. The paper also carefully distinguishes the local convergence guarantee from the signal-strength requirements governing initialization and statistical accuracy.
Experimental support for improved optimization. The conditioning experiment in Section 5.1 demonstrates nearly constant convergence rates for ScaledGD as the latent condition number increases from 5 to 70, while ordinary gradient descent slows substantially. The sampled-data experiments also show lower mean optimization runtimes than AltGD at every sample size tested.
Critical weaknesses
Limited latent-component recovery. Despite the motivation of separating sparse conditional dependencies from latent influences, Table 1 shows relatively poor estimation of the low-rank component. ScaledGD's relative error exceeds 1 at (n=800) and (n=1600), meaning it performs worse on this metric than the trivial zero estimator. The convex PPA baseline consistently achieves substantially lower latent-component error across all evaluated sample sizes. AltGD also exhibits relatively large latent-component errors (1.674, 1.086, 0.775, and 0.549), suggesting that this limitation is not specific to Gram scaling. These results limit the demonstrated effectiveness of the factorized estimation approach for recovering the latent-variable decomposition and warrant further discussion.
Incomplete validation of the central optimization advantage. The condition-number-independence claim is experimentally demonstrated only against ordinary gradient descent using the population covariance and true precision matrix for initialization. The comparison against AltGD, the more directly relevant existing method, is conducted at a single latent condition number . Consequently, the experiments do not establish whether ScaledGD provides a meaningful conditioning-related advantage over AltGD under sampling noise. Moreover, the theoretical guarantee covers equal step sizes, whereas the sampled-data experiments tune the sparse and factor step sizes separately.
Restrictive assumptions and limited practical evaluation. The convergence guarantee requires the correct latent rank, correct initial sparse support, and initialization within a neighborhood whose radius depends on the smallest nonzero latent eigenvalue and minimum sparse-edge strength. The spectral initialization guarantee also relies on thresholds involving unknown population quantities. The sampled-data experiments assume knowledge of the true rank and sparsity budget, and no experiments investigate robustness to their misspecification. All evaluations use synthetic data, leaving practical applicability insufficiently demonstrated.
Questions for authors
The consistently inferior latent-component recovery relative to PPA raises concerns about the factorized approach's practical effectiveness for its motivating decomposition task. Can you explain that? Overall, the theoretical contribution is promising, but the experimental validation and demonstrated practical advantages are not sufficiently compelling to warrant acceptance in the current form.
4 comments
I would regard the rebuttal as successful overall.
Thanks for addressing my comments. The central optimization concern has been convincingly addressed; the equal-step theoretical restriction has been resolved in the revised theorem, and the initialization concern has meaningful additional evidence. The latent-component recovery criticism remains partially valid, but it is no longer sufficient on its own to justify the original weak rejection recommendation. So, I will increase my rating to 3, weak accept.
Thank you for your feedback and for reconsidering our work. We appreciate your suggestions and are glad that our revisions have addressed your main concerns.
Thank you again for your positive feedback. We noticed that the rating in the Official Review still appears unchanged, so we just wanted to check whether the update has gone through. We appreciate your time and consideration.
Thank you for the detailed comments. We have revised the paper and added theoretical analysis and experiments.
1. Latent recovery and relative error above 1 in small samples
Even with the true rank, weak latent directions can be difficult to distinguish from sampling noise. Fitting them with r=r⋆ can introduce enough noise to push the relative low-rank error above 1. This reflects finite-sample estimation difficulty. Even after optimization converges, the estimate may remain far from the truth. PPA's nuclear-norm regularization shrinks weak spectral directions, helping explain its better small-sample latent recovery.
Data-driven selection supports this explanation. At n=1600,κL=5, validation selects r=2 instead of r⋆=5, reducing ScaledGD's latent error from 1.117 to 0.649 and precision error from 0.129 to 0.114. See appendix Table 2 and Figure 3 in Section 5.3 for recovery, and Appendix G.3 and Figure 4 for spectral analysis.
2. Condition-number advantage
We extended the sampled-data recovery experiments to larger latent condition numbers. At n=108, increasing κL from 5 to 70 raises AltGD's iteration count about 14-fold, compared with 1.7-fold for ScaledGD, which also accurately recovers both components (Section 5.2, Table 1).
Our spectral analysis explains the limited slowdown in the original small-sample experiments. Sampling error masks weak latent directions, so the estimated matrix does not reflect the true ill-conditioning. At n=1600, the initial condition number stays around 1.4 as the true value increases from 5 to 70. With more samples, the estimated spectrum reflects the true ill-conditioning and AltGD's condition-number dependence becomes apparent (Appendix G.3, Figure 4).
3. Unequal steps and improved convergence bounds
Theorem 1 and its proof in Appendix C now allow unequal positive steps satisfying the step-size bound. We weight the two descent-energy terms separately and define
\tau=\frac{\min\lbrace\eta_S,\eta_B\rbrace}{\eta}.$$ For a fixed step ratio, the contraction factor remains independent of the latent condition number, allowing separate sparse and factor steps as used in the experiments. We also use strong convexity of the log-det loss to directly relate descent energy to the objective gap, tightening the original bound through matrix error. For equal steps at the largest allowed value, the contraction factor improves from $$1-\frac{\gamma^2}{9216\kappa_\ell^2} \quad\text{to}\quad 1-\frac{\gamma}{576\kappa_\ell}.$$ The dependence on the likelihood curvature ratio $\kappa_\ell$ improves from quadratic to linear, while independence from $\kappa_L$ is preserved. See Lemma 9 in Appendix B and Eq. (20) in Appendix A.1. ## 4. Unknown parameters and inaccurate initialization We introduced $\mathrm{ScaledGD}^{\mathrm{sel}}$, which selects rank and sparsity using validation negative log-likelihood. It identifies the true rank and sparsity budget with large samples and improves small-sample recovery by reducing model complexity (Sections 5.2–5.3 and Appendix G.4). We also tested controlled initial support perturbations. With true rank and sparsity fixed, we replace up to 20 true edges and vary $\kappa_L=5,10,\ldots,70$. ScaledGD corrects these support errors and converges across the tested condition numbers. Error curves show similar decay after support recovery, supporting robustness to support mismatch and the condition-number advantage (Section 5.3, Figure 2).