- Rating
- 1
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 3
Read review
Paper summary
The paper studies the classical normal-score correlation matrix in a Gaussian copula model. Its main result is a high-probability approximation to the latent Gaussian sample Pearson correlation matrix: $ |\widehat\Sigma^{\mathrm{NS}}-\widehat R^G|_{\max} \lesssim \frac{\log p+\log\log n}{n}. $ The authors use it to prove restricted-eigenvalue guarantees at , assuming fixed positive population restricted curvature. They then apply these results to sparse regression and precision-matrix estimation. The is paper is theoretical; no experiments are reported.
Critical strengths
A useful refinement of normal-score theory. The paper gives a sharper finite-sample approximation to the latent Gaussian sample correlation matrix for the classical untruncated normal-score estimator. This extends first-order oracle equivalence to and yields RE guarantees at , under fixed positive population restricted curvature. These are useful additions to the theory of this estimator.
Critical weaknesses
-
The -versus- comparison uses a weak baseline. The paper repeatedly emphasizes this improvement, but the baseline comes from directly converting a first-order population max-norm bound to an RE bound. [Zhao–Genest (2019, Theorem F.1)], as summarized in the authors' Remark B.2, already give a uniform normal-score expansion with the same leading influence term and remainder $ r_{\mathrm{old}}=\frac{\log p(\log n)^2+(\log p)^{3/2}}{n}. $ Under their assumptions, subtracting the oracle expansion and applying the cone inequality appears to give an perturbation. With fixed positive population restricted curvature, this suggests RE transfer when , already linear in sparsity up to logarithmic factors. The comparison with the generic bound therefore does not clearly identify what the paper adds over this earlier expansion.
-
The improvement over Zhao–Genest is mainly a wider dimensional range. As Remark B.2 explains, the earlier remainder is negligible at the leading max-norm scale when , whereas the new bound establishes this throughout . This is a valid improvement as it strengthens the guarantee for the same classical estimator. I find this extension useful, but its statistical importance beyond the sharper analysis remains unclear.
-
The applications do not show a clear statistical advantage. Section 4 uses standard penalized quadratic programs and obtains familiar rates with target -norm factors. However, the paper does not show a better estimation rate or a substantial advantage over competing rank estimators under comparable assumptions. The applications leave me unconvinced that the refinement is enough for acceptance.
-
No algorithmic contributions and experiments. This paper is a purely technical polylogarithmic refinement of an existing estimator with no algorithmic innovation and no empirical evaluation. I feel the scope is a bit narrow for ICLR.
Questions for authors
- Would Zhao–Genest’s Theorem F.1, combined with Lemma 3.5 and the cone inequality, already give RE when $ \frac{s{(\log p)(\log n)^2+(\log p)^{3/2}}}{n}=o(1)? $ If so, please spell out what the new result improves in the rate, probability, and covariance assumptions.
- Which result in Section 4 needs the wider dimensional range or weaker spectral assumptions? Could the authors give a concrete setting covered by this result but not by the strongest earlier rank-based guarantees?
4 comments
Reply to Reviewer LaTM (2/3): Q2, which result in Section 4 benefits
Q2: "Which result in Section 4 needs the wider dimensional range or weaker spectral assumptions?" Corollaries 4.1 and 4.2 use two inputs: the max-norm rate (Corollary 2.7) for the gradient, and RE preservation (Corollary 2.4) for the curvature. The benefit is of two different kinds, which we keep separate.
(a) Wider dimensional range for the same estimator. Take and , which satisfies the hypotheses of Theorem F.1. Then
The second quantity is the stated earlier remainder bound, which therefore does not certify a vanishing max-norm error in this regime. Corollaries 2.7 and 2.4 supply the max-norm and RE bounds used in Section 4; for a bounded target norm and a fixed population RE, Corollary 4.1 gives an error of order . A slow-rate argument based on Corollary 2.7 gives an error of order under the same two conditions, the same order here. The example illustrates the wider dimensional range of the guarantees for this normal-score procedure. The comparison is between two guarantees for the same estimator: the stated earlier bound and ours.
(b) A weaker spectral assumption than a named Kendall-based theorem. The result is Corollary 4.1. Let the covariates be equicorrelated, with fixed , and for fixed let
- and , so the target norm is fixed and the signal does not vanish.
- . Adding independent Gaussian noise of variance gives a latent response of unit variance, hence a valid Gaussian copula regression model.
- , so , and Corollary 4.1 gives once .
- Every principal submatrix of has condition number . These are also principal submatrices of the joint correlation matrix, so the uniformly bounded sparse condition number assumed in Theorem 1 of Cai & Zhang (2018) fails as .
This example separates the covariance assumptions of Corollary 4.1 from those of Cai and Zhang's Theorem 1; the comparison is between these two theorems. Corollary 4.2 benefits in the first way, through the wider range of dimensions.
Reply to Reviewer LaTM (3/3): W3 and W4
W3: "The applications do not show a clear statistical advantage." The programs of Section 4 are the standard ones and the rates are the familiar ones, with the factors and ; under their stated assumptions, Kendall-based procedures attain the same order (Table B of the general response). The advantage of the normal-score matrix is not in the rate. It is in the spectral assumption, in the computation and in finite-sample accuracy.
Spectral assumption. Corollary 4.1 needs a population RE lower bound as its only spectral condition; in example (b) of our reply to Q2 (2/3) the bounded sparse condition numbers of Cai & Zhang's Theorem 1 fail. The Discussion (p. 10) now states this comparison.
Computation. The normal-score matrix is positive semidefinite by construction, so the -penalized quadratic programs of Section 4 are convex as they stand: Lasso-type solvers apply directly and every local minimizer is global. With a Kendall or Spearman matrix, which need not be positive semidefinite, these programs need not be convex. The remedies in the literature are a projection step (Liu et al., 2012; Cai & Zhang, 2018), linear programs such as the Dantzig selector or CLIME (Liu et al., 2012), or nonconvex theory with an -ball constraint (Barber & Kolar, 2018). Barber & Kolar note that for large problems Dantzig-selector-type methods are computationally much slower than Lasso-type methods.
Accuracy. The normal-score matrix is asymptotically efficient in fixed dimension, and in the experiments of Section 4.3 it is closer to the latent oracle than the Kendall and Spearman matrices (numbers under W4). The paper gives this estimator high-dimensional guarantees that match the Gaussian oracle at the max-norm scale.
W4: "No algorithmic contributions and experiments." The contribution is theoretical: guarantees for a classical estimator whose programs are solved by existing Lasso-type solvers, so no new algorithm is called for. The factor removed is polylogarithmic when is polynomial in , and of order when (W2). The revision adds Section 4.3, with details in Appendix E: two experiments on two designs with the normal-score, latent-oracle, Kendall and Spearman matrices.
- Distance to the oracle (Figure 1a). For , as grows from 100 to 1600 the mean of falls by a factor of 14 to 16, and its product with stays between 1.17 and 1.35 in all 20 settings, in agreement with Theorem 2.1. The Kendall and Spearman distances fall by a factor of 4.4 to 5.3, and at they are 4.5 to 6.2 times the normal-score distance. The mean max-norm error with respect to exceeds the oracle's by at most 3.1% for normal scores, and by 3.3% to 11.5% for the other two matrices.
- Regression, with from 2 to 32 (Figures 1b and 2). In every cell the normal-score error exceeds the oracle's by at most 5%; the Kendall and Spearman errors exceed it by up to 9%. Under cross-validation, the ratio to the oracle exceeds one by more than two paired standard errors in 1 of 30 cells for normal scores, and in 24 of 60 for Kendall and Spearman. Each count compares one method with the oracle.
On scope: the nonparanormal model and its rank-based correlation estimators were developed at machine-learning venues (Liu, Lafferty & Wasserman, JMLR 2009; Liu et al., ICML 2012; Liu, Han & Zhang, NeurIPS 2012), and restricted eigenvalues are the standard design condition for -regularized learning. The paper establishes that condition, at the Gaussian-design sample size, for the design obtained by rank-based Gaussianization of the features in a Gaussian copula model, whenever the latent correlation matrix satisfies it.
Response to authors
Thank you for your clarification. The table is very clear for the comparison with the related literature, Zhao–Genest. However, this also enhances my thought that the paper's main contribution is a technical polylogarithmic refinement of an existing estimator, which I feel is a bit scope limited. Therefore. I would like to keep my score.
Reply to Reviewer LaTM (1/3): W1, Q1 and W2, the earlier expansion as baseline
Thank you for the precise questions. Your derivation in W1 and Q1 is correct, and the revision adopts it as the baseline. In short: relative to your route, Theorem 2.1 removes the factor (logn)2+logp and is nonasymptotic; the probability at t=2logp is of the same order, and the covariance assumptions are the same.
W1: "The s2-versus-s comparison uses a weak baseline." Under the hypotheses of Theorem F.1 of Zhao & Genest (logp=o(n) and logn=O(p)), subtracting the oracle expansion of our Lemma 3.5 (with t=2logp) from their expansion gives ∥ΣNS−RG∥max≤Clogp(logn)2+(logp)3/2/n with probability at least 1−C/p2. The cone inequality and the Gaussian-oracle comparison (Lemma 3.6) then preserve restricted eigenvalues, for a fixed population lower bound and fixed relative accuracy, once
n≳slogp(logn)2+logp.
So the earlier expansion gives a sufficient condition that is linear in s, as you say; the s2logp figure of the submission was the generic conversion of a population max-norm bound. The Introduction (p. 2) now writes out the earlier remainder bound and states the condition above, Remark B.2 (p. 16) records the derivation, and the abstract no longer makes the s2logp comparison.
Q1: "… what the new result improves in the rate, probability, and covariance assumptions."
W2: "The improvement over Zhao–Genest is mainly a wider dimensional range." The improved remainder has two consequences beyond the reach of the earlier bound.
Example (a) in our reply to Q2 (2/3) works through one sequence in the added range, logp≍n3/4.