acceptodds
Under review as a conference paper at ICLR 2027

Restricted Eigenvalues and Max-Norm Bounds for Normal-Score Correlation Matrices

Abstract

In a -dimensional Gaussian copula model, the latent correlation matrix determines the dependence structure while the marginal distributions remain unspecified. The normal-score, or van der Waerden, correlation matrix is the sample correlation matrix of the standard normal quantiles of the scaled ranks of independent observations; it is asymptotically efficient in fixed dimension. A previous high-dimensional analysis of this estimator gives an entrywise expansion whose remainder bound constrains relative to . We show that, with high probability, differs from the oracle sample correlation matrix of the latent Gaussian data by at most a constant multiple of in max norm. When this difference is negligible relative to the max-norm scale , so the entrywise and restricted-eigenvalue guarantees of the Gaussian oracle carry over to . In particular, satisfies a restricted-eigenvalue condition whenever does, once , the Gaussian-design sample-size order; the generic conversion of a max-norm bound relative to would instead require . Moreover, with no restriction on relative to , and this rate is sharp. For sparse targets of bounded norm, normal-score procedures for Gaussian copula regression and inverse correlation estimation therefore attain the usual rate at Gaussian-design sample sizes, without the earlier dimensional restrictions. The proof couples each normalized score column with the latent Gaussian column. The column error is a one-dimensional matching error; its Euclidean norm is a Lipschitz function of the latent column with mean square of order . Gaussian concentration therefore controls all columns at once.

Primary area:
other topics in machine learning (i.e., none of the above)
Keywords:
Gaussian Copula, Normal Score Estimator, Restricted Eigenvalue, Max-Norm Bounds, High Dimension

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_LaTMOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

The paper studies the classical normal-score correlation matrix in a Gaussian copula model. Its main result is a high-probability approximation to the latent Gaussian sample Pearson correlation matrix: $ |\widehat\Sigma^{\mathrm{NS}}-\widehat R^G|_{\max} \lesssim \frac{\log p+\log\log n}{n}. $ The authors use it to prove restricted-eigenvalue guarantees at n≳slog⁡pn\gtrsim s\log p, assuming fixed positive population restricted curvature. They then apply these results to sparse regression and precision-matrix estimation. The is paper is theoretical; no experiments are reported.

Critical strengths

A useful refinement of normal-score theory. The paper gives a sharper finite-sample approximation to the latent Gaussian sample correlation matrix for the classical untruncated normal-score estimator. This extends first-order oracle equivalence to log⁡p=o(n)\log p=o(n) and yields RE guarantees at n≳slog⁡pn\gtrsim s\log p, under fixed positive population restricted curvature. These are useful additions to the theory of this estimator.

Critical weaknesses
  1. The s2s^2-versus-ss comparison uses a weak baseline. The paper repeatedly emphasizes this improvement, but the baseline comes from directly converting a first-order population max-norm bound to an RE bound. [Zhao–Genest (2019, Theorem F.1)], as summarized in the authors' Remark B.2, already give a uniform normal-score expansion with the same leading influence term and remainder $ r_{\mathrm{old}}=\frac{\log p(\log n)^2+(\log p)^{3/2}}{n}. $ Under their assumptions, subtracting the oracle expansion and applying the cone inequality appears to give an OP(srold)O_P(s r_{\mathrm{old}}) perturbation. With fixed positive population restricted curvature, this suggests RE transfer when srold=o(1)s r_{\mathrm{old}}=o(1), already linear in sparsity up to logarithmic factors. The comparison with the generic s2log⁡ps^2\log p bound therefore does not clearly identify what the paper adds over this earlier expansion.

  2. The improvement over Zhao–Genest is mainly a wider dimensional range. As Remark B.2 explains, the earlier remainder is negligible at the leading max-norm scale when log⁡p=o(n)\log p=o(\sqrt n), whereas the new bound establishes this throughout log⁡p=o(n)\log p=o(n). This is a valid improvement as it strengthens the guarantee for the same classical estimator. I find this extension useful, but its statistical importance beyond the sharper analysis remains unclear.

  3. The applications do not show a clear statistical advantage. Section 4 uses standard penalized quadratic programs and obtains familiar rates with target ℓ1\ell_1-norm factors. However, the paper does not show a better estimation rate or a substantial advantage over competing rank estimators under comparable assumptions. The applications leave me unconvinced that the refinement is enough for acceptance.

  4. No algorithmic contributions and experiments. This paper is a purely technical polylogarithmic refinement of an existing estimator with no algorithmic innovation and no empirical evaluation. I feel the scope is a bit narrow for ICLR.

Questions for authors
  1. Would Zhao–Genest’s Theorem F.1, combined with Lemma 3.5 and the cone inequality, already give RE when $ \frac{s{(\log p)(\log n)^2+(\log p)^{3/2}}}{n}=o(1)? $ If so, please spell out what the new result improves in the rate, probability, and covariance assumptions.
  2. Which result in Section 4 needs the wider dimensional range or weaker spectral assumptions? Could the authors give a concrete setting covered by this result but not by the strongest earlier rank-based guarantees?
4 comments
AuthorsOpenReview ↗

Reply to Reviewer LaTM (1/3): W1, Q1 and W2, the earlier expansion as baseline

Thank you for the precise questions. Your derivation in W1 and Q1 is correct, and the revision adopts it as the baseline. In short: relative to your route, Theorem 2.1 removes the factor (log⁡n)2+log⁡p(\log n)^2+\sqrt{\log p} and is nonasymptotic; the probability at t=2log⁡pt=2\log p is of the same order, and the covariance assumptions are the same.

W1: "The s2s^2-versus-ss comparison uses a weak baseline." Under the hypotheses of Theorem F.1 of Zhao & Genest (log⁡p=o(n)\log p=o(n) and log⁡n=O(p)\log n=O(p)), subtracting the oracle expansion of our Lemma 3.5 (with t=2log⁡pt=2\log p) from their expansion gives ∥Σ^NS−R^G∥max⁡≤Clog⁡p(log⁡n)2+(log⁡p)3/2/n\Vert\widehat\Sigma^{\rm NS}-\widehat R^G\Vert_{\max}\le C\\{\log p(\log n)^2+(\log p)^{3/2}\\}/n with probability at least 1−C/p21-C/p^2. The cone inequality and the Gaussian-oracle comparison (Lemma 3.6) then preserve restricted eigenvalues, for a fixed population lower bound and fixed relative accuracy, once

n≳slog⁡p(log⁡n)2+log⁡p.n\gtrsim s\log p\\{(\log n)^2+\sqrt{\log p}\\}.

So the earlier expansion gives a sufficient condition that is linear in ss, as you say; the s2log⁡ps^2\log p figure of the submission was the generic conversion of a population max-norm bound. The Introduction (p. 2) now writes out the earlier remainder bound and states the condition above, Remark B.2 (p. 16) records the derivation, and the abstract no longer makes the s2log⁡ps^2\log p comparison.

Q1: "… what the new result improves in the rate, probability, and covariance assumptions."

Theorem F.1 + Lemma 3.5Theorem 2.1
Oracle remainderlog⁡p(log⁡n)2+(log⁡p)3/2/n\\{\log p(\log n)^2+(\log p)^{3/2}\\}/n(log⁡p+t+log⁡log⁡n)/n(\log p+t+\log\log n)/n
Sufficient condition for RE preservationn≳slog⁡p(log⁡n)2+log⁡pn\gtrsim s\log p\\{(\log n)^2+\sqrt{\log p}\\}n≥A,slog⁡pn\ge A\\,s\log p
Probabilityat least 1−C/p21-C/p^2at least 1−Ce−t1-Ce^{-t} for every t≥0t\ge0; t=2log⁡pt=2\log p gives 1−C/p21-C/p^2
Range of (n,p)(n,p)asymptotic, under log⁡p=o(n)\log p=o(n) and log⁡n=O(p)\log n=O(p)every n≥3n\ge3 and p≥2p\ge2
Remainder negligible relative to log⁡p/n\sqrt{\log p/n}when log⁡p=o(n)\log p=o(\sqrt n)when log⁡p=o(n)\log p=o(n)
Covariance assumptionsnone on the spectrum of Σ\Sigmanone on the spectrum of Σ\Sigma
  • Rate. The factor (log⁡n)2+log⁡p(\log n)^2+\sqrt{\log p} disappears from the remainder, leaving an additive log⁡log⁡n\log\log n, and the RE condition becomes the Gaussian-design order slog⁡ps\log p.
  • Probability. At t=2log⁡pt=2\log p the two probabilities are of the same order. The gain is in form: the bound is nonasymptotic, with an adjustable exponential tail and without the condition log⁡n=O(p)\log n=O(p).
  • Covariance assumptions. The same in both routes: neither oracle comparison restricts the spectrum of Σ\Sigma, and both RE routes need the same population RE lower bound.

W2: "The improvement over Zhao–Genest is mainly a wider dimensional range." The improved remainder has two consequences beyond the reach of the earlier bound.

  • Oracle equivalence at the max-norm scale. ∥Σ^NS−R^G∥max⁡=oP(log⁡p/n)\Vert\widehat\Sigma^{\rm NS}-\widehat R^G\Vert_{\max}=o_P(\sqrt{\log p/n}) whenever log⁡p=o(n)\log p=o(n); the earlier remainder gives this when log⁡p=o(n)\log p=o(\sqrt n). Oracle equivalence is meant in this sense.
  • Restricted eigenvalues at the Gaussian-design order. The additional factor in the earlier sufficient condition is (log⁡n)2(\log n)^2 when pp is polynomial in nn, and nα/2n^{\alpha/2} when log⁡p≍nα\log p\asymp n^{\alpha} with 0<α<10<\alpha<1. These are factors in the certified sufficient conditions.

Example (a) in our reply to Q2 (2/3) works through one sequence in the added range, log⁡p≍n3/4\log p\asymp n^{3/4}.

AuthorsOpenReview ↗

Reply to Reviewer LaTM (2/3): Q2, which result in Section 4 benefits

Q2: "Which result in Section 4 needs the wider dimensional range or weaker spectral assumptions?" Corollaries 4.1 and 4.2 use two inputs: the max-norm rate (Corollary 2.7) for the gradient, and RE preservation (Corollary 2.4) for the curvature. The benefit is of two different kinds, which we keep separate.

(a) Wider dimensional range for the same estimator. Take log⁡p≍n3/4\log p\asymp n^{3/4} and s≍n1/8s\asymp n^{1/8}, which satisfies the hypotheses of Theorem F.1. Then

slog⁡pn≍n−1/8→0,log⁡p(log⁡n)2+(log⁡p)3/2n≍n1/8→∞.\frac{s\log p}{n}\asymp n^{-1/8}\to0,\qquad\frac{\log p(\log n)^2+(\log p)^{3/2}}{n}\asymp n^{1/8}\to\infty.

The second quantity is the stated earlier remainder bound, which therefore does not certify a vanishing max-norm error in this regime. Corollaries 2.7 and 2.4 supply the max-norm and RE bounds used in Section 4; for a bounded target ℓ1\ell_1 norm and a fixed population RE, Corollary 4.1 gives an ℓ2\ell_2 error of order n−1/16n^{-1/16}. A slow-rate argument based on Corollary 2.7 gives an error of order (log⁡p/n)1/4(\log p/n)^{1/4} under the same two conditions, the same order here. The example illustrates the wider dimensional range of the guarantees for this normal-score procedure. The comparison is between two guarantees for the same estimator: the stated earlier bound and ours.

(b) A weaker spectral assumption than a named Kendall-based theorem. The result is Corollary 4.1. Let the covariates be equicorrelated, ΣXX=(1−ρ)Ip+ρ11⊤\Sigma_{XX}=(1-\rho)I_p+\rho\mathbf 1\mathbf 1^\top with fixed ρ∈(0,1)\rho\in(0,1), and for fixed b∈(0,1)b\in(0,1) let

β1⋆=b2,βj⋆=b2(s−1) (2≤j≤s),βj⋆=0 (j>s).\beta^\star_1=\frac b2,\qquad\beta^\star_j=\frac{b}{2(s-1)}\ (2\le j\le s),\qquad\beta^\star_j=0\ (j>s).

  • ∥β⋆∥1=b\Vert\beta^\star\Vert_1=b and ∥β⋆∥2≥b/2\Vert\beta^\star\Vert_2\ge b/2, so the target ℓ1\ell_1 norm is fixed and the signal does not vanish.
  • (β⋆)⊤ΣXXβ⋆≤b2<1(\beta^\star)^\top\Sigma_{XX}\beta^\star\le b^2<1. Adding independent Gaussian noise of variance 1−(β⋆)⊤ΣXXβ⋆1-(\beta^\star)^\top\Sigma_{XX}\beta^\star gives a latent response of unit variance, hence a valid Gaussian copula regression model.
  • λmin⁡(ΣXX)=1−ρ\lambda_{\min}(\Sigma_{XX})=1-\rho, so κΣXX(s,3)≥1−ρ\kappa_{\Sigma_{XX}}(s,3)\ge\sqrt{1-\rho}, and Corollary 4.1 gives ∥β^−β⋆∥2≤C(1+b)(1−ρ)−1slog⁡p/n\Vert\widehat\beta-\beta^\star\Vert_2\le C(1+b)(1-\rho)^{-1}\sqrt{s\log p/n} once n≥A,slog⁡pn\ge A\\,s\log p.
  • Every s×ss\times s principal submatrix of ΣXX\Sigma_{XX} has condition number 1+ρs/(1−ρ)1+\rho s/(1-\rho). These are also principal submatrices of the joint correlation matrix, so the uniformly bounded sparse condition number assumed in Theorem 1 of Cai & Zhang (2018) fails as s→∞s\to\infty.

This example separates the covariance assumptions of Corollary 4.1 from those of Cai and Zhang's Theorem 1; the comparison is between these two theorems. Corollary 4.2 benefits in the first way, through the wider range of dimensions.

AuthorsOpenReview ↗

Reply to Reviewer LaTM (3/3): W3 and W4

W3: "The applications do not show a clear statistical advantage." The programs of Section 4 are the standard ones and the rates are the familiar ones, with the factors (1+B)κ0−2(1+B)\kappa_0^{-2} and Mκ0−2M\kappa_0^{-2}; under their stated assumptions, Kendall-based procedures attain the same order (Table B of the general response). The advantage of the normal-score matrix is not in the rate. It is in the spectral assumption, in the computation and in finite-sample accuracy.

Spectral assumption. Corollary 4.1 needs a population RE lower bound as its only spectral condition; in example (b) of our reply to Q2 (2/3) the bounded sparse condition numbers of Cai & Zhang's Theorem 1 fail. The Discussion (p. 10) now states this comparison.

Computation. The normal-score matrix is positive semidefinite by construction, so the ℓ1\ell_1-penalized quadratic programs of Section 4 are convex as they stand: Lasso-type solvers apply directly and every local minimizer is global. With a Kendall or Spearman matrix, which need not be positive semidefinite, these programs need not be convex. The remedies in the literature are a projection step (Liu et al., 2012; Cai & Zhang, 2018), linear programs such as the Dantzig selector or CLIME (Liu et al., 2012), or nonconvex theory with an ℓ1\ell_1-ball constraint (Barber & Kolar, 2018). Barber & Kolar note that for large problems Dantzig-selector-type methods are computationally much slower than Lasso-type methods.

Accuracy. The normal-score matrix is asymptotically efficient in fixed dimension, and in the experiments of Section 4.3 it is closer to the latent oracle than the Kendall and Spearman matrices (numbers under W4). The paper gives this estimator high-dimensional guarantees that match the Gaussian oracle at the max-norm scale.

W4: "No algorithmic contributions and experiments." The contribution is theoretical: guarantees for a classical estimator whose programs are solved by existing Lasso-type solvers, so no new algorithm is called for. The factor removed is polylogarithmic when pp is polynomial in nn, and of order nα/2n^{\alpha/2} when log⁡p≍nα\log p\asymp n^{\alpha} (W2). The revision adds Section 4.3, with details in Appendix E: two experiments on two designs with the normal-score, latent-oracle, Kendall and Spearman matrices.

  • Distance to the oracle (Figure 1a). For p∈50,200p\in\\{50,200\\}, as nn grows from 100 to 1600 the mean of ∥Σ^NS−R^G∥max⁡\Vert\widehat\Sigma^{\rm NS}-\widehat R^G\Vert_{\max} falls by a factor of 14 to 16, and its product with n/(log⁡p+log⁡log⁡n)n/(\log p+\log\log n) stays between 1.17 and 1.35 in all 20 settings, in agreement with Theorem 2.1. The Kendall and Spearman distances fall by a factor of 4.4 to 5.3, and at n=1600n=1600 they are 4.5 to 6.2 times the normal-score distance. The mean max-norm error with respect to Σ\Sigma exceeds the oracle's by at most 3.1% for normal scores, and by 3.3% to 11.5% for the other two matrices.
  • Regression, with n/(slog⁡p)n/(s\log p) from 2 to 32 (Figures 1b and 2). In every cell the normal-score error exceeds the oracle's by at most 5%; the Kendall and Spearman errors exceed it by up to 9%. Under cross-validation, the ratio to the oracle exceeds one by more than two paired standard errors in 1 of 30 cells for normal scores, and in 24 of 60 for Kendall and Spearman. Each count compares one method with the oracle.

On scope: the nonparanormal model and its rank-based correlation estimators were developed at machine-learning venues (Liu, Lafferty & Wasserman, JMLR 2009; Liu et al., ICML 2012; Liu, Han & Zhang, NeurIPS 2012), and restricted eigenvalues are the standard design condition for ℓ1\ell_1-regularized learning. The paper establishes that condition, at the Gaussian-design sample size, for the design obtained by rank-based Gaussianization of the features in a Gaussian copula model, whenever the latent correlation matrix satisfies it.

Reviewer_LaTMOpenReview ↗

Response to authors

Thank you for your clarification. The table is very clear for the comparison with the related literature, Zhao–Genest. However, this also enhances my thought that the paper's main contribution is a technical polylogarithmic refinement of an existing estimator, which I feel is a bit scope limited. Therefore. I would like to keep my score.

Reviewer_zWF6OpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper sharpens the theory of the normal-score correlation estimator for Gaussian copula models. Its main result gives a second-order max-norm approximation to the sample correlation matrix of the latent Gaussian data. This improves an earlier remainder bound and yields restricted-eigenvalue guarantees at the Gaussian-design sample-size order, together with sharp entrywise rates under weaker dimensional conditions. Applications include sparse latent regression and inverse correlation estimation. The paper contains no experiments.

Critical strengths
  1. Theorem 2.1 gives a precise, nonasymptotic approximation for an existing and widely studied estimator. It improves the earlier normal-score remainder and handles both extreme scores and dependence within each column. The proof uses the estimator's structure effectively: rank matching controls column errors, while fixed norms and a conditional Gaussian decomposition isolate the variance correction.

  2. The approximation transfers Gaussian-design guarantees to the normal-score matrix. In particular, it gives restricted-eigenvalue guarantees at n≳slog⁡pn\gtrsim s\log p, improving on the s2log⁡ps^2\log p requirement from a generic entrywise-bound argument. Together with the max-norm bound, this supports standard sparse-estimation procedures.

Critical weaknesses
  1. The paper does not clearly explain how much it adds beyond sharper bounds for the normal-score estimator. The s2log⁡ps^2\log p requirement comes from a generic proof route; it is not a known lower bound for this estimator. The introduction should make clear what was previously known about its restricted eigenvalues and compare the final guarantees with Kendall- and Spearman-based methods under matched assumptions. The applications largely follow standard ℓ1\ell_1 arguments, and their slog⁡p/n\sqrt{s\log p/n} rates additionally require the target-norm bounds BB or MM to remain constant. Section 4 notes that the automatic regression bound can grow as s\sqrt{s}, weakening the rate.

  2. Sharpening the bounds for this estimator is valuable, but I would like to see what else the approach can offer beyond this technical improvement. The proof relies on a fixed score grid, exact column norms, and Gaussian conditional independence, and it is unclear whether these ideas can help analyze other estimators or models. A numerical comparison with Kendall-based methods would also help show whether the improved sample-size regime makes a difference in practice.

Questions for authors
  1. How large can the constant AA in n≥Aslog⁡pn\geq As\log p be, particularly through its dependence on κ0,c0,η\kappa_0,c_0,\eta? A numerical illustration comparing normal-score, latent-oracle, and Kendall-based estimators as n/(slog⁡p)n/(s\log p) varies would help show whether the improved regime matters in practice.

  2. Which parts of the argument extend beyond normal scores and Gaussian copula models? Can the oracle comparison also give useful spectral-norm or structured-submatrix bounds without additional factors of ss or pp, or does the proof rely too closely on the present setting?

4 comments
AuthorsOpenReview ↗

Reply to Reviewer zWF6 (1/3): W1 and W2

Thank you for the detailed and constructive review.

W1: "… how much it adds beyond sharper bounds …"

The s2log⁡ps^2\log p baseline. It comes from a generic proof route, as you note, and was not presented as a lower bound. The sharper baseline is the earlier expansion of Zhao & Genest (2019, Theorem F.1): combined with the oracle expansion of our Lemma 3.5 and the cone inequality, it gives RE preservation under n≳slog⁡p(log⁡n)2+log⁡pn\gtrsim s\log p\\{(\log n)^2+\sqrt{\log p}\\}; Theorem 2.1 gives n≥A,slog⁡pn\ge A\\,s\log p. The Introduction (p. 2) and Remark B.2 (p. 16) now state this baseline, and the abstract no longer makes the s2log⁡ps^2\log p comparison (Table A of the general response).

What was previously known, and Kendall/Spearman under matched assumptions. The Introduction now writes out the earlier remainder bound (p. 2), and the Discussion compares the final guarantees with those for the Kendall and Spearman matrices (p. 10). The roadmap at the end of the Introduction points to this comparison, and Table B of the general response has the details. In short: the max-norm rate for normal scores now has the same generality as for Kendall and Spearman (Liu et al., 2012); for sparse regression the rates are of the same order and the assumptions differ. Theorem 1 of Cai & Zhang (2018) assumes uniformly bounded condition numbers of s×ss\times s principal submatrices; Corollary 4.1 assumes only a population RE lower bound on the spectrum, and a bounded target ℓ1\ell_1 norm. With rates of the same order, the difference shows in finite samples: in the experiments of Section 4.3 the normal-score matrix is closer to the latent oracle than the Kendall and Spearman matrices (Q1, 2/3).

The factors BB and MM. The corollaries state them explicitly:

∥β^−β⋆∥2≤C(1+B)κ02slog⁡pn,max⁡j∥θ^j−θj⋆∥2≤CMκ02slog⁡pn.\Vert\widehat\beta-\beta^\star\Vert_2\le\frac{C(1+B)}{\kappa_0^2}\sqrt{\frac{s\log p}{n}},\qquad\max_j\Vert\widehat\theta_j-\theta_j^\star\Vert_2\le\frac{CM}{\kappa_0^2}\sqrt{\frac{s\log p}{n}}.

The usual rate holds for bounded BB or MM, the setting of the abstract, and with the automatic bound B≤s/κ0B\le\sqrt s/\kappa_0 the regression bound is of order slog⁡p/ns\sqrt{\log p/n}, as Section 4.1 notes. The applications are meant to be standard: they transfer the matrix guarantees to the classical normal-score procedures.

W2: "… what else the approach can offer beyond this technical improvement." The reusable part is the columnwise coupling: it controls the discrepancy in design norms using Gaussian-design concentration and a scalar matching error. Our reply to Q2 (3/3) states what each part of the proof needs and gives the resulting bound for other score grids. The numerical comparison you asked for is in Section 4.3 and Appendix E, summarized in our reply to Q1 (2/3).

AuthorsOpenReview ↗

Reply to Reviewer zWF6 (2/3): Q1

Q1: "How large can the constant AA in n≥A,slog⁡pn\ge A\\,s\log p be …?" The proof of Corollary 2.4 in Appendix C (pp. 17–18 of the revision) allows

A=\max\bigl\\{9,\ C_1K\\,[1+\log\log(e^e+K)]\bigr\\},\qquad K=\frac{2C^2(1+c_0)^2}{\eta^2\kappa_0^2},

where CC is the constant in the first display of Corollary 2.4 and C1C_1 is a sufficiently large universal constant. The dependence is therefore (1+c0)2/(η2κ02)(1+c_0)^2/(\eta^2\kappa_0^2) up to an iterated logarithm, with no dependence on the upper spectrum or the condition number of Σ\Sigma. The universal constants are not optimized, so the proof gives this dependence, not a numerical value of AA.

Q1: "A numerical illustration …" We have added Section 4.3, with details in Appendix E. Its second experiment runs the regression program of Section 4.1 with the latent-oracle, normal-score, Kendall and Spearman matrices on two designs (AR(1) and equicorrelation, p=200p=200), with s∈5,10,20s\in\\{5,10,20\\} and n/(slog⁡p)n/(s\log p) from 2 to 32. Figure 1(b) shows the mean ℓ2\ell_2 errors for the equicorrelation design, and Figure 2 shows each method's mean ℓ2\ell_2 error divided by the oracle's, with paired standard errors.

  • The oracle's mean error falls from 0.200.20–0.460.46 at n/(slog⁡p)=2n/(s\log p)=2 to 0.090.09–0.170.17 at n/(slog⁡p)=32n/(s\log p)=32, for targets with ℓ2\ell_2 norm between 0.410.41 and 0.450.45.
  • Under five-fold cross-validation, the normal-score ratios lie in [0.96,1.03][0.96,1.03] and exceed one by more than two standard errors in 1 of 30 cells. The Kendall and Spearman ratios lie in [0.96,1.08][0.96,1.08] and exceed one by that margin in 24 of 60 cells.
  • Under simulation-oracle tuning, which selects the best value on each method's path using the true coefficients, the ranges are [0.99,1.05][0.99,1.05] and [0.99,1.09][0.99,1.09].

So the normal-score procedure tracks the oracle over the whole range: in every cell its mean error exceeds the oracle's by at most 5%, against up to 9% for the Kendall and Spearman versions. Each count compares one method with the oracle. The curves measure estimation error, not restricted eigenvalues over the cone or the worst-case constant AA.

The first experiment of Section 4.3 illustrates Theorem 2.1: for p∈50,200p\in\\{50,200\\}, as nn grows from 100 to 1600 the mean of ∥Σ^NS−R^G∥max⁡\Vert\widehat\Sigma^{\rm NS}-\widehat R^G\Vert_{\max} falls by a factor of 14 to 16, and its product with n/(log⁡p+log⁡log⁡n)n/(\log p+\log\log n) stays between 1.17 and 1.35 in all 20 settings (Figure 1a). The Kendall and Spearman distances to the oracle fall by a factor of 4.4 to 5.3, and at n=1600n=1600 they are 4.5 to 6.2 times the normal-score distance. The mean max-norm error with respect to Σ\Sigma exceeds the oracle's by at most 3.1% for normal scores, and by 3.3% to 11.5% for the other two matrices.

On whether the improved regime matters in practice: the estimator is the classical one, so our results change what can be guaranteed about it, not how it performs. The experiments show that its behavior agrees with the new guarantees, and that it is closer to the latent oracle than the Kendall and Spearman alternatives.

AuthorsOpenReview ↗

Reply to Reviewer zWF6 (3/3): Q2

Q2: "Which parts of the argument extend beyond normal scores and Gaussian copula models?" The proof has three components, with different scope.

ComponentRoleWhat it needs
Column matching (Lemma 3.1)Each normalized score column is the nearest rearrangement of a fixed grid to the latent column. The error norm is a distance to a fixed set, hence 11-Lipschitz, and Gaussian concentration controls all columns at once.Latent Gaussian columns; any fixed nondecreasing score grid
Exact column normsEvery score column has squared norm nn, which gives the exact diagonal correction in the expansion.Any grid with the same normalization
Gaussian conditional decompositionZk−ΣkjZjZ_k-\Sigma_{kj}Z_j is independent of the whole column ZjZ_j, so the cross terms are conditionally centered Gaussian. This produces the second-order cancellation.Gaussian copula; this step uses Gaussian conditioning

Three consequences.

  • Restricted eigenvalues need only the first component and Gaussian-design concentration. With Z~=Z+E\widetilde Z=Z+E, Lemma 3.1 and the Gaussian design comparison (Lemma C.1) give, for all vv,

    ∣v⊤Σ^NSv−v⊤Σv∣≤max⁡j∥E⋅j∥2n∥v∥1+Clog⁡p+tn∥v∥1,\Bigl|\sqrt{v^\top\widehat\Sigma^{\rm NS}v}-\sqrt{v^\top\Sigma v}\Bigr|\le\frac{\max_j\Vert E_{\cdot j}\Vert_2}{\sqrt n}\Vert v\Vert_1+C\sqrt{\frac{\log p+t}{n}}\Vert v\Vert_1,

    which is the comparison of Corollary 2.3 and hence yields Corollary 2.4. (The paper derives Corollary 2.3 through Theorem 2.1; the display is a shorter route that uses the coupling and Gaussian-design concentration.)

  • Other score grids. For another fixed nondecreasing grid a1≤⋯≤ana_1\le\cdots\le a_n with ∑rar=0\sum_ra_r=0 and ∑rar2=n\sum_ra_r^2=n, let Σ^a\widehat\Sigma^a be the 1/n1/n-normalized Gram matrix of the scored columns and mn(a)=E,dist2(G,Πa)m_n(a)=\mathbb E\\,\mathrm{dist}^2(G,\\{\Pi a\\}) for G∼N(0,In)G\sim N(0,I_n). For any t≥0t\ge0, with probability at least 1−Ce−t1-Ce^{-t}, the same argument gives

    ∣κΣ^a(s,c0)−κΣ(s,c0)∣≤C(1+c0)smn(a)+log⁡p+tn,\bigl|\kappa_{\widehat\Sigma^a}(s,c_0)-\kappa_\Sigma(s,c_0)\bigr|\le C(1+c_0)\sqrt{\frac{s\\{m_n(a)+\log p+t\\}}{n}},

    so restricted eigenvalues are preserved when the right side is small relative to the population value. For the grid of the paper we establish mn(a)=O(log⁡log⁡n)m_n(a)=O(\log\log n).

  • The second-order oracle approximation (Theorem 2.1), the oracle RE comparison (Corollary 2.5) and the max-norm rate without a log⁡log⁡n\log\log n term (Corollary 2.7) also use the third component, which is specific to the Gaussian copula.

Q2: "… spectral-norm or structured-submatrix bounds without additional factors of ss or pp …" Two questions should be separated.

  • Relative to the sample oracle. Taking t=log⁡pt=\log p, Theorem 2.1 alone gives, with probability at least 1−C/p1-C/p, a change of at most s,ϵs\\,\epsilon in sparse quadratic forms and p,ϵp\\,\epsilon in operator norm, with ϵ=C(log⁡p+log⁡log⁡n)/n\epsilon=C(\log p+\log\log n)/n. Max-norm control alone cannot remove these factors in general: for 0<ϵ≤10<\epsilon\le1, the correlation matrices IdI_d and (1−ϵ)Id+ϵ11⊤(1-\epsilon)I_d+\epsilon\mathbf 1\mathbf 1^\top differ by ϵ\epsilon in max norm and by (d−1)ϵ(d-1)\epsilon in operator norm. A sharper bound for the normal-score and oracle matrices themselves is not ruled out; it would need an analysis of the coupling error that goes beyond max-norm control.
  • Relative to the population matrix. Corollary 2.6 gives sup⁡∣v⊤(Σ^NS−Σ)v∣≤2Λs,rs+r2s\sup\lvert v^\top(\widehat\Sigma^{\rm NS}-\Sigma)v\rvert\le2\sqrt{\Lambda_s}\\,r\sqrt s+r^2s over ss-sparse unit vectors, with r=C(log⁡p+t+log⁡log⁡n)/nr=C\sqrt{(\log p+t+\log\log n)/n} and Λs\Lambda_s the largest eigenvalue of an s×ss\times s principal submatrix of Σ\Sigma. For bounded Λs\Lambda_s and rs≤1r\sqrt s\le1, this is of order rsr\sqrt s, with no factor of pp.

I appreciate the authors’ clarifications and additional results. While these resolve some of my concerns, they do not change my overall assessment of the contribution. My original rating therefore stands.

Reviewer_QKJ2OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
1
Soundness
3
Read review
Paper summary

This work shows that the normal-score correlation matrix is non-asymptotically close to an oracle sample correlation matrix of the latent Gaussian data in the max norm, with high probability. The authors also show that its max-norm error to the true correlation matrix is of order sqrt(logp/n) and that this rate is sharp. In particular, the conditions only require a number of samples that scales linearly with the sparsity and with the log-dimension. The authors analyze the error by considering entire columns with correlated entries, rather than individual entries of the matrix, at once. Moreover, they show how to apply the results to Gaussian copula regression, and to estimating the precision matrix in Gaussian copula graphical models.

Critical strengths

The paper introduces a novel analysis of the normal-score correlation matrix that considers each column of scores together, in order to handle the dependence between the entries in each column. Moreover, the analysis handles the fluctuations of extreme order statistics without truncation techniques. This leads to a non-asymptotic result that bounds the error to an oracle, with high probability, at a (log p + log log n)/n rate. This is smaller than the typical sqrt(log p / n) rate of concentration bounds.

Critical weaknesses
  • The authors note that one approach to estimate the correlation matrix for the Gaussian copula model is to convert Kendall’s tau and Spearman’s rho rank estimators to the latent correlation through sine transforms, which we understand has been applied frequently in past work. It is not clear how the bounds for the normal score compare to the theory developed for these alternative estimators.
  • The paper involves a significant amount of notation that is not always defined before it is used. For example, the sparsity level s does not seem to be defined in the abstract.
Questions for authors

We refer to the critical weaknesses above.

2 comments

Thank you for both points, which we have addressed in the revision (changes are in blue).

"It is not clear how the bounds for the normal score compare to the theory developed for these alternative estimators." The Discussion now states this comparison (p. 10), and Table B of the general response lists the results side by side. In brief:

  • Max-norm rate. The Kendall and Spearman sine matrices attain log⁡p/n\sqrt{\log p/n} without an upper-growth restriction on the dimension (Liu et al., 2012). Our Corollary 2.7 gives the normal-score matrix this rate without an upper-growth restriction; the stated remainder bound in Zhao & Genest (2019, Theorem F.1) does not establish that conclusion over the same range.
  • Sparse regression. The rates are of the same order. The assumptions differ. Theorem 1 of Cai & Zhang (2018) assumes uniformly bounded condition numbers of s×ss\times s principal submatrices. Our Corollary 4.1 assumes a lower bound on the population restricted eigenvalue as its only spectral condition, together with a bounded ℓ1\ell_1 norm of the target. With equicorrelated covariates the bounded-condition-number assumption fails as ss grows, while the restricted-eigenvalue lower bound holds. Our reply to Reviewer LaTM (2/3) works out this example.
  • Convexity. The normal-score matrix is positive semidefinite by construction, which the Kendall and Spearman matrices need not be, so the penalized quadratic programs of Section 4 are convex without a projection step.
  • Finite samples. Rates of the same order do not mean the same accuracy. In the new experiments of Section 4.3 the normal-score matrix is closer to the latent oracle than the Kendall and Spearman matrices: at n=1600n=1600 their max-norm distance to the oracle is 4.5 to 6.2 times that of the normal-score matrix; their mean max-norm error with respect to Σ\Sigma exceeds the oracle's by 3.3% to 11.5%, against at most 3.1% for normal scores; and in regression their error exceeds the oracle's by up to 9%, against at most 5%.

"… a significant amount of notation that is not always defined before it is used." The abstract now defines the sparsity level ss where it first appears. Section 2.1 (p. 3) now states that ZZ and XX also denote the n×pn\times p data matrices and that Z⋅jZ_{\cdot j} and X⋅jX_{\cdot j} are their jjth columns.

Reviewer_QKJ2OpenReview ↗

Response to rebuttal

We thank the authors for responding to our questions and addressing our concerns, in particular to compare the proposed rates with those already available in the literature. The paper provides a modest improvement in the poly-logarithmic rate. At the moment we would like to keep our score.

Comments on the paper

3 comments
AuthorsOpenReview ↗

General response (1/3): what changed, and the strongest earlier baseline

We thank the three reviewers for their careful reading. The reviews share one question: what does the paper add to existing results, both for the normal-score estimator itself and relative to Kendall- and Spearman-based methods? In short: Theorem 2.1 removes the factor (log⁡n)2+log⁡p(\log n)^2+\sqrt{\log p} from the earlier sufficient condition for restricted eigenvalues and is nonasymptotic (part A); relative to Kendall and Spearman, the rates are of the same order, the assumptions differ, and in the new experiments the normal-score matrix is closer to the latent oracle (parts B and C). Each reply can be read on its own; this response lists the changes and holds the two tables they cite.

Revision. The uploaded PDF differs from the submission in the places below; added text is marked in blue. Theorem, lemma and equation numbers are unchanged.

  1. Abstract. The sparsity level ss is now defined; the restricted-eigenvalue statement now reads "has restricted eigenvalues bounded away from zero whenever Σ\Sigma does"; and the comparison with the generic s2log⁡ps^2\log p conversion is removed.
  2. Introduction, p. 2. The remainder bound of Zhao & Genest (2019, Theorem F.1) is now written out, with the range of dimensions in which it does not tend to zero, and one sentence states the restricted-eigenvalue (RE) condition that follows from it.
  3. Section 2.1, p. 3. One sentence states the data-matrix notation: ZZ and XX also denote the n×pn\times p data matrices, and Z⋅jZ_{\cdot j}, X⋅jX_{\cdot j} their columns.
  4. Section 4.3, p. 10, with Figure 1, p. 9 (new). Two simulation experiments: the max-norm distance of the normal-score, Kendall and Spearman matrices to the latent oracle as nn grows, and the regression program with the four matrices.
  5. Discussion, p. 10. One paragraph compares our guarantees with those for the Kendall and Spearman matrices; the roadmap at the end of the Introduction points to it.
  6. Remark B.2, p. 16. The derivation behind item 2.
  7. Appendix E, pp. 19–21 (new). Details of the simulations, with Figure 2. The AI-use and reproducibility statements (p. 11) are updated to match.

A. The strongest earlier baseline (Reviewers LaTM and zWF6). The relevant baseline is the earlier normal-score expansion, not the generic conversion. Under the hypotheses of Theorem F.1 of Zhao & Genest, subtracting the oracle expansion of our Lemma 3.5 cancels the common leading term, and the cone inequality then turns the remainder into an RE perturbation. Fix the population RE lower bound κ0\kappa_0, the cone constant c0c_0 and the relative accuracy η\eta; the constant AA in the last row depends only on these. The entrywise bounds hold with high probability.

RouteEntrywise controlSufficient condition for RE preservation
Generic conversion of a population max-norm bound∥Σ^NS−Σ∥max⁡≲log⁡p/n\Vert\widehat\Sigma^{\rm NS}-\Sigma\Vert_{\max}\lesssim\sqrt{\log p/n}n≳s2log⁡pn\gtrsim s^2\log p
Theorem F.1 + oracle expansion (Lemma 3.5)∥Σ^NS−R^G∥max⁡≲log⁡p(log⁡n)2+(log⁡p)3/2/n\Vert\widehat\Sigma^{\rm NS}-\widehat R^G\Vert_{\max}\lesssim\\{\log p(\log n)^2+(\log p)^{3/2}\\}/nn≳slog⁡p(log⁡n)2+log⁡pn\gtrsim s\log p\\{(\log n)^2+\sqrt{\log p}\\}, under the hypotheses of F.1
This paper (Theorem 2.1)∥Σ^NS−R^G∥max⁡≲(log⁡p+log⁡log⁡n)/n\Vert\widehat\Sigma^{\rm NS}-\widehat R^G\Vert_{\max}\lesssim(\log p+\log\log n)/nn≥A,slog⁡pn\ge A\\,s\log p

The earlier expansion therefore already gives a condition that is linear in ss, with the additional factor (log⁡n)2+log⁡p(\log n)^2+\sqrt{\log p}. Theorem 2.1 removes this factor. The s2log⁡ps^2\log p figure of the submission is the first row: the generic conversion, not a lower bound for the estimator. The Introduction and Remark B.2 now state the second row, and the abstract no longer makes the s2log⁡ps^2\log p comparison. The reply to Reviewer LaTM compares rate, probability and covariance assumptions item by item.

AuthorsOpenReview ↗

General response (2/3): comparison with Kendall- and Spearman-based results

B. Comparison with Kendall- and Spearman-based results (all reviewers). The table lists each result with the assumptions of its formal statement.

ResultGuaranteeAssumptions beyond sparsity
Liu et al. (2012), Thms 4.1–4.2max-norm rate log⁡p/n\sqrt{\log p/n} for the Spearman and Kendall sine matricesno upper-growth restriction on the dimension
Cai & Zhang (2018), Thm 1regression ℓ2\ell_2 rate slog⁡p/n\sqrt{s\log p/n}slog⁡p=o(n)s\log p=o(n); population restricted strong convexity of ΣXX\Sigma_{XX}; uniformly bounded condition numbers of the s×ss\times s principal submatrices of the joint latent correlation matrix
Barber & Kolar (2018), Cor 3.8 (transelliptical models)rate klog⁡p/n\sqrt{k\log p/n} for their initial regressions, computed under an ℓ1\ell_1-ball constraint of radius of order k\sqrt kbounded global condition number; kk-sparse relevant precision columns; n≳klog⁡pn\gtrsim k\log p; no additional O(1)O(1) bound on the target ℓ1\ell_1 norm
This paper, Cor 2.7max-norm rate log⁡p/n\sqrt{\log p/n} for Σ^NS\widehat\Sigma^{\rm NS}none: every n≥3n\ge3 and p≥2p\ge2
This paper, Prop 2.8the rate of Cor 2.7 is sharpΣ=Ip\Sigma=I_p; n,p→∞n,p\to\infty with log⁡p=o(n)\log p=o(n)
This paper, Cor 2.4 and Cor 4.1RE preservation at n≥A,slog⁡pn\ge A\\,s\log p; regression ℓ2\ell_2 error at most C(1+B)κ0−2slog⁡p/nC(1+B)\kappa_0^{-2}\sqrt{s\log p/n}population RE lower bound κ0\kappa_0 as the only spectral assumption; target ℓ1\ell_1 norm at most BB

Four conclusions follow.

  • Max-norm rate. The rate log⁡p/n\sqrt{\log p/n} now holds for normal scores in the same generality as for Kendall and Spearman. The earlier remainder bound gave it for normal scores only when log⁡p≲n\log p\lesssim\sqrt n.
  • Sparse regression. The rates are of the same order, and the assumptions differ: the Kendall-based theorems above assume bounded (sparse) condition numbers, whereas Corollary 4.1 assumes a population RE lower bound as its only spectral condition, and a bounded target ℓ1\ell_1 norm.
  • Finite samples. Rates of the same order do not mean the same accuracy: in the experiments of Section 4.3 the normal-score matrix is closer to the latent oracle than the Kendall and Spearman matrices (part C).
  • Convexity. The normal-score matrix is a Gram matrix, hence positive semidefinite, which the Kendall and Spearman matrices need not be, so the quadratic programs of Section 4 are convex without a projection step.

The Discussion (p. 10) now states the first three comparisons. A concrete setting that separates the assumptions of Corollary 4.1 from those of Cai & Zhang's Theorem 1 is in the reply to Reviewer LaTM.

AuthorsOpenReview ↗

General response (3/3): simulations

C. Simulations (Reviewers zWF6 and LaTM). The new Section 4.3 reports two paired experiments with four matrices: latent oracle, normal score, Kendall and Spearman. Both use two designs, AR(1) and equicorrelation. Appendix E gives the details. The normal-score matrix has the smallest mean max-norm error with respect to Σ\Sigma of the three rank-based matrices in all 20 settings of the first experiment, and its regression error is within 5% of the oracle's in every cell of the second.

Distance to the oracle (Figure 1a). For p∈50,200p\in\\{50,200\\}, as nn grows from 100 to 1600 the mean of ∥Σ^NS−R^G∥max⁡\Vert\widehat\Sigma^{\rm NS}-\widehat R^G\Vert_{\max} falls by a factor of 14 to 16, and its product with n/(log⁡p+log⁡log⁡n)n/(\log p+\log\log n) stays between 1.17 and 1.35 in all 20 settings, in agreement with Theorem 2.1. The Kendall and Spearman distances to the oracle fall by a factor of 4.4 to 5.3, and at n=1600n=1600 they are 4.5 to 6.2 times the normal-score distance. The max-norm errors with respect to Σ\Sigma behave accordingly: the mean error of the normal-score matrix exceeds the oracle's by at most 3.1%, those of the other two matrices by 3.3% to 11.5%.

Regression (Figures 1b and 2). The regression program of Section 4.1 is run with p=200p=200, s∈5,10,20s\in\\{5,10,20\\}, and n/(slog⁡p)n/(s\log p) from 2 to 32; the target has fixed ℓ1\ell_1 norm 0.80.8 and ℓ2\ell_2 norm above 0.40.4. Figure 2 shows each method's mean ℓ2\ell_2 error divided by the oracle's, with paired standard errors.

  • Under five-fold cross-validation, the normal-score ratios lie in [0.96,1.03][0.96,1.03]; they exceed one by more than two standard errors in 1 of 30 cells. The Kendall and Spearman ratios lie in [0.96,1.08][0.96,1.08] and exceed one by that margin in 24 of 60 cells.
  • Under simulation-oracle tuning, which selects the best value on each method's path using the true coefficients, the ranges are [0.99,1.05][0.99,1.05] and [0.99,1.09][0.99,1.09].
  • The oracle's own mean error falls from 0.200.20–0.460.46 at n/(slog⁡p)=2n/(s\log p)=2 to 0.090.09–0.170.17 at n/(slog⁡p)=32n/(s\log p)=32.

In every cell and under both tuning rules, the mean error of the normal-score estimate exceeds the oracle's by at most 5%; for the Kendall and Spearman estimates the excess is up to 9%. Each count above compares one method with the oracle.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.