- Rating
- 1
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
The paper proposes an estimator for covariance matrices that admits a piecewise smooth structure. The main idea is to regularize this smoothness by penalizing the so-called masked second divided differences via a folded-concave penalty. The optimization problem is solved using the ADMM algorithm, and both theory and simulations are presented to illustrate the properties of the estimator. While this is a well-executed work that leverages existing approaches from the literature, the particular combination brings limited advantage, which is a major shortcoming of the work.
Critical strengths
This is a nicely implemented approach that builds on a well-known line of work in the relevant area. All the important components of the approach are already established in the literature, so the paper reads coherently. I did not read the proofs carefully, but I believe most of the claims are similar to those already present in the literature.
Critical weaknesses
-
Limited novelty and no demonstrated added value. The paper combines well-established ingredients such as covariance regularization, masked second divided differences, the folded-concave penalty, and ADMM, but does not go beyond this combination to show what it uniquely contributes. There is no ablation or comparison against the individual components, so it remains unclear why this particular combination is necessary or preferable to existing alternatives.
-
Incomplete numerical work and inherited theory. The data analysis reads as incomplete, with insufficient baselines and no isolation of each modeling choice, making the method appear to win by assertion rather than evidence. Similarly, the theoretical results appear to follow closely from existing literature, without clearly identifying which claims are genuinely new or how the theory addresses the specific challenges of the proposed structure.
Questions for authors
-
Is there a specific regime, application, or data setting where the combination of the folded-concave penalty with masked second divided differences provably outperforms the individual components—and if so, can the authors point to it directly?
-
What is the strongest evidence in the paper that this combination is necessary rather than incidental, i.e., that a simpler alternative would fail?
-
If the novelty is not in the components but in the combination, what is the precise problem for which this combination is the right tool, and why do existing combinations not solve it?
-
How do you ensure the positive definiteness of the resulting estimator which is a covariance matrix?
5 comments
On W2: Incomplete numerical work and inherited theory.
We thank the reviewer for pointing out these two concerns. We understand the reviewer’s desire to see more complete numerical evidence and a clearer boundary of theoretical novelty.
On the numerical work and baselines. The current manuscript does not compare against only a single baseline. In Table 1, we compare SCM, SRCov, LRCov, and CholCov, covering , two covariance families (full-rank lag and low-rank piecewise-linear), and 100 Monte Carlo simulations. On the full-rank lag family, our method outperforms the competing methods at all dimensions. The appendix also reports held-out NLL and roughness, which show that our method not only reduces the Frobenius error but also better recovers the piecewise-smooth structure. The real electricity-load experiment also provides out-of-sample predictive evidence. Therefore, “insufficient baselines” is not accurate.
As for “no isolation of each modeling choice,” we agree that component ablations would further strengthen the argument. We note that the core contribution of this paper is the piecewise-smooth modeling and statistical guarantee for symmetric PSD covariance matrices, rather than the isolated effect of individual components; the ablations will serve as supplementary evidence to further demonstrate the necessity of each modeling choice, but will not change the positioning of our contributions.
On the theoretical results. We disagree with the judgment that “theory appears to follow closely from existing literature.” The theoretical object of this paper is a nonconvex, PSD-constrained covariance estimation problem with row-column difference coupling, and existing theorems for scalar trend filtering or graph trend filtering cannot be directly applied. Specifically:
-
We construct a masked second-order divided difference operator , whose mask excludes the central variance , thereby avoiding duplicate row/column penalization on the space of symmetric matrices while preserving near-diagonal dependence information. The corresponding oracle subspace, effective dimension , and support-dependent Green scale $ g_{\mathcal{A}}=\max_{q\in\mathcal{P}\setminus\mathcal{A}}\left|L_{\mathcal{P}\setminus\mathcal{A}}^{+}e_q\right|_{\mathrm{F}} $ are all defined for this masked operator and are used to calibrate the dual noise on off-support coordinates. These quantities are not readily available in the existing covariance estimation literature.
-
Under the nonconvex folded-concave penalty and the PSD constraint, we establish conditions for ADMM to converge to a first-order stationary point, including $ 0<\eta<1-\mu\kappa_K,\qquad \tau>2\mu+\frac{\mu^2\kappa_K}{\eta}, $ where characterizes the weak concavity of the penalty and . Because of the PSD normal cone and the symmetry coupling, this convergence analysis cannot be directly derived from existing scalar trend filtering results.
-
We provide an oracle inequality and a finite-sample rate for a piecewise-linear covariance surface, $ \left|\hat{\Sigma}-\Sigma^{\star}\right|_{\mathrm{F}}^2 \lesssim \frac{\overline{K}+\log p+C_L^2\zeta^2K\log p}{n}, \qquad \overline{K}\leq K+\nu, $ which shows that the estimator adapts at a near-parametric rate to the unknown slope-change support, with each unknown slope change costing only . The key to this result is the combination of the masked operator, the weak convexity of the nonconvex penalty, the PSD normal-cone condition, and the support-dependent Green scale, rather than transplanting existing theorems verbatim.
Q1. Is there a regime where the combination provably outperforms the components?
There is a regime we can name, explain and measure, though not one where we can prove dominance by theorem, and we do not claim that. It is ordered data with heterogeneous marginal scales: load curves, yield curves across maturities, spectral profiles.
The mechanism is an identity, not a heuristic. On a uniform grid of spacing , the unmasked stencil centred at contains the coordinate
$ (L_{\mathrm{unmasked}}\Sigma)_ {(c,c)}=\frac{1}{2h^2}\left(\Sigma_ {c,c-1}-2\sigma_ {cc}+\Sigma_ {c,c+1}\right), $
so the variance enters that coordinate with coefficient , twice the weight of either neighbour and, under heteroscedasticity, the largest magnitude in the row. A penalty on that coordinate is therefore, to leading order, a penalty on the marginal variance itself. The larger the scale heterogeneity, the more of the curvature budget is spent flattening the scale profile rather than the dependence profile.
Q2. What is the strongest evidence that the combination is necessary rather than incidental?
We thank the reviewer for this question. We believe that the strongest evidence in the paper is not any single isolated experiment, but rather a converging chain of evidence formed by the theoretical guarantees in the target regime, experiments on data with the true structure, and the failure mechanisms of the individual components. Together, they show that this combination is not an incidental concatenation, but is necessary for the specific structure of “sparse slope changes on a symmetric PSD covariance surface.”
Specifically:First, the strongest theoretical evidence is that the oracle inequality and the piecewise-linear rate depend precisely on the key components of the combination. Corollary 5.4 gives $ |\hat{\Sigma}-\Sigma^{\star}|_ {\mathrm{F}}^2 \lesssim \frac{\overline{K}+\log p+C_L^2\zeta^2K\log p}{n}, \qquad \overline{K}\leq K+\nu, $ where is the effective dimension of the oracle subspace induced by the masked operator , and the Green scale is also defined for the masked operator. If the mask is removed, the row and column differences double-penalize the same linear forms and smooth the central variance , so the above oracle subspace and noise calibration no longer hold. If MCP is replaced by , the proximal map $ \operatorname{prox}_ {\ell_1}(r)=\operatorname{sign}(r)(|r|-\lambda) _+ $ systematically shrinks every nonzero slope change by , whereas MCP is unbiased when . Therefore, in this regime, the simpler alternatives are not merely “possibly slightly worse”; rather, they respectively suffer from irreducible double-penalization bias, variance-smoothing bias, or large-kink shrinkage bias.
Second, the strongest experimental evidence is the full-rank lag family. The true covariance in this family is exactly piecewise linear in lag, which matches the structural assumption of this paper. In this regime, where the method’s assumption holds, Table 1 shows that our method outperforms SCM, SRCov, LRCov, and CholCov at ; for example, at , the relative Frobenius error is 0.3261, compared with 0.3832/0.4335/0.4239 for SRCov/LRCov/CholCov. The held-out NLL and roughness results in the appendix further show that our method not only reduces the Frobenius error but also better recovers the piecewise-smooth structure. The numerical results of the real electricity-load experiment also provide out-of-sample NLL evidence. These results show that the gain of the combination appears in the regime where it should appear.
Third, the failure mechanisms of the components are explicit, not post hoc explanations. The unbiased interval of MCP directly eliminates the shrinkage of large kinks; the masked operator directly avoids row-column double penalization and preserves marginal variances; and the strict PD constraint directly guarantees that the estimate is invertible. Conversely, removing any component leads to an identifiable failure mode: shrinks large kinks, the unmasked operator smooths variances and double-penalizes, and without the PSD constraint the estimate may be non-invertible.
Response to Reviewer URuC
On W1: Limited novelty and no demonstrated added value
We thank the reviewer for raising this important concern. We understand the reviewer’s desire for a clearer statement of novelty and for validation of the necessity of each component.
Specifically:
The novelty lies not in “using these ingredients,” but in identifying and resolving a key obstacle. Covariance estimation cannot be directly handled by scalar trend filtering or graph trend filtering, because the target must be a symmetric PSD matrix. Symmetry couples the difference operators along the row and column directions: applying a difference penalty to a row simultaneously modifies Σij, and through Σij=Σji directly perturbs the structure of column j. If rows and columns are penalized separately, the same linear forms are penalized twice. The masked second-order divided difference operator proposed in this paper is designed precisely to address this coupling. By masking out the central variance Σcc, it preserves near-diagonal dependence information while allowing marginal variances to adapt freely to heteroscedastic or nonsmooth variations. This is not a direct combination of existing operators, but a dedicated construction tailored to the structure of the covariance surface.
The combination of the folded-concave penalty and the PSD constraint is also not cosmetic. The nonconvex penalty reduces shrinkage bias on genuine large slope changes, while the PSD/strict-PD constraint ensures that the estimate is invertible and therefore applicable to downstream tasks requiring the inverse covariance matrix. The ADMM algorithm, its convergence analysis, and the oracle inequality based on the slope-change support are all nontrivial results developed around this nonconvex, PSD-constrained, symmetry-coupled problem.
Experimental evidence of added value. We have compared our method with SCM, SRCov, LRCov, and CholCov in Table 1. On the full-rank lag family, our method outperforms the competing methods at p=50,100,200; for example, at p=200, the relative Frobenius error is 0.3261, compared with 0.3832/0.4335/0.4239 for SRCov/LRCov/CholCov. On the low-rank family, our method is competitive with low-rank estimators, which is consistent with the expectation that this family naturally favors low-rank covariance estimators. The held-out NLL and roughness results in the appendix further support the ability of our method to recover the piecewise-smooth structure. On the theoretical side, we establish finite-sample error bounds with a support-dependent calibration that adapts to the oracle support.
Regarding ablation. We agree that component ablations would further strengthen the argument. The current manuscript focuses on comparison with existing complete methods and on theoretically characterizing the roles of the masked operator and the folded-concave penalty. The following table reports the related ablation
Table: relative Frobenius error at p=50 over ten paired draws.
Therefore, we respectfully disagree with the assessment that the paper has “limited novelty and no demonstrated added value.” We believe that the paper makes clear incremental contributions in problem modeling, operator design, nonconvex PSD optimization, statistical theory, and experiments. We will further strengthen the statement of contributions in the revision and add the ablation experiments suggested by the reviewer.