- Rating
- 1(−1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper proposes attenuated PCA, which interpolates between ordinary PCA and diagonal-deletion PCA by computing the leading eigenspace of
The endpoints and recover ordinary PCA and diagonal deletion, respectively. Proposition 2.1 derives the range from a masked-input, full-output reconstruction objective, with for mask rate . The authors then extend the attenuation range directly to . The evaluation target is clean reconstruction risk relative to the population PCA subspace.
The main theory assumes Gaussian observations, fixed dimension and rank , and a positive population eigengap as sample size grows. Under balanced marginal variances, attenuation preserves the population subspace and yields an explicit quadratic leading risk curve. Further results account for changing eigengaps and population-subspace movement; the appendix also gives a finite-moment extension for balanced marginals. Controlled simulations examine these risk predictions, and real-data studies compare cross-validated attenuation with PCA and several alternatives on image patches, speech spectra, and acceleration signals under fitting budgets of 64 and 128 observations.
Critical strengths
- Simple and interpretable estimator. The method requires a diagonal modification of the sample covariance followed by an eigendecomposition. Its candidate path includes ordinary PCA, and the parameter has a direct interpretation as the amount of diagonal attenuation.
- Explicit account of the risk tradeoff. Theorem 3.1 isolates the sampling effect under balanced marginal variances, while the general-covariance analysis accounts for changing eigengaps and population bias. Figure 2 illustrates how these effects can move the preferred attenuation strength toward either endpoint.
- Common-budget experimental comparisons. The real-data studies use the same fitting observations and validation folds across methods. Both attenuation variants have positive mean gains over PCA in all six reported data-budget settings, with the reported intervals above zero.
Critical weaknesses
-
The interpolation remains weakly motivated as a new PCA method. The proposed estimator is a one-parameter interpolation between ordinary PCA and the established diagonal-deletion estimator. The exact masking calculation provides an interpretation of the interval for a particular full-output, orthogonal-projector reconstruction objective. It does not by itself establish the practical value of extending the path to . The balanced-marginal analysis gives a favorable setting in which attenuation leaves the population target unchanged. For general covariances, the diagonal can contain directional information, and attenuation can introduce population bias. Although the paper analyzes this tradeoff, it provides limited guidance on how a practitioner could recognize when partial attenuation should help. The additional value of this interpolation therefore needs stronger justification through identifiable conditions or evidence of a meaningful advantage over both endpoints.
-
A central weakness is the gap between the theoretical regime and the main small-sample signal experiments. Assumption 2.2 fixes and as grows. Theorems 3.1 and 3.2 provide uniform risk expansions for sufficiently large sample sizes, with constants and thresholds that can depend on the population problem. They do not quantify whether the approximation or improvement condition is informative at the reported experimental budgets. In Phoneme, with ; in HAR, with the same budgets. Thus ranges from 1 to 4 in these studies, and the fitting samples within validation folds are smaller still. A result with fixed dimension and sufficiently large does not establish a guarantee for these dimension-to-sample-size ratios.
The controlled Gaussian simulations use and support the perturbation analysis in low-dimensional settings. They do not bridge the gap to the signal experiments. The finite-moment extension likewise retains fixed dimension. In addition, Proposition D.2 selects among projectors fitted on one training sample using an independent validation sample, without refitting; it does not cover the five-fold cross-validation followed by refitting used in the real-data studies. The issue is the missing quantitative connection between the stated theory and the regimes in which the practical benefit is demonstrated. This limits the explanatory force of the theory for a method presented as a finite-sample PCA correction.
-
The empirical gains are modest, and the value beyond fixed diagonal deletion is unclear. Figure 3 and the appendix tables report that direct attenuation removes and of PCA's excess reconstruction risk on Optdigits, and on Phoneme, and and on HAR, at budgets 64 and 128 respectively. The consistency is encouraging, but the signal-data gains are small. Equation (4.1) normalizes by PCA's excess risk above a test-set reference, so these percentages do not describe equivalent reductions in total reconstruction error. Raw errors and reductions relative to total PCA error would make the practical magnitude easier to assess.
The real-data comparisons also omit the standalone performance of fixed diagonal deletion at . This matters because the selected strengths are often large: their means on Optdigits are 0.927 and 0.931 (Table 3). Showing that the full path improves over the mask-derived range does not isolate the benefit of tuning the interpolation over using the known diagonal-deletion endpoint. A paired endpoint comparison, with uncertainty, is needed to assess the empirical contribution of the additional flexibility.
-
The AI-use statement does not sufficiently specify the scope of assistance. The statement on p. 10 reports assistance with literature discovery and synthesis, proof cross-checking, code review, and language editing, together with author verification of sources, derivations, and computations. It does not clarify whether proof cross-checking only assessed completed arguments or also supplied proof steps, whether code review included implementing or substantively modifying methods or experiments, or whether AI contributed to formulating claims, designing experiments, or interpreting results. These distinctions matter for understanding AI's role in the central theoretical and empirical contributions. The authors should describe the scope of assistance and verification more concretely, including explicit statements of nonuse where needed to remove ambiguity.
Questions for authors
- Motivation and practical conditions: Under what practically identifiable conditions should an interior attenuation strength improve over both ordinary PCA and fixed diagonal deletion? How can these conditions be assessed from limited data, given the potential population bias?
- Connection between theory and experiments: What quantitative support is available for the approximation and improvement conditions at the ratios used in Phoneme and HAR? Controlled experiments varying and comparing predicted with observed risk would help establish the range in which the analysis is informative. Please also clarify the scope of Proposition D.2 relative to the implemented cross-validation-and-refitting procedure.
- Magnitude and attribution of the gains: Please report fixed using the existing fitting budgets and subsamples, along with paired differences and uncertainty relative to selected attenuation. Please also report raw reconstruction errors or reductions relative to total PCA reconstruction error, particularly for Phoneme and HAR.
- Completeness of AI disclosure: Please clarify whether AI contributed to formulating theoretical claims, developing or writing proofs, implementing methods, designing experiments, or interpreting results. Distinguish these activities from proof checking, code review, literature assistance, and language editing, and describe the verification performed for any substantive contributions.
6 comments
The authors are working on the rebuttal
The authors are still working on the rebuttal - we will likely submit the rebuttal in a day or so.
Official response
Thank you for identifying the endpoint comparison and for asking us to clarify the scope of the theory and AI assistance.
1. When partial attenuation helps
When an interior strength helps. Attenuation can increase the separation of retained sample eigenvalues from the noise spectrum while also moving the population PCA subspace. The new Theorem 3.1 in the revised manuscript now quantifies the resulting excess reconstruction risk for Gaussian data with a fixed coordinate signal block and separated sample outliers. In the two-spike example at , moving from partial attenuation to full deletion adds more population reconstruction cost than it saves in the sampling contribution to risk (Figure 2).
Comparison with both endpoints. Table 1 reports the expected rank-two reconstruction error above that of population PCA (Eq. 2.2). In the simulation row, all three strengths use the same 100 datasets.
| Setting | PCA () | Partial () | Full deletion () |
|---|---|---|---|
| Proportional limit | 0.2952 | 0.2611 | 0.2935 |
| , | 0.2951 | 0.2603 | 0.2930 |
The limiting risk at is 11.6% lower than PCA and 11.1% lower than full deletion.
In practice. Proposition 2.2 gives two checks for exact target preservation. The population diagonal must leave the PCA subspace invariant, and the leading block needs to retain a positive gap. A large gap and a small cross-block component of the population diagonal can keep the population cost small. We can compare held-out reconstruction error or downstream prediction accuracy along a grid that includes both endpoints.
2. Proportional risk and Proposition E.1
Multiple spikes in proportional dimensions. Phoneme and HAR use fitting-budget ratios from 1 to 4. When grows with , estimation error across the noise directions can contribute a nonzero limiting risk. The revised Theorem 3.1 accounts for this contribution when under a Gaussian model with a fixed signal block, identity covariance for the noise coordinates, and simple, separated sample outliers. The formula uses each retained outlier and the direction and squared length of its eigenvector's projection onto the signal block. Appendix C.1 proves the proportional limit, and Appendix D covers the fixed-dimensional regime.
Table 1 and Figure 2 compare the risk limit with paired simulations at . At and , the mean excess reconstruction risk at is 0.2603, close to the limit of 0.2611. This example illustrates the risk comparison at one aspect ratio under the stated covariance model. Section 4.2 evaluates Phoneme and HAR directly on held-out data at their larger ratios.
Scope of Proposition E.1. This proposition selects from projectors fitted once on a single training set, using an independent Gaussian validation sample. It bounds the selected candidate's population reconstruction risk by the best candidate's risk plus a validation error term. The guarantee concerns the selected projector before any refitting. In Section 4.2, we use five-fold reconstruction cross-validation with fold-specific fits, then refit at the selected strength on all observations. We evaluate the refitted procedure on the official test sets.
3. Fixed diagonal deletion and raw error
Raw errors and deletion. Section 4.2 and Table 6 compare fixed diagonal deletion () with tuned direct attenuation on the same fitting subsamples and test sets. The table below also gives each method's percentage reduction in total PCA error. The last column gives the tuned-minus-fixed difference in percentage points of PCA test excess risk removed (Eq. 4.1). Positive values favor tuning. The paired intervals are conditional on the observed data and official splits.
| Data | PCA error | Selected error | Fixed error | Selected total gain | Fixed total gain | Selected − fixed gain [95% CI] | |
|---|---|---|---|---|---|---|---|
| Optdigits | 64 | 140.1459 | 139.6910 | 139.6218 | 0.322% | 0.370% | -0.81 [-1.07, -0.54] |
| Optdigits | 128 | 136.7641 | 136.5522 | 136.5313 | 0.154% | 0.170% | -0.52 [-0.69, -0.36] |
| Phoneme | 64 | 692.9732 | 691.8562 | 691.7833 | 0.161% | 0.172% | -0.08 [-0.21, +0.06] |
| Phoneme | 128 | 648.7325 | 648.3239 | 648.3326 | 0.063% | 0.062% | +0.01 [-0.14, +0.15] |
| HAR | 64 | 1.4562 | 1.4522 | 1.4506 | 0.282% | 0.393% | -0.30 [-0.50, -0.10] |
| HAR | 128 | 1.2569 | 1.2526 | 1.2512 | 0.341% | 0.453% | -0.38 [-0.56, -0.20] |
Tuned direct attenuation reduces total PCA reconstruction error by 0.063% to 0.341% across the six settings. Fixed deletion also improves on PCA on average in all six settings. It has lower mean error than tuned direct attenuation at both budgets on Optdigits and HAR, with paired intervals favoring deletion. Both Phoneme intervals include zero.
Where is the author's rebuttal?
Since the author did not provide a rebuttal, I have lowered my original score.