- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper introduces C-PCR, which extends PCA++ to incorporate response information into component selection. Under a linear paired-factor model with shared latent factors across two views of each unit and outcomes depending linearly on those factors, the authors characterize when supervised selection retains a predictive direction that pair-only selection discards. They derive an asymptotic prediction-risk formula for one retained component, extend the analysis to multiple responses and components, and evaluate the method on five paired spectroscopy datasets. The method is potentially useful, and the theoretical analysis provides insight into its prediction risk. However, the methodological contribution appears to be a relatively direct supervised extension of PCA++. The substantial tuning burden associated with four hyperparameters, limited breadth of the experimental evaluation, and modest real-data gains over the competing methods limit the significance of the contribution.
Critical strengths
• A useful supervised extension of PCA++ for naturally paired measurements. • A multi-response version (Appendix D) that could serve as an alternative to PLS. • Theoretical investigation and asymptotic prediction-risk characterization.
Critical weaknesses
• Limited methodological novelty. The estimator adds a response-association term to the PCA++ criterion; it seems to be a direct extension of PCA++. • Table 1 does not clearly demonstrate an advantage. C-PCR is best on 3 of 5 datasets, by ≤ 0.005 relative MSE on Raman and Beef. Mango is the only dataset with a clear gain over PCA++, but the performance is still very close to Ridge. • Tuning burden and practicality. The method has four tuning parameters (q, λ, τ, ρ), and preprocessing is additionally selected by validation on four of the five datasets. Table 7 reports more than 100,000 candidate evaluations per fitting split for C-PCR on four datasets, versus far fewer evaluations for PLS and PCR. It is mentioned that fits are reused, but no runtime is reported, and I doubt the practicality for larger datasets. • Simulations are narrow relative to the real data. All simulated settings have p/(2n) ≤ 1 (maximum γ = 1 in Fig. 2b; γ = 1/3, 0.2 and 0.5 elsewhere), whereas Raman, Corn and Tablets have γ > 1.
Questions for authors
• As stated in the weaknesses, the authors should expand the simulation studies to investigate the performance of the method in diverse scenarios. • The computational burden of the tuning parameter selection should be clearly reported. • When I looked at Table 6, the fitting dataset size is quite small compared with the validation and test sets (e.g., Raman: 50, 150, 1088). If more samples were used for training, so that n increases and γ decreases, I wonder whether the relative performance among the competing methods would change.
6 comments
Continued: Against both, the 95% intervals over splits lie above zero on four of the five datasets (Table 13), including the small margins on Raman and Beef; on Corn, both baselines are better (ridge by a median of 2.1%, no-pair by 0.1%). On Corn, C-PCR thus falls back almost to no-pair, as designed: because lies in its grid, validation can return the supervised fit, so the method should match the best supervised method when the pairs carry no extra information and improve on it when they do. These intervals reflect variability over random splits, not sampling of the dataset itself. Across all 18 datasets (Tables 12 and 13), C-PCR is better than each of the nine comparators below on most datasets (one-sided Wilcoxon tests over datasets, each):
| C-PCR against | Better / worse | |
|---|---|---|
| PCR | 18 / 0 | |
| PLS | 15 / 3 | |
| Ridge | 15 / 3 | |
| Elastic net | 14 / 4 | 0.01 |
| No-pair | 14 / 4 | |
| Ji et al. (linear) | 17 / 0 | |
| PCA++-PCR | 15 / 0 | |
| PCA++, then supervised screening | 16 / 2 | |
| PCA++ with no-pair directions | 12 / 6 | 0.03 |
Appendix F.6 documents the protocol of the original fixed splits, including how the Raman preprocessing and the Tablets fit were selected.
Tuning burden and practicality. The method has four tuning parameters (, , , ), and preprocessing is additionally selected by validation on four of the five datasets. Table 7 reports more than 100,000 candidate evaluations per fitting split for C-PCR on four datasets, versus far fewer evaluations for PLS and PCR. It is mentioned that fits are reused, but no runtime is reported, and I doubt the practicality for larger datasets. The computational burden of the tuning parameter selection should be clearly reported.
The candidate count overstates the cost, because almost all candidates reuse earlier work. The search reuses work at every level (Result B.5, Algorithm 2; ):
| Step | How often | Cost each |
|---|---|---|
| SVD and projections | once | |
| Eigenproblem | once per pair | |
| Refit and validation | once per candidate |
The refit cost does not depend on and grows only linearly with the validation size , so for large data, where , T=O\bigl\\{(n+n_{\rm val})\\,p\\,r+N_{\rm eig}\\,r^3\bigr\\},\qquad N_{\rm eig}=\text{number of }(\tau,\lambda)\text{ pairs}. Thus enters only through the SVD, linearly, and the large candidate count multiplies only solves.
Preprocessing is chosen on validation in the same way for every method, from three options (Appendix F.1). Each option repeats the same search, so it triples the cost; this is why four of the five datasets exceed 100,000 candidates. The runtimes below include it.
On one core, the median full search takes 0.9 to 14.4 s per split across the 18 datasets, and at most 16.6 s on the four largest (up to 2,866 training pairs). On larger simulated data, the times per split in seconds are, including the reduced C-PCR (rC-PCR), which fixes and (261 candidates):
| C-PCR | rC-PCR | No-pair | PLS | Ridge | |
|---|---|---|---|---|---|
| 8.0 | 1.6 | 1.5 | 3.3 | 1.0 | |
| 39.8 | 5.7 | 5.7 | 3.8 | 2.6 |
rC-PCR runs about as fast as no-pair, and on the five datasets of Table 1 it still beats no-pair on four, keeping the full gain on Beef and Mango but less of it on Raman and Tablets (Appendix F.1). In the other direction, with no-pair given as many candidates as C-PCR, C-PCR still wins on Raman, Tablets and Mango; on Beef the difference is no longer significant, and on Corn no-pair is better (Appendix F.1). Beyond these sizes, the eigenproblems, each , dominate once reaches several thousand. rC-PCR needs only one per value of . The cost formula is now in Appendix B.3, and the runtimes are in Table 11 and Appendix F.1.
Simulations are narrow relative to the real data. All simulated settings have (maximum in Fig. 2b; , 0.2 and 0.5 elsewhere), whereas Raman, Corn and Tablets have . As stated in the weaknesses, the authors should expand the simulation studies to investigate the performance of the method in diverse scenarios.
We extended the simulations to in three scenarios: complementary signals (S1), a shared factor plus variation in only one view (S2), and the model of Theorem 4.2 (S3). The natural datasets now span from 0.005 to 13.5. All methods are tuned on validation, and the lower rows break the assumptions at . Mean rMSE reduction over no-pair (%):
| Setting | S1 | S2 | S3 |
|---|---|---|---|
| +9.3 | +3.7 | +0.5 | |
| +2.5 | +2.7 | +0.9 | |
| +0.6 | +1.4 | +1.3 | |
| +0.3 | +0.8 | +1.2 | |
| noise | +2.3 | +2.4 | +1.7 |
| AR(1) noise | +9.7 | +4.0 | +8.2 |
| Unequal view scales | +0.6 | +1.6 | +1.8 |
| 50% mismatched pairs | +0.2 | +0.4 | +0.5 |
These simulations are Appendix E.4 (Tables 5 and 6):
-
C-PCR wins in at least 131 of 150 simulated datasets per , with the 95% interval of the mean difference above zero in every setting;
-
the gain shrinks as grows in S1 and S2, to +0.3% and +0.8% at , but stays significant, whereas in S3 it stays between +0.5% and +1.3% at every ;
-
it has the lowest risk of all six methods in 11 of 12 settings, tying the linear objective of Ji et al. in the last;
-
when all pairs are broken, it falls back to no-pair in S1 and S2 and is 1.9% worse in S3.
Separately, the risk formula stays within 0.21% of simulation up to (Appendix E.2).
When I looked at Table 6, the fitting dataset size is quite small compared with the validation and test sets (e.g., Raman: 50, 150, 1088). If more samples were used for training, so that increases and decreases, I wonder whether the relative performance among the competing methods would change.
In theory the absolute gain vanishes with enough training data, but on two of the three datasets we could test, it persists at every available size. Pairing helps only by reducing estimation error, which shrinks as grows, and Result C.2 makes this precise: as goes to zero, the limiting risk tends to the same population risk for every , so with fixed, no-pair eventually catches up in risk, although the gain relative to the shrinking excess risk can remain large (Appendix E.6). We enlarged the training set on the three datasets with the largest pools, keeping the validation size fixed and testing on the remaining units (now Appendix F.7):
| Dataset | Gain in mean rMSE vs no-pair (%) | Splits won | ||
|---|---|---|---|---|
| Tablets | 450 | 0.72 | +11.6 | 60/60 |
| Mango | 300 | 0.46 | +6.6 | 47/60 |
| Raman | 200 | 1.76 | 20/60 |
On Raman, no-pair and ridge already slightly overtake C-PCR at , by 0.7% to 0.8%, whereas on Tablets and Mango the gain persists at the largest training sets we can form.
What C-PCR adds beyond PCA++ and consistency ridge, with 150 random splits, 18 datasets, runtimes and simulations up to γ = 4
We thank the reviewer for the careful review. The concerns come down to two questions: is C-PCR more than PCA++ with a response term, and are its gains real and practical? We answer the first below. For the second, we ran the larger experiments the reviewer asked for: 150 random splits per dataset instead of the original fixed splits, 13 more datasets, simulations up to γ=4, and timings up to p=4000.
C-PCR does add a response term to the PCA++ criterion, but the term changes where the response enters: at truncation, not after it.
PCA++-PCR keeps q directions without the response and only then regresses, so a direction dropped at truncation cannot be recovered, and a weakly shared but predictive direction is exactly what unsupervised truncation drops.
Retaining more directions avoids this only with a wider representation: in Appendix E.5, PCA++-PCR needs q0≥32 pair directions before supervised compression to come close to C-PCR, which still has lower risk (Table 7).
C-PCR, in contrast, uses the response in the truncation itself, so it keeps the predictive direction with a single component.
Both criteria are generalized eigenproblems in the same metric, PCA++U⊤ΣτU=Iqmax⟨Σct,UU⊤⟩vsC-PCRU⊤ΣτU=Iqmax⟨bb⊤+λΣct,UU⊤⟩, and the added term has three consequences.
First, the added term changes which direction survives truncation. If the predictive direction u1∗ is weaker in the pair term than a shared distractor u2∗ (η2>η1), Proposition 3.1 gives PCA++ selects u2∗for every τ⟹R=Var(y)for every τ,ρ,λ<λc⟹C-PCR selects u1∗, so PCA++ misses the predictive direction under every tuning, while C-PCR finds it; with each method at its own optimum, C-PCR’s risk is lower by a1ϑ2/(1+a1).
Second, it changes the selected direction itself. For one component, C-PCR selects the direction of a ridge fit that also penalizes disagreement between the two views, with penalties set by the selected eigenvalue (Proposition B.8, Appendix B.6). The connection explains why C-PCR works: the pair term acts as a penalty on disagreement between the two measurements of a unit. Because the penalties are set by each component’s eigenvalue, directions with a strong response signal are penalized less and weak ones are pushed harder toward agreement, and, whereas consistency-regularized reduced-rank regression fixes only L directions and leaves the rest arbitrary, C-PCR ranks the components beyond the response rank by pair alignment (Result B.9); these components carry most of C-PCR’s gain on Raman (Table 14).
Third, it has its own risk theory: Theorem 4.2 and Proposition D.4 give the exact limiting risk of the full procedure, in which the same responses both select the direction and fit its coefficient. PCA++ theory does not account for this.
With 150 random splits per dataset, C-PCR is significantly better than both no-pair and ridge on four of the five datasets, and across all 18 datasets it is better than every comparator on most datasets. We agree that the original fixed splits could not separate margins of this size. Median rMSE reduction of C-PCR against the two strongest baselines (%):