- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper studies distributed low-rank regression when workers may have fewer observations than predictors. The authors propose a method that uses a stabilized metric based on the master's local covariance matrix. It first uses nuclear norm regularization to obtain an initial estimate, then applies SCAD regularization to reduce shrinkage and recover the rank. The paper gives prediction error bounds, convergence and rank recovery results, including for singular pooled designs. It also studies communication costs and reports synthetic and real-data experiments.
Critical strengths
1. Handling singular local designs.
The method remains well defined when local covariance matrices are singular and still allows exact singular value thresholding. This is useful when each worker has fewer observations than predictors.
2. Guarantees for the actual iterates.
The paper studies the estimates produced after a finite number of updates, rather than only an ideal solution. It gives prediction and rank recovery guarantees, including cases where the pooled design is singular and only part of the coefficient matrix can be identified.
Critical weaknesses
1. The need for the proposed metric is not clear.
A simpler isotropic metric can also stabilize the updates and allow exact singular value thresholding. In the ablation study in Appendix S40.5, adding the master's local covariance gives only small improvements, and both choices recover the correct rank.
The choice of metric also changes the regularization objective, not just the optimization steps. The paper does not clearly explain why this new objective is preferable or show a substantial benefit over the simpler alternative. This is a major concern because the metric is central to the proposed method.
2. The paper is too long and does not make its main contribution clear.
The method relies on familiar proximal gradient and spectral thresholding techniques. The paper contains many lemmas, propositions, and corollaries, including results that follow directly from standard properties of these methods. The appendix often repeats the statements and explains short proofs at unnecessary length.
As a result, it is hard to tell which results require new ideas and what the main advance over prior work is. The paper needs to focus on its most important results and explain why they matter.
Questions for authors
1. What are the main new theoretical insights?
Could the authors identify the one or two most important new results and explain what makes their proofs different from existing work on proximal methods and low-rank regression?
2. Why is the master's local covariance needed in the metric?
A simpler isotropic metric already gives stable updates and exact spectral thresholding. What clear benefit comes from adding the local covariance, especially since doing so changes the regularization objective?