- Rating
- 3(+1)
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper proposes distributed robust reduced-rank regression under a row-sparse additive response-contamination model. A block-diagonal surrogate separates local contamination adjustment from global coefficient estimation. Workers hard-threshold their residual rows and send adjusted gradients; the central machine performs a closed-form rank-constrained regression update using its local covariance. The analysis gives coefficient-error contraction and a target rate combining low-rank estimation error with a contamination term, using an oracle threshold schedule. Experiments compare D4R with distributed alternatives and a full-sample oracle under several noise distributions and contaminated public datasets.
Critical strengths
-
The estimator has a useful decomposition.
-
The coefficient-error analysis has substantive value within its assumptions.
Critical weaknesses
-
The practical tuning and experimental implementation are not sufficiently reliable as supplied. Equation (21) permits a saturated contamination estimate that absorbs every residual row, giving zero residual sum of squares and an undefined or negative-infinite log term.
-
The central efficiency claims are stronger than the demonstrated evidence. The advertised centralized term is avoidable: with a thin QR factorization , the reduced-rank step can use and the singular vectors of F, without materializing an N-by-N projection.
Questions for authors
-
What explicit constraint or criterion prevents the saturated hBIC solution, and how is that rule related to the oracle threshold schedule? Does the corrected practical procedure recover the clean rank-one example above and remain stable to the loss floor and small noise perturbations?
-
Which exact code and parameter settings generated the reported experiments? Please explain the DC-D4R sqrt(n)-versus-sqrt(N) scoring mismatch, distinguish selected candidate rank from the rank of an averaged coefficient matrix, and provide corrected comparisons using the stated tuning protocol. Do the results hold for moderate, difficult-to-detect contamination as well as the large injected shifts?
-
What are the computation and communication costs against an optimized centralized robust RRR implementation using factored projections, including all candidate-rank fits and scoring exchanges? Which contributions to wall time arise from parallel execution rather than reduced total arithmetic work?
-
Please correct or restrict Lemma C.1's claim for arbitrary epsilon: the displayed Bernstein bound has a linear-tail regime, while Assumption 2 does not involve log(1/epsilon). Also, which experimental regimes satisfy the conditions for the constant-round result, and which demonstrate robustness empirically outside those conditions?
7 comments
Response to W1 and W2
W1
The practical tuning and experimental implementation are not sufficiently reliable as supplied. Equation (21) permits a saturated contamination estimate that absorbs every residual row, giving zero residual sum of squares and an undefined or negative-infinite log term.
Response:
Thank you for this observation. We score only candidates with and a positive residual sum of squares. When the current residual matrix is nonzero, we include the zero contamination estimate. When the residual matrix is zero, we set the contamination estimate to zero without evaluating hBIC.
Restricting candidate model complexity is consistent with established model selection procedures. Wang et al. [1] impose a model-size restriction in hBIC, while She and Chen [2] impose a model-complexity constraint in robust reduced-rank regression. We state this rule after Eq. (21) in the revised manuscript.
References
[1] Wang, L., Kim, Y., and Li, R. (2013). Calibrating nonconvex penalized regression in ultra-high dimension. The Annals of Statistics, 41(5), 2505–2536.
[2] She, Y., and Chen, K. (2017). Robust reduced-rank regression. Biometrika, 104(3), 633–647.
W2
The central efficiency claims are stronger than the demonstrated evidence. The advertised centralized term is avoidable: with a thin QR factorization , the reduced-rank step can use and the singular vectors of , without materializing an -by- projection.
Response:
Thank you for this observation. With a thin QR factorization , the centralized reduced-rank update can be computed through a rank- approximation of , followed by a triangular solve, without explicitly constructing the projection matrix.
Under balanced partitions with , the computational advantages of D4R arise primarily from its distributed architecture. The centralized method requires a QR factorization of the full design matrix, with complexity , whereas D4R only requires factorizing the central machine's design matrix, with complexity . Both methods involve rank- approximations of matrices. Moreover, D4R distributes the computation of adjusted gradients across machines, requiring operations per machine per round, compared with for the corresponding centralized computation. The total gradient computation remains , while the communication cost is scalars per round.
These comparisons highlight the computational benefits of D4R in reducing the central factorization cost and enabling parallel computation without pooling the full dataset. Section 3 gives the one-time QR factorization costs, the gradient computation costs per machine and in total, and the communication volume per round.
Response to Q1 and Q2
Q1
What explicit constraint or criterion prevents the saturated hBIC solution, and how is that rule related to the oracle threshold schedule? Does the corrected practical procedure recover the clean rank-one example above and remain stable to the loss floor and small noise perturbations?
Response:
Thank you for the question. As described in W1, we score only candidates with and a positive residual sum of squares. This restriction is consistent with the model-size constraints used in established model selection procedures, including Wang et al. (2013) and She and Chen (2017).
Within the admissible candidate set, hBIC selects the threshold by balancing goodness of fit against model complexity. For a fixed coefficient estimate, the hBIC minimizer in Eq. (21) also minimizes a penalized least-squares criterion with a row-sparsity penalty over the same candidate set. The associated implicit squared threshold is
\lVert Y_{I_k\cdot} -X_{I_k\cdot}B^{(t)} -\sqrt{N}C_{I_k\cdot}^{(t)} \rVert_F^2.$$ This provides a connection between hBIC selection and the theoretical threshold schedule in Theorem 5. Since hard-thresholding depends on the row norms of the residuals, different thresholds may yield the same contamination support. Therefore, if the support selected by hBIC can also be obtained using a threshold satisfying the conditions of Theorem 5, the corresponding concentration bounds and contraction analysis remain applicable, yielding the same order of estimation error. #### Q2 > Which exact code and parameter settings generated the reported experiments? Please explain the DC-D4R $\sqrt{n}$-versus-$\sqrt{N}$ scoring mismatch, distinguish selected candidate rank from the rank of an averaged coefficient matrix, and provide corrected comparisons using the stated tuning protocol. Do the results hold for moderate, difficult-to-detect contamination as well as the large injected shifts? **Response:** Thank you for these questions. We clarify the implementation and tuning procedures and provide additional results under lower-magnitude contamination. **Code and experimental settings** We have provided the implementation code for D4R. The simulation setup, competing methods, evaluation metrics, and tuning procedures are described in Section 5.1, with additional experimental details in Appendix E. **DC-D4R rank selection** The $\sqrt{n}$ and $\sqrt{N}$ scaling conventions are equivalent under a corresponding reparameterization of the contamination matrix. Specifically, $\sqrt{n}C_j=\sqrt{N}\widetilde C_j$, where $\widetilde C_j=\sqrt{n/N}C_j$. With this consistent reparameterization, the fitted residuals and RSC scores are identical, so the scaling convention does not affect rank selection. DC-D4R selects its candidate rank by minimizing the RSC over $\bar r\in\lbrace1,\ldots,8\rbrace$, with complexity penalty $\mu\bar r$, where $\mu=N+q$. For DC-D4R, $\hat r$ in Table R1 denotes the selected candidate rank rather than the numerical rank of the averaged coefficient matrix. **Results under lower-magnitude contamination** We further evaluate the methods under Gaussian response contamination with standard deviation 0.10, following the simulation settings in Section 5.1. Under this setting, D4R correctly selects the true rank $r=3$ and achieves a Frobenius estimation error of 0.570, comparable to Dist-CSL (0.564), though higher than the full-sample oracle (0.241). These results show that D4R maintains competitive estimation accuracy and accurate rank selection under lower-magnitude contamination. **Table R1.** Coefficient estimation errors and selected ranks under Gaussian response contamination with standard deviation 0.10, following the simulation settings of Section 5.1. Entries are mean (SE) over 100 replications. | Method | $\lVert \Delta\rVert_F$ | $\lVert \Delta\rVert_\infty$ | $\lVert \Delta\rVert_\Sigma$ | $\hat r$ | | ----------- | ----------------------: | ---------------------------: | ---------------------------: | ---------: | | Full-sample | 0.241 (0.005) | 0.040 (0.002) | 0.205 (0.003) | 3.0 (0.0) | | D4R | 0.570 (0.057) | 0.104 (0.015) | 0.485 (0.043) | 3.0 (0.0) | | DC-D4R | 1.044 (0.014) | 0.145 (0.003) | 0.822 (0.009) | 8.0 (0.0) | | Dist-CSL | 0.564 (0.022) | 0.096 (0.006) | 0.488 (0.019) | 3.0 (0.0) | | Dist-QRR | 1.865 (0.189) | 0.383 (0.039) | 1.449 (0.127) | 3.8 (0.8) | | Dist-RRR | 1.154 (0.024) | 0.159 (0.007) | 0.902 (0.022) | 14.4 (0.3) |
Author Response
We thank the reviewer for the positive assessment of our work and the detailed comments on practical tuning, experimental implementation, and computational efficiency. We have carefully addressed the comments and revised the manuscript accordingly, with all revisions highlighted in red. Our point-by-point responses are provided below.
For clarity, our detailed responses to each weakness and question are provided in separate comments below.