- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper studies decentralized stochastic optimization on compact embedded submanifolds and proposes DPRSHCM, which estimates gradients within a gradient-tracking and metric-projection framework using Hessian-corrected momentum. The correction term is the full directional derivative of the Riemannian gradient, viewed as an ambient vector field. It is applied to the iterate difference after projection onto the current tangent space. The theoretical results are Theorem 6.1, Theorem 6.2 and Theorem 6.3. The experiments cover synthetic PCA and LRMC, PCA on Ionosphere and MNIST, and LRMC on MovieLens-100K.
Critical strengths
- The diminishing schedule does not depend on the total number of iterations. Runs can therefore be extended without retuning.
- The proofs are complete and checkable, and the geometric correction is well motivated. I checked the main derivations of Lemma 5.1, Lemmas E.2–E.18 and Theorems 6.1–6.3 and found no errors.
- The experimental reporting is also careful.
Critical weaknesses
- The theory does not cover the experiments. LRMC falls outside Assumption 4.1, the sampling and output rules differ from those assumed in the theory, and even in the most favorable case the stepsize used exceeds the theoretical bound.
- The complexity results do not improve on prior work.
- The benefit of the Hessian correction itself is not isolated. There is neither a diminishing-stepsize DPRSRM baseline nor an exact-gradient baseline without HVP, and the constant configuration shows no consistent advantage over DPRSRM. Moreover, the experiments are small, and the real-data runs are repeated only two or three times.
Questions for authors
- Could you add a diminishing-stepsize DPRSRM baseline and a separately tuned exact-gradient baseline without HVP?
- Are the guarantees meaningful at the values of L_d and the stepsizes actually used? Could you provide a tighter local bound?
- What exactly does "Inverse-Free" in the title refer to?
3 comments
After considering the author's response and other comments, I keep the score unchanged.
Thank you for reconsidering our submission. We appreciate your careful assessment.
Regarding your concern about isolating the correction’s benefit, the additional ten-node MPI comparison uses matching diminishing-schedule parameters for DPRSHCM and DPRSRM. DPRSHCM reduces time to the same joint-residual target by 35.5%–36.6% across four thresholds, from to . At , it succeeds in 10/10 runs, compared with 9/10 for DPRSRM. Together with the component ablations, these results provide direct evidence of the correction’s computational benefit.
Could you clarify which concern remains decisive for your assessment? This would help us address the remaining issue precisely.
Thank you again for your time and thoughtful feedback.
Thank you for checking the main derivations and recognizing the horizon-independent schedule.
W1. Relation between theory and experiments
The revised objective meets the smoothness conditions. Appendix A verifies regularized LRMC, while Theorem 6.3 adds reshuffling analysis for complete equal-sized batches and the specified schedules. Appendix G.1 identifies the per-iteration schedule and truncated MovieLens epochs used in the reported runs.
Each output criterion has a stated role. The theorem bounds a uniformly selected iterate from the second half of the run; the tables report terminal performance. The theoretical guarantee is attached to the former criterion.
The parameter discussion separates sufficient conditions from empirical tuning. The reported stepsizes exceed the theorem's sufficient thresholds. Remark E.20 adds a computable PCA clipping bound and states the remaining restrictions; the existing sweeps assess stability at the tested parameter values.
W2. Contribution beyond the complexity exponent
The advance is the correction mechanism and its guarantee for decentralized manifold optimization. DPRSHCM uses stochastic information at the current point and admits a schedule independent of the iteration budget.
Geometry determines the correction. Operator (5.1) accounts for both tangent and normal gradient variation, yielding the quadratic remainder of Lemma 5.1.
Network motion changes the error analysis. Lemmas E.9 and E.15 bound displacement moments jointly with consensus and tracking errors. This supplies the control needed for the horizon-independent schedule.
Appendix B compares these requirements alongside the shared ϵ−3/2 exponent and the respective oracle models. Theorem 6.3 further provides a complete-epoch reshuffling guarantee.
W3 and Q1. Correction ablations and baselines
The checkpoints are k=99,199,…; “Not reached” means no qualifying checkpoint within 105 iterations. Both corrected estimators reach these accuracy levels far sooner than the uncorrected recursion. Their close results support comparable estimator accuracy.
Removing either term increases the mean objective gap and gradient norm. The full correction attains accuracy comparable to DPRSRM while using gradient information at one point.
Q2. Clipping radius and admissible stepsizes
Remark E.20 gives a computable clipping radius for PCA. With fi(X)=−21tr(X⊤CiX), tangent projection gives
∥gradfi(X)∥F≤∥CiX∥F≤r∥Ci∥2.
Thus Ld=crmaxi∥Ci∥2, c≥1, satisfies the clipping assumption and makes the radius explicitly data-dependent.
The theorem quantifies sufficient stepsize conditions. The experimental stepsizes are larger than these thresholds. The trajectory-based estimate in Remark E.20 bounds the tracker a posteriori; a local convergence theorem would also need an invariant region and verification of the remaining parameter conditions.
Q3. Meaning of “Inverse-Free”
“Inverse-Free” describes the outer optimization update. The method uses HVPs and the tangent-projector derivative without assembling the outer objective Hessian or solving a Newton system. Appendix F now states this meaning explicitly. For reduced objectives such as LRMC, the inner least-squares factorization remains part of objective evaluation and is reused by the correction.