- Rating
- 3(+1)
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 3
Read review
Paper summary
PLADOS performs, per iteration, a polynomial landing map , one gossip round, and a projected stochastic gradient step, with no retraction. With the fixed coefficient , the penalty-only map satisfies a restricted secant inequality and contracts locally toward the consensus Stiefel manifold at any rate . Combined with a polar-projection expansion, the stochastic analysis gives an average squared Riemannian gradient bound of . Experiments cover PCA (synthetic, MNIST) and low-rank matrix completion with up to 32 MPI nodes, compared against DRSGD.
Critical strengths
(1) An elegant geometric core. The tangent/normal decomposition of at the consensus manifold yields a fixed-representative contraction equivalent to an RSI, with explicit radius conditions. The polar-projection expansion with a product remainder is a reusable tool. (2) To my knowledge it is the first tracker-free stochastic decentralized landing method on the Stiefel manifold with a speedup-type guarantee, using one communication round per step.
Critical weaknesses
(1) “Parameter-free” sounds too strong. Only the landing coefficient is fixed, while the stepsize remains the main parameter of the method, and the convergence depends on it critically. For the diminishing schedule , convergence itself is natural; but the speed of convergence is governed by . In Corollary 3.8, the optimization term scales as and the network term as . For the constant step used in the speedup result, the method converges only to a neighborhood of size (eq. 107), which is clearly visible as the plateaus in Fig. 4. So both the level the method converges to and the speed of convergence depend critically on . On top of that, the guarantees hold only for (eq. 105), and is determined by constants (, , , , , , ) that a practitioner cannot compute, so the admissible stepsize is unknown and tuning cannot be avoided. It is therefore unclear to me how the method can be positioned as parameter-free (including the contrast with DRSGD in Table 1 and the claim to answer Byrd & Nocedal’s question) when its convergence critically depends on its main parameter . Relatedly, all guarantees are local, and the speedup regime requires , far beyond the tested budgets. (2) From the experiments, the benefit of PLADOS over DRSGD is not clear. PLADOS outperforms DRSGD at identical stepsizes (Fig. 2a,b; Appendix D.2), but it is not at all clear that this implies PLADOS is the better method: the best need not coincide for the two methods. The wall-clock gain may simply reflect the cheaper retraction-free iteration, which matters little at , and the appendix itself reports only a “slight advantage.” It would be much more interesting to compare a tuned DRSGD with a tuned PLADOS, e.g., time-to-accuracy with each method at its best stepsize. The comparison also omits other relevant baselines (DRGTA, DESTINY, DRFGT, RF-EXTRA, EF-Landing). In addition, the LRMC step coefficients were chosen after an initial comparison and scale as by design, so the reported wall-clock speedup partly reflects the stepsize choice.
Questions for authors
Clarifications that could affect my assessment:
(a) How sensitive is PLADOS to in practice, and how should be chosen when depends on unknown constants? In what precise sense is the method parameter-free?
(b) Please compare PLADOS and DRSGD with each method's stepsize tuned separately (e.g., best per method on a common grid), reporting time-to-accuracy and per-iteration cost.
(c) What happens outside the local tube, e.g., with random initialization?
Suggestion on presentation:
(d) Algorithm 1 is quite hard to read. It refers to equation (2), which in turn uses the operator , and is defined only inline in the introductory paragraph of Section 2, which makes it very hard to find. I would strongly recommend writing the explicit formulas inside Algorithm 1, including and , with short comments on what happens at each step (stochastic gradient, landing correction, gossip, update). Also, since the iterates are infeasible, is in general not idempotent; as noted in Section 2, it coincides with the tangent projection only at feasible points. A short remark on its role at infeasible iterates (e.g., as an approximation to the tangent projection at the nearby feasible point , cf. (58)) would make the update easier to interpret.