- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper studies decentralized composite optimization on compact embedded submanifolds, where each agent has a smooth local loss and all agents share a convex nonsmooth regularizer. The authors propose DSPGT, which combines dynamic smoothing with projected consensus and partial gradient tracking. The method tracks only the heterogeneous smooth gradients while evaluating the smoothed regularizer locally.
The theoretical analysis introduces a local-regularizer Lyapunov function and uses weighted consensus estimates to establish an complexity bound for generalized stationarity of the original nonsmooth problem, matching the centralized dynamic-smoothing order under the stated oracle assumptions. Experiments on regularized subspace learning using MNIST and frozen CIFAR-10 features compare DSPGT with several existing decentralized methods across different network topologies and data distributions.
Critical strengths
-
The paper establishes an complexity guarantee for decentralized nonsmooth optimization on compact embedded submanifolds using a single communication round per iteration. The local-regularizer Lyapunov function and Jensen-gap argument provide a useful way to control smoothing-dependent errors without introducing the diverging Lipschitz factor into the main network recursion.
-
The analysis connects the smoothed convergence guarantee to generalized stationarity of the original nonsmooth problem, including the local outputs returned by individual agents. This strengthens the theoretical relevance of the result beyond convergence to a stationary point of a fixed smoothed objective.
Critical weaknesses
-
The algorithmic novelty appears somewhat limited, as DSPGT largely combines existing dynamic smoothing (Beck and Rosset, 2023) with decentralized projected Riemannian gradient tracking (Deng and Hu, 2025). The paper does not sufficiently clarify the technical novelty beyond this combination.
-
The theoretical benefit of gradient tracking is not sufficiently isolated. Without analyzing the corresponding dynamic-smoothing method without tracking, it remains unclear which heterogeneity-induced errors are eliminated and whether tracking is necessary to achieve the claimed rate.
-
The theoretical advantage over existing methods remains insufficiently characterized. DR-ProxGT (Wang et al., 2025) already achieves iteration complexity using a stronger proximal oracle, but the paper does not clearly establish the computational trade-off between the two approaches.
References
[1] Beck, A., and Rosset, I. (2023). A Dynamic Smoothing Technique for a Class of Nonsmooth Optimization Problems on Manifolds. SIAM Journal on Optimization, 33(3), 1473–1493.
[2] Deng, K., and Hu, J. (2025). Decentralized Projected Riemannian Gradient Method for Smooth Optimization on Compact Submanifolds Embedded in the Euclidean Space. Numerische Mathematik, 157(6), 2173–2210.
[3] Wang, L., Bao, L., and Liu, X. (2025). A Decentralized Proximal Gradient Tracking Algorithm for Composite Optimization on Riemannian Manifolds. Journal of Machine Learning Research, 26(106), 1–37.
Questions for authors
-
What is the main technical contribution beyond combining dynamic smoothing with decentralized projected gradient tracking? In particular, which difficulties cannot be directly addressed using the existing analyses of Beck and Rosset (2023) and Deng and Hu (2025)?
-
What convergence guarantee can be obtained for the proposed dynamic-smoothing framework without gradient tracking? Which heterogeneity-induced error terms arise, and how would they affect the complexity bound?
-
How does the proposed method compare theoretically with DR-ProxGT (Wang et al., 2025), particularly regarding the stationarity guarantees and local oracle requirements? Can the authors clarify the computational trade-off between the smoothing-based method and the existing proximal method?
2 comments
Thank you for the detailed response. I acknowledge that the local-regularizer potential, Jensen-gap argument, and weighted consensus estimates provide a convergence analysis for the proposed method. However, my main concern about the incremental nature of the contribution remains.
Specifically, Algorithm 1 closely resembles the projected Riemannian gradient tracking framework of Deng and Hu (2025), with the main modification being the addition of a dynamic-smoothing gradient correction. Moreover, tracking only the smooth component while handling the shared nonsmooth regularizer locally has already been adopted in DR-ProxGT (Wang et al., 2025).
Although the authors' response explains the technical challenges involved in analyzing this combination, it does not sufficiently clarify which part of the proposed theoretical framework constitutes a substantive advance beyond existing convexity-based analysis, projected-consensus analysis, and gradient-tracking techniques.
Therefore, I will maintain my original rating.
1. Weakness 1 and Question 1: Technical contribution
Thank you for recognizing the analysis. Two obstacles require separate treatments: smoothing increments in tracking and regularizer disagreement in descent.
Partial tracking removes recursive dependence on smoothing. Full tracking introduces regularizer-gradient increments with Lipschitz factor 1/μk (Eq. 3.3). Using the common h, DSPGT evaluates its smoothing gradient locally and tracks only heterogeneous smooth gradients. The tracker constants are independent of μk (Lemma E.3).
The local potential controls descent errors. Agents evaluate hμk at different points. We use Ψμ(x)=fˉ(xˉ)+n−1∑ihμ(xi). Convexity and doubly stochastic mixing yield a Jensen decrease (Eq. 3.5), absorbing regularizer-induced network errors (Lemma E.7). Intrinsic smoothing curvature remains in the step-size condition.
Together they close the cumulative estimate: (E.28) and (E.33) control the local-to-mean conversion. Neither centralized smoothing nor fixed-smoothness DPRGT supplies this network control as μk→0. We recover the centralized O(ε−3) order from the decentralized smoothing O(ε−4) bound of Wang and Liu (2023), with one neighbor exchange per iteration.
The contribution is resolving these network errors to obtain the cumulative-oracle guarantee, as clarified in Sections 1, 3 and 4.
2. Weakness 2 and Question 2: What changes without gradient tracking?
Without tracking, persistent local heterogeneity can invalidate the cumulative bounds used by our proof.
xi+−xˉ+=−α(gradfi(x)−gradfˉ(x))+O(α2).
The common regularizer cancels; local gradient mismatch can persist at aggregate stationarity.
- An exact example shows the obstruction. Set ek2=n−1∑i∥xi,k−xˉk∥2. For DSPGT, (E.28) and (E.33) give
\sum_{k=1}^K e_k^2/\alpha_{k-1}\le C\mathcal E_K.$$ Lemma F.1 considers two agents on $S^1$, $W=J$, $f_1(x)=x_1$, $f_2(x)=-x_1$, $h=0$, and common start $(0,1)$. For $\alpha_k=a(k+1)^{-p}$, $0<p<1$, and sufficiently small $a>0$, the untracked iterates satisfy $$\mathcal E_K^{\mathrm{nt}}\ge\tfrac34\sum_{k<K}\alpha_k,\qquad \sum_{k=1}^K e_k^2/\alpha_{k-1}\ge\tfrac12\sum_{k<K}\alpha_k.$$ Here $\mathcal E_K^{\mathrm{nt}}=\frac12\sum_{k<K}\alpha_k\sum_i\|d_{i,k}\|^2$. Both quantities diverge as $K^{1-p}$. Appendix F.2 verifies the exact recurrence and neighborhood conditions. 3. **Tracking replaces persistent mismatch by smooth-gradient increments**, controlled by a fixed Lipschitz constant times iterate motion. Partial tracking also excludes the $1/\mu_k$ regularizer increments. Without tracking, the pointwise estimate $e_k=O(\alpha_k)$ gives only $e_k/\mu_k=O(1)$ for coupled schedules. **The example establishes failure of the uniform energy bounds needed by this proof**, rather than a lower bound on all untracked methods' stationarity rates. The partial/full ablation separately tests which gradients to track. ### 3. Weakness 3 and Question 3: Stationarity and local-oracle tradeoff with DR-ProxGT 1. **Original KKT certificate.** Appendix F.1 gives a fixed-scale proximal KKT certificate at each returned primal point with $z_K=\nabla h_{\mu_{R_K}}(\bar x_{R_K})$, retaining $O(\varepsilon^{-3})$. Algorithm 1 returns primal points; evaluating this witness requires the projected network mean. 2. **Different local oracles.** DSPGT bounds cumulative gradient, ambient-prox and projection calls. DR-ProxGT's $O(\varepsilon^{-2})$ outer bound uses tangent-space proximal solves; total ambient-prox calls require inner-solver analysis. DSPGT's row-norm prox is rowwise; the tangent-space solve couples rows. Exact-oracle arithmetic costs depend on the problem; DSPGT does not improve the communication exponent. 3. **Common-accuracy costs.** After DR-ProxGT calibration, the strong synthetic setting uses 1,000-1,350 DR-ProxGT rounds and 3,000-4,050 Newton updates per agent, versus 2,450-2,500 rounds and no Newton updates for DSPGT. DSPGT's median paired algorithm-time advantage is **6.59**, while DR-ProxGT communicates less. CIFAR gives comparable algorithm times; DR-ProxGT reaches the stricter residual target. Appendix G.7 reports the protocol and costs, with all records retained. [Jiang et al. (2025), Table 1](https://arxiv.org/pdf/2509.08561), also separate outer iterations and cumulative proximal calls in the centralized setting.