- Rating
- 3
- Confidence
- 3
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
The paper studies statistical inference for RFedAGS with growing communication intervals. It derives a refined mean-square bound, establishes a CLT for a computable fixed-anchor average, and constructs consistent covariance estimates and Wald regions. Experiments quantify the trade-off between communication cost, coverage, and precision.
Critical strengths
- The refined state-error bound retains within-round variance reduction and permits substantially growing communication intervals. Controlling geometric and client-drift errors at the total-observation scale is a meaningful technical contribution.
- The paper provides feasible inference and counts its additional oracle, communication, and storage costs. The experiments report both calibrated cases and severe undercoverage, making the practical limitations clear.
Critical weaknesses
- The theory assumes that synchronized points, all local iterates, and reported averages remain in a sufficiently small neighborhood after a fixed burn-in. It does not establish this containment for the general unmodified recursion, limiting the applicability of the inference guarantees.
- The main communication comparison changes the interval schedule, step parameters, and discarded fraction together. Although tail ablations are provided, the advantage over a comparably tuned fixed-interval baseline remains unclear, especially at matched uncertainty size.
Questions for authors
- For the curved examples, can the required neighborhood containment be established for the unmodified recursion, rather than assumed?
- How does the growing-interval method compare with fixed intervals when both use the same tail fraction and are tuned to achieve comparable coverage and uncertainty size?
3 comments
Q1. For the sphere and rotation recursions at their experimental initializations, all-future containment in the required contraction basin remains an assumption. We make the radius restriction explicit below and use that same neighborhood for contraction and inference.
A conservative sufficient radius can be obtained from the constants already appearing in the proof. Let
\qquad b_{m+1}=\gamma_m r_{m+1},$$ and let $C_1 > 0$ be a uniform constant in the existing bound $$\mathbb E\\!\left[\\|b_{m+1}\\|\mid\mathcal F_m\right] \le C_1\gamma_m\bigl(\gamma_m+\\|\Delta_m\\|^2\bigr).$$ Fix this constant on a reference neighborhood where the geometric and oracle bounds hold. If $r_{\mathrm{geom}}$ is an admissible chart radius, a sufficient requirement on the neighborhood used in (G3) is $$\sup_{x\in\mathcal U} \\|R_{x^\star}^{-1}(x)\\| \le r \le \min\left\\{ r_{\mathrm{geom}},\frac{\mu}{8C_1} \right\\}.$$ The same fixed reference neighborhood also supplies constants $C_2,C_{\mathrm{noise}}$ such that, writing $\mathbb E_m=\mathbb E[\\,\cdot\mid\mathcal F_m]$ and $\kappa=\min(\delta,1)$, $$\mathbb E_m\\|b_{m+1}\\|^2\le C_2\\{\gamma_m^2\\|\Delta_m\\|^4+\gamma_m^{2+\kappa}\\},\qquad \mathbb E_m\\|\epsilon_{m+1}\\|^2\le C_{\mathrm{noise}}/E_m,\quad \mathbb E_m\epsilon_{m+1}=0.$$ With $M=\\|H_\star\\|_{\mathrm{op}}$, impose after the fixed burn-in $$\gamma_m\le\min\left\\{1,M^{-1},\frac{\mu}{2(M^2+3C_2r^2)}\right\\}.$$ For $D_m=(I-\gamma_mH_\star)\Delta_m$, the state–remainder cross term is bounded by $$2|\mathbb E_m\langle D_m,b_{m+1}\rangle| \le(\mu/4+2C_1r)\gamma_m\\|\Delta_m\\|^2+(4C_1^2/\mu)\gamma_m^3.$$ The noise–remainder cross term is at most $\gamma_m^2\mathbb E_m\\|\epsilon_{m+1}\\|^2/2+2\mathbb E_m\\|b_{m+1}\\|^2$. Together with the radius and step restrictions, this gives $$\mathbb E_m\\|\Delta_{m+1}\\|^2 \le(1-\mu\gamma_m)\\|\Delta_m\\|^2 +\tfrac32C_{\mathrm{noise}}\gamma_m\eta_m +(3C_2+4C_1^2/\mu)\gamma_m^{2+\kappa}.$$ Thus the original state rate follows with explicit constants. G3 requires containment under the original oracle law on this same predetermined neighborhood; the decreasing step schedule supplies the eventual step restriction. The experimental configurations are: - In the main sphere experiment, the tangent Gaussian observations are **truncated to radius $0.90$** before being mapped to the sphere. The initialization is $\operatorname{Exp}_{x^\star}(0.42,0.18)$, at distance approximately $0.457$ from the target. - In the rotation experiment, the observation rotation vectors have norm at most $0.5$. The initialization is $(0.3,0.2,-0.2)$, with norm approximately $0.412$. - The Euclidean experiments use unbounded Gaussian and Student noise and a clamped Pareto sampler. The bounded supports in the curved experiments allow local moment and smoothness conditions to be checked directly. In particular, preservation of a convex ball by an individual exponential SGD step does not automatically imply preservation under the RFedAGS server update, which aggregates transported gradient streams. The reported rotation diagnostic—all realized local rotation angles below $0.414$ in the 600 finite runs—is evidence about those simulated trajectories. The normalized conservative certificate obtained by taking $C_1\ge1$ has radius at most $1/8$ in these intrinsic-mean models, since $H_\star\preceq I$. For every deterministic burn-in, their sampling laws admit positive-probability finite sequences that remain outside this particular basin. On the sphere, observations can stay near the initial point. On rotations, $v_*=0.13(1,1,-1)/\sqrt3$ lies inside every client support because $\max_k\\|v_*-\mu_k\\|^2\le 0.13^2+0.2^2+0.052/\sqrt3 < 0.3^2$; the geodesic from the actual initialization to this point stays at target distance at least 0.13. Repeated observations at a common point $y$ give the exact gradient-stream update $x_{m+1}=\mathrm{Exp}_{x_m}[-(1-η_m)^{E_m}\mathrm{Log}_{x_m}v]$ The local iterates, synchronized points and fixed-anchor averages remain on the same segment; continuity extends each finite construction to a positive-probability event. Consequently this normalized small-basin certificate leaves the curved experiments outside its verified applicability. A sharper constant or a larger-basin/random-entry argument is a separate analytical route. The finite-trajectory diagnostics describe the observed stability, and the obstruction concerns this sufficient certificate alone. The radius and step restrictions appear in Assumption C.1(G3) and Lemma G.1; Appendix G gives the scope of the normalized certificate.Q2. We compare fixed and growing intervals with a common tail fraction and independent validation-loss tuning. Both families receive equal search budgets; tuning uses observed validation loss. The selected Gaussian fixed/growing settings are and . The sphere settings are and . A separate independent pilot selects budgets to match uncertainty size. Equal-budget and matched-size comparisons use separate 500-replication confirmation batches. Size is ; communication includes the dense optimization and inference messages.
| Model / comparison | Budget, fixed/growing | Fixed coverage | Growing coverage | Fixed size | Growing size | Fixed MB | Growing MB |
|---|---|---|---|---|---|---|---|
| Gaussian, equal budget | 16384/16384 | 0.950 | 0.970 | 0.0044113 | 0.0039021 | 1.5872 | 2.1440 |
| Gaussian, matched size | 16384/12288 | 0.944 | 0.942 | 0.0044187 | 0.0045214 | 1.5872 | 1.7696 |
| Sphere, equal budget | 16384/16384 | 0.982 | 0.938 | 0.000990411 | 0.000864611 | 0.863488 | 0.600320 |
| Sphere, matched size | 16384/12288 | 0.968 | 0.890 | 0.000990358 | 0.000997953 | 0.863488 | 0.495488 |
The selected Gaussian schedule communicates more often than fixed at these budgets. At matched size it has similar coverage and more communication. On the sphere, the growing schedule reduces communication by 42.6% while coverage falls from 0.968 to 0.890. On the sphere, validation-loss tuning matches size but gives lower coverage for the growing schedule. These results quantify the calibration–communication tradeoff after equal-tail, data-based tuning.
A separate model-specific calibration exploration selects settings using the known simulation target, followed by independent confirmation. At matched sphere uncertainty size, growing coverage is 0.952 versus 0.982 for fixed intervals, with 63.9% lower dense communication and actual per-client sample counts 12269 versus 16380. Its MSE is versus . This is evidence about attainable configurations in that simulation model; the validation-loss comparison above is the data-based tuning procedure. The validation-selected results are in Table 5 and Appendix B.5.
Thank you again for your valuable feedback. We hope our responses address your concerns and welcome any further questions or discussion.
Thank you for your time and valuable comments.
W1. The curved inference result requires the trajectory to remain in one fixed contraction neighborhood. The explicit radius and step restrictions are given below. This supplies a common domain for inverse retractions, transports and local expansions. The sphere and rotation data have bounded supports; their measured trajectories provide finite-run diagnostics, while the manifold inference theorem assumes the stated all-future containment. We extend the Gaussian quadratic result to finite-moment noise: for the common-Hessian quadratic model Gk(x,ξ)=H(x−ak)+ξ, H≻0, with fixed dimension and client count. The client noises have fixed centered laws, are independent across calls and clients, and have finite (2+δ) moments for some 0<δ≤2; let S=∑kpk2Cov(ξk)≻0. For the original unstopped recursion from any deterministic start, take Em=max1,⌊cEmβ⌋ and γm=cγ(m+m0)−α with 0≤β<1 and (1+β)/2<α<1. Then E∣xm−x⋆∣2=O(ηm). Averaging rounds after bT=maxb0,⌊ρT⌋, for fixed b0 and 0≤ρ<1, gives
\nu_\rho=\frac{1-\rho^{1-\beta}}{(1-\rho)^2(1-\beta^2)}.$$ Independent fresh pooled scores consistently estimate $S$; exact Hessians and this retained-round variance factor therefore give asymptotically calibrated Wald regions. The proof uses the exact linear round transition, finite-moment Lyapunov control and the fresh-score law of large numbers. It covers ideal $t_5$ and symmetric Pareto shape-2.8 noise, for example with $\delta=0.5$. The submitted Pareto simulation uses a clamped sampler and therefore has all moments; the finite-moment extension above applies to the ideal, unclamped shape-2.8 law as well. The curved result retains its explicit local stability hypothesis. The finite-moment result is Proposition K.1; the Pareto sampler is described in Section 4.2. **W2.** We compare fixed $E_m=8$ with growing $E_m=\max\\{1,\lfloor\sqrt m\rfloor\\}$ in the Gaussian quadratic model, holding $\alpha=0.8$, the gain and the discarded fraction equal. The following independent paired experiment uses 1000 replications per configuration, with per-client budgets counting both optimization and fresh covariance-score observations. Both the following table and the subsequent fraction-0.1 comparisons use these new streams. At discarded fraction 0.5 and gain 0.8: | Per-client budget | Fixed coverage | Growing coverage | Fixed size | Growing size | Fixed MB | Growing MB | |---|---:|---:|---:|---:|---:|---:| | 16384 | 0.977 | 0.969 | 0.004474 | 0.003880 | 3.084 | 1.350 | | 65536 | 0.965 | 0.957 | 0.002238 | 0.001931 | 12.336 | 3.407 | With these identical tuning choices, the growing schedule gives smaller regions and lower communication at both budgets. The submitted fixed baseline instead uses fraction 0.1 and exponent 0.65, which explains the different size comparison there. At discarded fraction 0.1 and gain 1.6, fixed/growing sizes are 0.003415/0.003343 and 0.001707/0.001655, with coverage 0.959/0.923 and 0.944/0.933. With fraction 0.1 and gain 0.8, coverage is 0.948/0.723 and 0.939/0.907. For the growing gain-0.8 schedule at budget 16384, this rerun gives 0.723/0.969 at fractions 0.1/0.5; the submitted batch gives 0.710/0.962. The communication advantage depends on the schedules and tuning, as the validation-selected comparison below shows. These comparisons appear in Section 4.2, Table 2 and Appendix B.5.