acceptodds
Under review as a conference paper at ICLR 2027

Distributed Inference for Manifold M-Estimators

Abstract

We develop one-round confidence regions for manifold -estimators from distributed data. An independent initial estimate places all workers in a common chart, where reported scores, Hessians and score second moments determine an estimate and its covariance. Under fixed-dimensional regularity, we separate the nonlinear error from the additional error caused by using only the master worker's Hessian. This decomposition yields explicit accuracy and machine-growth conditions for centralized first-order inference, with confidence regions invariant to orthonormal frame choice. For principal component analysis, the same summaries recover the exact centralized subspace, reducing the requirement to consistency. Spherical and subspace experiments connect coverage to accuracy and Hessian estimation, and quantify the communication trade-off between one-round recentring and two-round covariance estimation. A Dry Bean application illustrates uncertainty for feature contributions to a principal subspace.

Primary area:
optimization
Keywords:
Distributed learning, M-estimators, Manifolds data, Statistical inference

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_J3QeOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies communication-efficient distributed inference for manifold-valued parameters. An independent initial estimate establishes a common local coordinate system, in which workers compute scores, score second moments, and optionally Hessians. These summaries support an aggregated Newton estimator and a master-worker surrogate, with confidence regions constructed using sandwich covariance estimators. The theory separates the linearization remainder associated with the initial estimate from the additional error introduced by a single worker's Hessian, and establishes conditions for first-order equivalence to centralized estimation on the worker data. For PCA, the authors further recover the exact centralized worker-sample principal subspace from aggregated summaries. Experiments cover spherical intrinsic means, Grassmann subspaces, and the Dry Bean data.

Critical strengths

The theory clearly distinguishes statistical error, distributed approximation error, and covariance-estimation consistency, clarifying the requirements for valid inference.

The exact PCA recovery result has technical value and demonstrates how special algebraic structure can relax the initial-accuracy requirement of general one-step estimation.

The experimental comparisons are transparent: they use the same worker data, count no-report events as noncoverage, and discuss initialization costs and complete communication paths.

Critical weaknesses
  1. The methodology builds substantially on classical one-step estimation, surrogate likelihood, and sandwich inference. The additional difficulties introduced by the common random chart, and the substantive advances beyond existing Euclidean results, could be articulated more sharply. Practical diagnostics indicating when the asymptotic approximation is adequate would also strengthen the contribution.

  2. Finite-sample inferential reliability is a substantial concern. In one Gaussian setting in Table 1, nominal 95% coverage corresponds to empirical coverage of 95.6%, 84.6%, and 3.8% for the centralized, Newton, and surrogate procedures. These results do not contradict the asymptotic theorems, but show that the reporting checks do not ensure reliable coverage.

  3. Practical efficiency depends on initial-sample allocation, parameter dimension, and the communication environment. Let NN denote the total sample size across all workers and n0n_0 the independent initial-sample size, with T=N+n0T=N+n_0. The initial-sample observations are not reused in the pooled estimating equations, yielding a first-order variance factor of T/NT/N relative to an all-data benchmark. This factor approaches one when n0=o(N)n_0=o(N). Although the paper investigates allocation trade-offs, a practically implementable rule for selecting the initial-sample size remains unclear. Furthermore, transmitting Hessians and score second-moment matrices can make one-round procedures more expensive in bytes than multi-round alternatives, and the fixed-dimensional, i.i.d., local-regularity assumptions restrict applicability.

Questions for authors
  1. Beyond the distinctions already discussed, can the authors identify the specific new inferential results relative to Jordan et al. (2019), the distributed PCA results relative to Fan et al. (2019), and the advances beyond manifold optimization in Saparbayeva et al. (2018)? Which steps require arguments beyond existing theory and a local coordinate transformation?

  2. Can the worker summaries be used to diagnose one-round approximation error and indicate when additional communication is necessary or a confidence region should be withheld? In particular, can the adaptive covariance update in the appendix identify or mitigate the severe failures in Table 1?

3.Under a fixed total sample size T=N+n0T=N+n_0, how should the initial-sample size n0n_0 be chosen? Is there a theoretically justified, practically implementable rule that does not require unknown population quantities?

  1. For the same inferential target, and accounting for initialization, transmitted bytes, network latency, and computation, under what conditions does the one-round approach outperform dedicated multi-round procedures?
5 comments
AuthorsOpenReview ↗

Response to Reviewer J3Qe (1/3)

Thank you for your time and valuable comments.

W1. The contribution combines the conditional uniform Hessian law and centered score-transfer argument at the independent random chart, using an independent initial sample D0\mathcal D_0 (pilot), with the separate Hölder and master-Hessian error rates and mapped Wald inference. We also distinguish two observable diagnostics. For PCA, the existing summaries recover the exact worker-sample subspace and hence the point-approximation discrepancy. For a general loss with known domain-wide Hessian Hölder envelopes, a conservative local Newton-error bound follows from an empirical curvature and localization check, as detailed in Q2. These diagnostics assess center approximation; covariance calibration remains a separate requirement.

W2. The additional paired comparison applies exact recentring and covariance updating to the same severe design highlighted in the review.

The additional paired comparison uses the submitted severe Grassmann setting: D=20D=20, k=3k=3, eigengap 0.50.5, N=16000N=16000, m=32m=32, and an independent pilot of size n0=1000n_0=1000. The 500 submitted seeds per law are reused, with common initial and worker observations for all methods. Coverage is for nominal 95% joint regions.

MethodGaussian coverageStandardized t8t_8 coverage
Newton, pooled covariance0.8460.682
Surrogate, master covariance0.0380.008
Surrogate, pooled covariance0.0380.014
Exact recentring, initial covariance0.9520.874
Dedicated PCA, fitted covariance0.9560.924
Recentered adaptive, ε=0.25\varepsilon=0.250.9560.924

Exact recentring raises Gaussian coverage from 0.846 to 0.952 while retaining the initial covariance. In the heavy-tailed setting, moving to the fitted-chart covariance protocol raises coverage from 0.874 to 0.924. The adaptive rule requests the fitted-covariance round on every path in this severe setting. The dedicated and adaptive rows therefore coincide with the worker-centralized Wald region. The initial-covariance comparison isolates the center correction; the fitted-chart protocol additionally changes covariance evaluation and coordinates. The t8t_8 fitted-covariance region still undercovers, and retaining the surrogate center while replacing its covariance gives coverage only 0.014.

The reporting checks assess numerical feasibility. Among returned severe Gaussian regions, coverage is 0.870 for Newton and 0.083 for the surrogate; reliable coverage also requires a sufficiently accurate center and consistent covariance. A numerical fit inherits the exact-minimizer theorem under local strong convexity and an optimization residual of order op(N−1/2)o_p(N^{-1/2}). Numerical inheritance additionally requires derivative and linear-solve errors to preserve this accuracy and the reporting gates to succeed with probability tending to one. Fixed spectral thresholds require separation of the limiting spectra from those thresholds.

The submitted comparison at the different allocation T=19200T=19200, n0=3840n_0=3840, N=15360N=15360 also shows the finite-sample cost of updating: initial-covariance recentring covers 0.938 (Gaussian) and 0.906 (t5t_5), versus 0.932 and 0.890 for the adaptive update. Initial-chart and fitted-chart regions differ in both covariance evaluation and coordinate representation. These results compare complete protocols across the stated settings.

W3. The allocation rule below uses only T,mT,m. The cost comparison separates local and remotely collected initial samples, as in the submitted protocol comparison, and distinguishes dense from structure-aware messages. With a root-n0n_0 independent pilot, the sufficient conditions are N=o(n01+ρ)N=o(n_0^{1+\rho}) and, for the surrogate, m=o(n0)m=o(n_0). The rule in Q3 satisfies both with T/N→1T/N\to1 under m=o(T)m=o(T) and fixed ρ>0\rho > 0. At finite budget, the worker-only first-order variance factor is T/NT/N.

In the fixed-budget Gaussian comparison at T=19200,m=32T=19200,m=32, increasing the independent pilot from 960 to 3840 raises Newton coverage from 0.804 to 0.924 and surrogate coverage from 0.040 to 0.382. Worker-centralized coverage changes from 0.944 to 0.938 as fewer observations remain with the workers. This illustrates the competing approximation and sample-allocation effects.

For PCA, the structure-aware one-round payload is 395,808 bytes at D=20D=20, k=3k=3, m=32m=32, equal to the dedicated two-round payload; the dense implementation uses 685,224 bytes. A remotely collected pilot adds 8Dn08Dn_0 bytes and a collection stage; a pooled-Hessian initialization fallback requires those Hessian messages or another request. The statistical guarantees retain the stated fixed-dimensional, iid, local model.

AuthorsOpenReview ↗

Response to Reviewer J3Qe (2/3)

Q1. The mathematical targets and communication protocols differ as follows.

  • Jordan, Lee and Yang (2019), Theorems 1 and 3 and Corollary 4, establish Euclidean surrogate/one-step approximation and sandwich inference. Our conditional uniform Hessian and centered score-transfer lemmas verify the changing independent chart; the resulting Hölder rates control its approximation error, and the inference theorem maps studentized regions to the manifold.
  • Fan, Wang, Wang and Zhu (2019) analyze aggregation of local eigenspace projectors, including variance and aggregation bias and unbiasedness under symmetric innovations. Our PCA result uses a different summary protocol to reconstruct the exact empirical worker-sample subspace and support central-centered covariance selection. Direct ambient second-moment aggregation is the exact point benchmark.
  • Saparbayeva, Zhang and Lin (2018), Algorithm 1 and Theorem 2, study manifold surrogate optimization and distance to a centralized empirical minimizer under transported-Hessian regularity. Our target is the population parameter and its studentized mapped confidence region, with separate point-approximation and covariance-consistency conditions.

The random-chart verification supplies the additional uniformity needed for these inferential conclusions.

Q2. For PCA, the communicated summaries reconstruct the exact centralized worker-sample point v^N\widehat v_N. Whenever the reconstruction and chart checks succeed, they give the observable standardized discrepancy

ηN=N(v~N−v^N)TΣ^N−1(v~N−v^N)1/2.\eta_N=\\{N(\widetilde v_N-\widehat v_N)^{\mathsf T}\widehat\Sigma_N^{-1}(\widetilde v_N-\widehat v_N)\\}^{1/2}.

The covariance-update rule centers the region at this recovered point. It retains the initial covariance when ηN≤ε\eta_N\le\varepsilon and the chart check passes, and otherwise requests score second moments at the recovered fit. With the prespecified ε=0.25\varepsilon=0.25, every path in the severe Gaussian and t8t_8 comparison triggers that additional round, giving coverage 0.956 and 0.924. For comparison, exact recentring with initial covariance gives 0.952 and 0.874. Keeping the original surrogate point with pooled covariance gives 0.038 and 0.014. The observed improvement thus involves correcting the center; heavy-tailed fitted-covariance coverage still falls below nominal.

For a general loss, a computable conservative bound is available when valid numerical Hessian Hölder envelopes LjiL_{ji} are known throughout a declared chart ball Br0B_{r_0}. Set L^=N−1∑jiLji\widehat L=N^{-1}\sum_{ji}L_{ji}, a=λmin⁡(AN)>0a=\lambda_{\min}(A_N) > 0, λ=2a/3\lambda=2a/3, q=∣gN∣q=\\|g_N\\|, and

r=min⁡r0/2,(λ/(2L^))1/ρr=\min\\{r_0/2,(\lambda/(2\widehat L))^{1/\rho}\\}

when L^>0\widehat L > 0, using r=r0/2r=r_0/2 otherwise. The check 2q/λ≤r2q/\lambda\le r yields a unique anchor-local empirical minimizer uNu_N and

=\frac{\widehat Lq^{1+\rho}}{(1+\rho)\lambda^{2+\rho}},\qquad \eta_{N,\mathrm{local}}\le\frac{\sqrt N b_N}{\sqrt{\lambda_{\min}(\widehat\Sigma_N)}}.$$ The full-chart check $a-\widehat Lr_0^\rho > 0$, or another localization argument for the designated benchmark, identifies this root with the centralized fit. Each worker supplies one additional scalar envelope average; scores/Hessians must be exact or accompanied by derivative-error control. The bound controls center approximation conditional on these checks. Its inputs include a known valid envelope evaluator, beyond merely assuming an envelope exists. The certificate follows because the Hessian is at least $\lambda I$ throughout the closed $r$-ball. On the sphere of radius $2q/\lambda$, its gradient points outward, so the empirical loss has an interior stationary minimizer; strong convexity gives $\\|u_N\\|\le q/\lambda$. Taylor expansion gives a remainder at most $\widehat L\\|u_N\\|^{1+\rho}/(1+\rho)$; multiplication by $A_N^{-1}$ yields a bound $(2/3)b_N\le b_N$. The full-chart inequality ensures uniqueness on the entire declared chart. If $q=0$ both points are zero; if $\widehat L=0$ Newton is exact. Asymptotic validity of adaptive PCA covariance selection follows from coupling: the two central-centered Wald statistics satisfy $W_A-W_C=o_p(1)$ and share a continuous chi-square limit. Their coverage indicators therefore differ with probability $o(1)$, so observable selection preserves the limiting coverage under the paper's assumptions.
AuthorsOpenReview ↗

Response to Reviewer J3Qe (3/3)

Q3. A population-free sufficient allocation is

n=\lfloor(T-h_T)/m\rfloor,\qquad n_0=T-mn.$$ Choose the independent split before observing the data and require $n\ge1$. For fixed $\rho > 0$, $m=o(T)$ and a root-$n_0$ independent pilot, the rule gives $h_T\le n_0 < h_T+m$, $N\sim T$ and $$\frac{N}{n_0^{1+\rho}}\le\frac{(\log T)^{1+\rho}}{T^\rho}\to0,\qquad \frac{m}{n_0}\le\sqrt{m/T}\to0.$$ These imply the Newton and surrogate remainder conditions while $T/N\to1$. The rule is a sufficient asymptotic allocation using only $T,m$; finite-budget choices also depend on conditioning and the target basin. At $T=19200,m=32$, the formula gives $n_0=1952$ and $n=539$. This exact allocation was not simulated. The nearest evaluated allocation, $n_0=1920$, gives Newton/surrogate coverage 0.892/0.162; thus the asymptotic allocation rule is not a finite-budget coverage prescription. The fixed-budget Gaussian results are: | Independent pilot $n_0$ | Newton | Surrogate | Worker-centralized PCA | | ---: | ---: | ---: | ---: | | 960 | 0.804 | 0.040 | 0.944 | | 1920 | 0.892 | 0.162 | 0.940 | | 3840 | 0.924 | 0.382 | 0.938 | A larger pilot substantially improves approximation in this setting, while the worker sample shrinks. The surrogate remains sensitive to the master-worker Hessian even at the largest pilot. **Q4.** The comparison includes pilot fitting/collection, broadcasts, worker score-moment computation, coordinator factorization, and iterative surrogate solves. For symmetric 64-bit payloads with the master colocated, $D=20$, $k=3$, $m=32$ and $d=51$, the dense one-round, encoded one-round and dedicated two-round payloads are respectively 685,224, 395,808 and 395,808 bytes. A remote pilot adds $8n_0D$ bytes; this is 160,000 bytes for $n_0=1000$ and 614,400 for $n_0=3840$. Under the additive cost model $K=rL_u+bL_b+B/\beta+C$, where $L_u$ is upload latency, $L_b$ broadcast latency, $\beta$ bandwidth and $C$ computation time, a local-pilot encoded one-round method is faster when $$L_u > C_{\mathrm{encoded}}-C_{\mathrm{dedicated}}.$$ For the dense implementation, the condition becomes $$L_u > 289416/\beta+C_{\mathrm{dense}}-C_{\mathrm{dedicated}}.$$ With remote collection of the $n_0=3840$ pilot, the encoded cost difference is $614400/\beta+C_{\mathrm{encoded}}-C_{\mathrm{dedicated}}$; the saved worker-upload stage is offset by pilot collection. These cost comparisons use the same $N$ worker observations and the specified latency model. At fixed total $T$, dedicated PCA can also use observations otherwise allocated to the pilot. End-to-end distributed runtime and network latency were not measured, so the crossover depends on the deployment-specific computation costs $C$. Computation includes dense score-moment accumulation $O(nd^2)$, coordinator factorization $O(d^3)$ and pilot PCA $O(n_0D^2+D^3)$. The severe-design surrogate uses a pooled-Hessian initialization fallback that requires the corresponding messages or another request; the fixed-total comparison initializes from the master Hessian or zero. The matched statistical comparison informs whether these costs buy adequate inference: Gaussian exact-recentered initial covariance covers 0.952 versus fitted covariance 0.956, whereas the $t_8$ values are 0.874 versus 0.924. For a general nonlinear loss, the fixed-total sphere comparison at $T=4000,m=20$ gives one-/two-round surrogate coverage 0.948/0.960 and 0.972/0.970 for the isotropic design at initial sizes 20 and 400; the asymmetric values are 0.944/0.950 and 0.930/0.934. The second round adds a worker pass and an anchor broadcast, while these coverage differences are small. Thank you again for your valuable feedback. We hope our responses address your concerns and welcome any further questions or discussion.
Reviewer_Csx1OpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
2
Soundness
3
Read review
Paper summary

In "Distributed Inference for Manifold M-estimators" the authors propose two methods based on a common local chart. The first is a Newton estimator using aggregated Hessians and the second is a surrogate method using the master agent's objective. They establish conditions under which both methods achieve the centralized asymptotic distribution and construct confidence regions. They also consider PCA on Grassmann manifolds, for which the communicated summaries allow exact recovery of the centralized empirical subspace.

Critical strengths
  1. The paper addresses statistical inference and not just estimation
  2. The exact PCA recovery result is interesting
Critical weaknesses
  1. Limited novelty: once a common local chart is established, the analysis largely reduces to standard Euclidean distributed M-estimation. The bounds follow from conventional Taylor expansions and Hessian regularity. It is not clear what new theoretical difficulty is addressed.
  2. Geometric formulation: the paper does not sufficiently distinguish Riemannian quantities from their coordinate representations. In particular the assumptions in (2.4) depend on the chosen frame, and the Hessians used by the algorithms are not generally intrinsic Riemannian Hessians.
  3. Restrictive assumptions: the theory assumes fixed manifold dimension and independent consistent initialization. No dimension-dependent guarantees are provided, limiting the relevance of the results to high-dimensional problems.
  4. Questionable practical advantages for PCA. The centralized PCA solution can already be recovered by aggregating empirical second-moment matrices. It is unclear what advantage the proposed geometric framework offers over this simpler approach.
  5. Weak empirical performance in some settings. The proposed methods exhibit substantial undercoverage in several experiments, particularly the master-worker surrogate. This raises concerns about their practical robustness.
Questions for authors
  1. Can the authors clarify what is genuinely new in the theoretical analysis compared with applying existing Euclidean distributed inference results in a local chart?
  2. Can the authors address the geometric issues discussed in wekanesses.
  3. Can the authors better justify the practical advantages of the framework over direct covariance aggregation for PCA and discuss the limitations highlighted by the experiments?
3 comments
AuthorsOpenReview ↗

Response to Reviewer Csx1 (1/3)

Thank you for your time and valuable comments.

W1. The random-anchor argument combines conditional uniform Hessian control with centered score transfer. Here the independent pilot is the initial sample D0\mathcal D_0 in the paper. Conditioning on this pilot makes the chart fixed for the worker analysis, but its location still changes with sample size. The conditional uniform Hessian and centered L2L^2 score-transfer lemmas provide the uniform control needed to connect that chart to the population target. They lead to separate Hölder linearization and master-Hessian remainder conditions and to mapped, studentized confidence regions. We attribute the Euclidean surrogate, one-step and sandwich ingredients to Jordan, Lee and Yang (2019).

W2. For a smooth chart retraction Rp\mathcal R_p with frame EpE_p, the Hessian used by the algorithm is

=\operatorname{Hess}f(p)[E_pa,E_pb] +\langle\operatorname{grad}f(p),\mathcal A_p(a,b)\rangle,$$ where $\mathcal A_p$ is the retraction's covariant acceleration. The extra term vanishes for a second-order retraction or at a stationary population target. Sample losses are handled through the full identity. For a smooth orthogonal frame change $U_p$ satisfying $\\|U_p-U_q\\|_{\mathrm{op}}\le Kd(p,q)$, the smooth-frame calculation preserves the coordinate envelopes and Hölder exponents, and replaces the anchor envelopes by $C_s+KB$ and $C_H+2KM$. Their required moments are preserved. Thus regularity is stated in the chosen smooth frame, and mapped point estimates and regions are exactly frame-invariant. **W3.** Our result assumes fixed intrinsic dimension, iid worker observations, local regularity and an independent consistent pilot. The PCA pilot is computable from its independent sample; empirical second-moment concentration and eigenvector perturbation give $O_p(n_0^{-1/2})$ under finite fourth moments and a positive eigengap. Dimension enters through the $d(d+1)/2$ entries of each symmetric covariance/Hessian message, the $O(d^3)$ factorization cost and the local envelope and concentration constants. The theorem holds with $d$ fixed; it gives no uniform growing-dimension rate. With a prespecified split, the sufficient allocation $$h_T=\max\\{T/\log T,\sqrt{Tm}\\},\quad n=\lfloor(T-h_T)/m\rfloor,\quad n_0=T-mn$$ satisfies the required approximation rates for fixed $\rho > 0$ when $m=o(T)$ and the root-$n_0$ pilot is available. Its role is to provide an implementable allocation within this fixed-dimensional local model. **W4.** Direct ambient second-moment aggregation recovers the exact centralized PCA point. This is the submitted exact-point benchmark. Joint uncertainty additionally requires score second moments. For the common-anchor method, transmitting $(T_j,S_j)$, where $T_j=n^{-1}\sum_iX_{ji}X_{ji}^{\mathsf T}$ and $S_j$ is the score second moment, exactly reconstructs the common-chart score, Hessian and score covariance. The score covariance involves fourth moments, whereas ambient second moments alone determine the PCA point. At $D=20$, $k=3$, $m=32$, symmetric 64-bit encoding of this one-round protocol costs 395,808 bytes, including the anchor broadcast. This matches the dedicated two-round payload and improves on the submitted dense-Hessian implementation's 685,224 bytes. The two protocols differ in communication rounds, pilot cost and the location at which score covariance is evaluated. The PCA result therefore describes an exact recovery and inference protocol; the general framework also covers losses without ambient second-moment sufficient statistics.
AuthorsOpenReview ↗

Response to Reviewer Csx1 (2/3)

W5. We compare approximate centering, exact recentring and fitted-chart covariance on the same severe-case observations.

The additional paired comparison uses the submitted severe Grassmann setting: D=20D=20, k=3k=3, eigengap 0.50.5, N=16000N=16000, m=32m=32, and an independent pilot of size n0=1000n_0=1000. The 500 submitted seeds per law are reused, with common initial and worker observations for all methods. Coverage is for nominal 95% joint regions.

MethodGaussian coverageStandardized t8t_8 coverage
Newton, pooled covariance0.8460.682
Surrogate, master covariance0.0380.008
Surrogate, pooled covariance0.0380.014
Exact recentring, initial covariance0.9520.874
Dedicated PCA, fitted covariance0.9560.924
Recentered adaptive, ε=0.25\varepsilon=0.250.9560.924

Exact recentring raises Gaussian coverage from 0.846 to 0.952 while retaining the initial covariance. In the heavy-tailed setting, moving to the fitted-chart covariance protocol raises coverage from 0.874 to 0.924. The adaptive rule requests the fitted-covariance round on every path in this severe setting. The dedicated and adaptive rows therefore coincide with the worker-centralized Wald region. For one-round surrogate inference with a root-n0n_0 pilot, the sufficient conditions are N=o(n01+ρ)N=o(n_0^{1+\rho}) and m=o(n0)m=o(n_0). In these severe finite-sample cells, exact recentring is the effective center correction. The initial-covariance comparison isolates the center correction; the fitted-chart protocol additionally changes covariance evaluation and coordinates. The t8t_8 fitted-covariance region still undercovers, and retaining the surrogate center while replacing its covariance gives coverage only 0.014.

In the Gaussian cell, conditioning on a returned region still gives coverage 0.870 for Newton and 0.083 for the surrogate. The exact-minimizer result transfers to a numerical surrogate when its residual is op(N−1/2)o_p(N^{-1/2}) inside the same uniformly strongly convex neighborhood. Numerical inheritance additionally requires derivative and linear-solve errors to preserve this accuracy and the reporting gates to succeed with probability tending to one. Fixed spectral thresholds require separation of the limiting spectra from those thresholds.

The submitted comparison at the different allocation T=19200T=19200, n0=3840n_0=3840, N=15360N=15360 also shows the finite-sample cost of updating: initial-covariance recentring covers 0.938 (Gaussian) and 0.906 (t5t_5), versus 0.932 and 0.890 for the adaptive update. Initial-chart and fitted-chart regions differ in both covariance evaluation and coordinate representation. These results compare complete protocols across the stated settings.

Q1. The inferential result combines independent-random-chart uniformity and centered score transfer with the two explicit approximation conditions

\sqrt N\\,n^{-1/2}\delta_N\to0$$ (the second is needed for the surrogate), where $\delta_N=a_{n_0}+N^{-1/2}$. These conditions transfer the centralized limit and consistent sandwich covariance to mapped manifold regions. For PCA, the exact summary reconstruction removes the point-approximation remainder: a consistent independent chart suffices for central-centered inference under the stated moment and eigengap conditions. These are the inferential conclusions supplied by the random-chart analysis. **Q2.** We give the coordinate-Hessian acceleration identity and quantitative smooth-frame envelope changes in W2, together with the explicit Fréchet/PCA regularity conditions. For different retractions, the comparison requires that each estimator be $o_p(N^{-1/2})$ from the same centralized local minimizer. The need for this rate is visible even in one dimension. For squared loss with standard Gaussian observations, anchor $p_N=N^{-1/4}$ and retractions $\mathcal R_p^0(v)=p+v$ and $\mathcal R_p^c(v)=p+v+cv^2/2$, let $g=p_N-\bar Z$. Their mapped Newton estimates satisfy $$\widetilde\theta^c-\widetilde\theta^0 =\frac{cg^2(3/2+cg)}{(1+cg)^2},\qquad \sqrt N(\widetilde\theta^c-\widetilde\theta^0)\to3c/2.$$ At this boundary $\sqrt N a_N^2=1$. Under the paper's strict negligible-remainder condition, the triangle inequality gives statistical-scale equivalence to the shared centralized fit.
AuthorsOpenReview ↗

Response to Reviewer Csx1 (3/3)

Q3. For PCA the practical trade-off is one-round inference from a common anchor versus fitted-covariance inference after exact ambient aggregation. The encoded one-round and dedicated two-round protocols each transmit 395,808 bytes at D=20D=20, k=3k=3, m=32m=32 (the submitted dense one-round implementation uses 685,224 bytes); a local pilot allows the one-round protocol to save one upload latency, with pilot fitting and score-moment computation included in its computational cost. A remote pilot of size 1000 adds 160,000 bytes and a collection stage.

Statistically, the severe Gaussian comparison gives coverage 0.952 after exact recentring with the initial covariance and 0.956 with fitted covariance; the corresponding t8t_8 values are 0.874 and 0.924. Thus the one-round recentered procedure is useful when initial-chart covariance is adequately calibrated, while the fitted-chart protocol has higher coverage in this t8t_8 example. The original surrogate remains poorly calibrated there. These comparisons support choosing the protocol jointly by inferential quality, pilot availability and communication costs.

Thank you again for your valuable feedback. We hope our responses address your concerns and welcome any further questions or discussion.

Reviewer_5BrdOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper considers M-estimators on manifolds with a general loss ( a general optimization problem with parameters lying on manifold where the data can be defined on the general sampling spaces) in a distributed settings. The data is distributed among mm workers, each with nn observations (N=mnN=mn). A separate initial sample of n0n_0 is used for locating the chart or estimating the anchor point, this part of data is denoted as D0\mathcal D_0. The idea seems simple: each work maintains the local loss function, score and Hessian which are then sent to the central server for aggregation. The central serve operates the optimization over the common local chart of p0p_0 obtained from the initial stage of estimation based on D0D_0. They also proposed a master-work surrogate procedure where worker 1 is colocaed with the master with f~N(v)=f1(v)−<g1−gN,v>\tilde f _N(v)=f_1(v)-<g_1-g_N, v> where g1g_1 is the score of worker 1 and gNg_N is the aggregated score. They then construct a x-squared confidence interval based on the pooled covariance estimator based on some asymptotic normal law that they established for the estimators.

Critical strengths

The results of the paper are correct and the surrogate procedure is interesting

Critical weaknesses

I do not see compelling innovation in the proposed algorithm or derived theory. The technical machinery for obtaining the theory is also somewhat standard.

The paper essentially lacks fundamentally new insights.

The example considered are too simple

Questions for authors

(1) how do you choose or estimate the initial anchor based on the data D0\mathcal D_0? How does it impact the analysis? (2) It would be good to elaborate on conditions on the Hessian regularity such as conditions (2.3) and (2.4). One is quite standard but good to illustrate with an example. One should at least verify these for the examples considered in the papers such as the estimation of Frechet mean, principle subspaces and so on.

2 comments
AuthorsOpenReview ↗

Response to Reviewer 5Brd (1/2)

Thank you for your time and valuable comments.

W1. The contribution is inference in an independently estimated common chart. The analysis proves a conditional uniform Hessian law and centered score transfer at the random anchor, separates the Hölder linearization remainder from master-Hessian error, and constructs mapped Wald regions with frame-invariant coverage. This builds on the surrogate and one-step constructions of Jordan, Lee and Yang (2019), including their sandwich inference result. The additional step is inference transfer through the changing independent chart: uniform Hessian control and centered score transfer connect worker summaries to the population sampling law. For PCA, the communicated summaries additionally identify the exact centralized worker-sample subspace, allowing central-centered inference under a consistent pilot.

W2. The error decomposition identifies two distinct requirements for reliable inference. With δN=an0+N−1/2\delta_N=a_{n_0}+N^{-1/2}, the Newton approximation error is Op(δN1+ρ)O_p(\delta_N^{1+\rho}), whereas the surrogate has the additional Op(n−1/2δN)O_p(n^{-1/2}\delta_N) term. These terms must be negligible relative to N−1/2N^{-1/2}, independently of covariance consistency. This distinction explains how a consistent sandwich covariance can accompany poor coverage when the center is insufficiently accurate. In PCA, exact recovery of the centralized point removes this centering remainder and yields an observable point-error diagnostic.

W3. The sphere and PCA examples isolate nonlinear chart effects and distributed covariance estimation, respectively. The numerical evidence covers nonlinear spherical Fréchet means, a Grassmann problem with 51 intrinsic coordinates, and feature-contribution inference in the Dry Bean application. An independent pilot means the initial sample D0\mathcal D_0 in the paper. In the fixed-total sphere comparison, T=4000T=4000 and m=20m=20; nominal 95% joint coverage is:

DesignPilot n0n_0NewtonSurrogateTwo-round surrogateWorker-centralized
Isotropic200.9540.9480.9600.958
Isotropic4000.9760.9720.9700.976
Asymmetric200.9480.9440.9500.948
Asymmetric4000.9300.9300.9340.932

These comparisons place the distributed procedures close to the worker-centralized benchmark across the two distributions and pilot allocations. In the submitted large-block Grassmann setting (N=16000N=16000, n=2000n=2000, m=8m=8, n0=4000n_0=4000, eigengap 0.50.5), Newton coverage is 0.934 versus centralized coverage 0.936; surrogate coverage is 0.856. The more demanding m=32,n0=1000m=32,n_0=1000 setting identifies the need for better centering: the additional paired comparison on the submitted observations gives Gaussian coverage 0.952 after exact recentring, compared with Newton's 0.846. For standardized t8t_8, exact recentring with the initial covariance gives 0.874; fitted-chart covariance gives 0.924, equal to the worker-centralized benchmark.

For Dry Bean, the distances to the same worker-centralized empirical fit are 0.000154 for Newton, 0.00142 for projection averaging and 0.00634 for the surrogate. This comparison measures empirical-fit approximation; the feature-contribution intervals use the application's stated iid working model.

Q1. For PCA, we estimate the pilot anchor as the leading eigenspace of the empirical second-moment matrix from the independent sample D0\mathcal D_0. The algorithm uses only that sample. In fixed dimension, a positive population eigengap Δ\Delta and a finite fourth moment give ∣Σ^0−ΣX∣F=Op(n0−1/2)\\|\widehat\Sigma_0-\Sigma_X\\|_F=O_p(n_0^{-1/2}). Eigenvector perturbation then yields d(p0,θ⋆)=Op(n0−1/2)d(p_0,\theta_\star)=O_p(n_0^{-1/2}).

For the sphere experiments, we normalize the ambient initial-sample mean and iterate logarithm averaging. For a general loss, the initial-rate proposition applies to a local empirical minimizer in a reference chart inside the designated target basin. The independent split permits conditioning on the estimated anchor in the worker analysis. Substituting its rate into the approximation bounds gives N=o(n01+ρ)N=o(n_0^{1+\rho}) for Newton and the additional m=o(n0)m=o(n_0) for the surrogate.

AuthorsOpenReview ↗

Response to Reviewer 5Brd (2/2)

Q2. The within-chart Hölder bound and the cross-anchor score/Hessian bounds follow from the following example-specific derivatives. For intrinsic Fréchet loss, a compact set of chart/data pairs uniformly separated from the cut locus has bounded derivatives DvℓD_v\ell, Dv2ℓD_v^2\ell, Dv3ℓD_v^3\ell, DpDvℓD_pD_v\ell and DpDv2ℓD_pD_v^2\ell. Integrating these derivatives gives the required coordinate and anchor Lipschitz bounds. On the unit sphere, if Z=Exp⁡θ⋆(V)Z=\operatorname{Exp}_{\theta_\star}(V) with EV=0\mathbb EV=0 and ∣V∣≤a<π/2\\|V\\|\le a < \pi/2, then at θ=θ⋆\theta=\theta_\star

Hess⁡d(θ,Z)2/2[u,u]≥acot⁡(a)∣u∣2.\operatorname{Hess}\\{d(\theta,Z)^2/2\\}[u,u]\ge a\cot(a)\\|u\\|^2.

With disk radius R=1.35R=1.35, the submitted isotropic and asymmetric designs respectively have

Q_\star=R^2\operatorname{diag}(401/1600,9/1600).$$ Both are positive definite. Shrinking the anchor/chart neighborhood gives the stated local curvature and separation conditions. For PCA, the graph projection $P(p,v)$ is jointly smooth and $\ell=-X^{\mathsf T}P(p,v)X/2$. Each required derivative is bounded by $C\\|X\\|^2$ on a compact chart neighborhood, so $\mathbb E\\|X\\|^4 < \infty$ supplies the squared envelopes. A positive eigengap supplies population curvature; score-covariance nonsingularity is checked separately for the data law. For Gaussian observations, column-wise vectorization and independent parallel/perpendicular eigenbasis blocks with covariance matrices $\Lambda_1,\Lambda_2$ give $$Q_\star=\Lambda_1\otimes\Lambda_2.$$ For the standardized $t_\nu$ model $X=\sqrt{(\nu-2)/W}\\,G$, with independent $W\sim\chi_\nu^2$, the score is multiplied by $(\nu-2)/W$. Consequently $$Q_\star=\frac{\nu-2}{\nu-4}\Lambda_1\otimes\Lambda_2,\qquad \nu > 4,$$ using $\mathbb E W^{-2}=\\{(\nu-2)(\nu-4)\\}^{-1}$. The positive simulated eigenvalues verify nonsingularity for the Gaussian, $t_5$ and $t_8$ designs; the $t_3$ control lies outside these finite-fourth-moment conditions. Thank you again for your valuable feedback. We hope our responses address your concerns and welcome any further questions or discussion.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.