- Rating
- 4
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
The paper deals with bootstrap estimation of the asymptotic covariance of Riemannian stochastic approximation iterates.
It deals with this problem by running independent copies of the stochastic approximation algorithm (with the same noise and inputs) but with random weights.
Under general assumptions, it obtains the asymptotic joint distribution of comparison vectors and the Student and Hotelling confidence regions.
The paper features numerical validation on real data, but this is limited to just one (rather basic) manifold, the sphere.
Critical strengths
A well written paper, which clearly defines its problem and contribution. It deals with a specialised issue, but manages to remain accessible.
The main theorems are obtained under general (less restrictive) assumptions and are clearly stated.
These theorems are easy to understand and useful even to people who do not care about their conditions/proofs.
Finally, the results in this paper are both mathematically motivated and of clear practical usefulness.
Critical weaknesses
I did not identify any strong weaknesses in this work. I have to say that I did not read much of the supplementary material (in particular the proofs). However, I am familiar with the proof techniques for this kind of problem, and the hypotheses and theorems were formulated in such a way that already make it clear the proof techniques are well understood by the authors. Perhaps a more in depth verification of the proofs will be possible if this work is submitted to a journal, later on.
Questions for authors
This submission tackles an apparently simple, yet in fact quite tricky problem : estimating the asymptotic covariance of Riemannian stochastic approximation iterates. The overal idea is to use a bootstrap strategy by running independent copies of the (..stochastic approximation) algorithm, with random weights. These copies produce averages (estimates) that are compared in the tangent space at the observed average.
Four contributions are put forward in the introduction. All of these are well formulated and very interesting to anyone who works on Riemannian stochastic approximation (in the board sens of this term).
Questions/remarks about Section 2 :
-
why can't alpha be equal to 1/2 or 1 (in equation (1))
-
equation (2) is correct, by possibly problematic in practice : the inverse of a retraction is very rarely defined throughout the manifold. It is usually defined in some sphere of radium strictly less than the radius of the manifold at the base point
-
equation (7) is perfectly fine, but it took me a while to realise I is an interval and not a number. You should say it is an interval.
Section 3 :
I appreciate the conditions/assumptions were stated in the most general form possible. This is clearly a strong point ... Moreover, the main results in Theorems 3.6 and so on are quite compelling.
When stuyding a "Riemannian extension" of a problem that is well known in Euclidean space, I always ask : how is the curved setting different from the flat one ? Are the results ultimately an extension of Euclidean one (with no "surprises") or is there a really new behavior due to curvature.
I think not discussing this issue is a weak point of this submission, but it is otherwise quite solid and well written.
4 comments
Part 2/2 of our response to Reviewer 2GxD, continued from Part 1/2. This final comment addresses curvature and comparison at the observed base point.
Curvature and comparison at the observed base point.
Our results concern first-order inference in a regular, locally stable regime. Under the assumptions, the original and multiplier states eventually localize in a common neighborhood of an isolated target. The population derivative is nonsingular, and the states satisfy the second-moment bound , with . These conditions together with the geometric comparison bounds make the additional errors from retraction, intrinsic averaging, and changing tangent base points negligible at the scale.
Likewise, once the original and multiplier averages have errors of order , the difference between their inverse-retraction comparison and their difference in target coordinates is of order . Consequently, the leading asymptotic behavior is governed by a linear stochastic approximation in , giving the covariance
Thus, the familiar form of the limit follows from localization, regularity, and the scale of approximation.
Nevertheless, this does not make the limiting covariance independent of geometry. The operator is the derivative of the Riemannian population field and is the covariance with respect to the tangent space. Both can depend on the geometry of the problem. For instance, for the spherical intrinsic mean in Section C.3.1, writing , , and , we have
e\otimes e+r\cot(r)(I-e\otimes e) \right],$$ with the continuous extension at $r=0$. The factor $r\cot r$ explicitly reflects spherical geometry. Therefore, geometry can already change the leading uncertainty even though the covariance retains the same form. What vanishes under our assumptions is the additional first-order contribution from the nonlinear averaging and comparison operations. More explicit geometric effects would be relevant in several extensions. First, finite sample or higher order inference would retain geometric terms that are negligible in our first order analysis. Curvature, admissible neighborhood radii, and retraction properties could then enter approximation and coverage error bounds. Establishing such results would require quantitative control of the probabilistic terms, while separating geometric effects from step size and nonlinear drift effects. Second, cut locus interactions, singular population derivatives, or failure to converge to a common attracting solution can require a different analysis. Cut locus effects may alter the effective linearization, while degenerate targets may exhibit nonstandard rates or non-Gaussian limits. In these settings, one would need to establish the appropriate asymptotic behavior before determining whether the present bootstrap remains valid or requires modification. Thus establishing quantitative geometric effects and inference beyond this regime are important directions for future work.Thank you again for your careful review and constructive feedback.
Thank you for answering my questions. In the first place, my decision was "clear accept", and it (of course) still holds.
Thank you for these helpful questions about the assumptions, implementation, and geometric interpretation.
Note: Because of the 5000-character limit, we divided our response into two comments.
We address the four points in turn.
Step-size endpoints.
The manuscript discusses the step size range on p. 22. In the proofs of Lemmas C.17 and C.18, the Abel boundary is bounded by O(n(α−1)/2) and the intrinsic-average remainder by O(n1/2−α). Both vanish for α∈(1/2,1). At α=1/2, the displayed averaging bound need not vanish; at α=1, the displayed Abel bound need not vanish. These estimates explain the range used by the present argument and do not establish impossibility at either endpoint. In particular, an analysis at α=1 requires separate control involving the step constant and the spectrum of the population derivative.
Local inverse retractions.
Section 2.3 and Assumption 3.3 require the inverses used by the state and intrinsic-average comparisons to be defined along the implemented trajectory, together with eventual localization in a common comparison neighborhood. The final reporting statistic uses a measurable completion when its inverse is unavailable, and the theory makes that reporting event asymptotically negligible. This completion does not supply a missing inverse needed earlier by an iterative-average update. The state safeguard, state-rate verification, and full-average localization are separate parts of the sufficient conditions.
Explicit confidence-interval notation.
Equation (7) displays the endpoints, and Corollary 3.7(i) identifies the result as an interval. The revised first definition now explicitly says "confidence interval." For a differentiable scalar functional ϕ, fixed B≥1, and confidence error probability τ∈(0,1), the construction is
s_{\phi,n,B}^{2} &=\frac1B\sum_{b=1}^{B} \bigl\\{\phi(\bar x_n^{\star(b)})-\phi(\bar x_n)\bigr\\}^{2},\\\\ I_{\phi,n,B}(1-\tau) &=\left[ \phi(\bar x_n)-t_{B,1-\tau/2}s_{\phi,n,B},\\; \phi(\bar x_n)+t_{B,1-\tau/2}s_{\phi,n,B} \right]. \end{aligned}$$ Here $t_{B,1-\tau/2}$ is the Student quantile with $B$ degrees of freedom. The scale is already computed from unscaled estimator differences, so the endpoints have no additional factor of $n^{-1/2}$. Centering is at the original estimate, which gives $B$ rather than $B-1$ degrees of freedom. The stated fallback is $\mathbb R$ when $s_{\phi,n,B}=0$; under the nonzero limiting-variance condition, this event is asymptotically negligible. *Continued in Part 2/2, which addresses curvature and comparison at the observed base point.*