- Rating
- 3
- Confidence
- 3
- Informativeness
- 3
- Presentation
- 2
- Soundness
- 3
Read review
Paper summary
This paper studies signed Fr'echet regression for manifold-valued responses. At a query point, the fitted response minimizes a weighted sum of squared geodesic distances, where the regression weights may be negative. The resulting objective has a difference-of-convex structure, but positive curvature limits the global convexity of squared-distance losses. Existing implicit methods also require repeated full-gradient evaluations, which can be expensive for large datasets.
The authors propose VariancE-Reduced Implicit FRIDA (VERI-FRIDA). At each outer iteration, the method samples a small batch of positively weighted observations to retain a nonlinear distance model, estimates the positive and negative component gradients using variance-reduced estimators, and adds a tangent-space correction so that the model gradient at the current iterate exactly matches the estimated full signed gradient. An adaptive regularizer enforces strong geodesic convexity of each local model, which is then approximately minimized. The framework accommodates PAGE, SARAH/SPIDER, SVRG, SVRRM, SRVRG, STORM, and Hybrid-SGD estimators.
Under explicit local geometric and negative-weight assumptions, the paper proves an bound on the expected squared Riemannian gradient norm at a uniformly selected iterate for refresh-based methods. For STORM and Hybrid-SGD, it proves almost sure convergence of the gradient norm to zero under a diminishing step-size schedule. Experiments on a torus, robot-motion regression, and comparing the proposed methods with explicit Riemannian baselines and deterministic FRIDA.
Critical strengths
-
Directly sampling the nonlinear Fréchet model introduces gradient noise that may persist even near a stationary point, making it difficult to retain the first-order accuracy needed for convergence. The paper addresses this challenge through a tangent-space correction that makes the sampled model's gradient at the current iterate exactly match the variance-reduced estimate of the full signed gradient.
-
Beyond proposing a stochastic implicit algorithm, the paper develops a common analysis that links gradient-estimation error and inexact inner solves to stationarity. It then applies this framework to several estimators, including PAGE, SARAH/SPIDER, SVRG, SVRRM, SRVRG, STORM, and Hybrid-SGD. The results distinguish the two sampling regimes: refresh-based variants obtain an expected stationarity bound, while refresh-free variants obtain almost-sure convergence under diminishing parameter schedules. This breadth makes the contribution more useful than an analysis of a single estimator.
Critical weaknesses
-
The paper is well organized, moving from the geometric setting to the algorithm, convergence results, and experiments. However, many symbols, tangent-space transports, geometric constants, and estimator variants are introduced in quick succession. It is difficult for readers who are in contact with manifold optimization for the first time to understand the research content quickly.
-
The convergence analysis requires all responses to lie in a sufficiently small geodesic ball (Assumption 2.1) and imposes on the total negative weight (Assumption 2.2). These conditions are motivated by the proof, but their practical scope is not fully characterized. For example, requires , while must also satisfy the geometric restrictions in Assumption 2.1. The authors not clarify how these conditions can be verified for a given dataset and characterize the query points for which the convergence guarantees apply.
-
The paper reports that the STORM and Hybrid variants make substantially less progress than the other variants within the experimental iteration budget, but does not investigate the cause. Since both are included in the proposed unified framework, it would be useful to examine whether their behavior is driven by estimator variance, parameter choices, or the implicit subproblem solves.
-
Algorithm 1 specifies an acceptance condition for the implicit subproblem, but the paper does not state which numerical solver is used in the experiments. It also omits the solver’s initialization, stopping tolerance, iteration limit, and observed inner iteration counts. These omissions make the timing results difficult to reproduce. The authors should provide the implementation details for each manifold.
Questions for authors
In the algorithm framework, the value range of step size is , but in the numerical experiment, the step size is adjusted separately for each method. How to obtain the specific step size?