- Rating
- 2
- Confidence
- 2
- Informativeness
- 3
- Presentation
- 1
- Soundness
- 2
Read review
Paper summary
This paper proposes SPD Hierarchical Sliced-Wasserstein (SPDHSW) for comparing distributions of symmetric positive definite (SPD) matrices. The core of the method involves mapping SPD matrices to the space of symmetric matrices via the matrix logarithm, then first computing bottleneck projections and linearly combining them with random mixing vectors to generate multiple one-dimensional Wasserstein comparisons. The goal is to reduce the projection cost of the standard SPDSW from to ; is assumed. This idea is a combination of SPDSW (Bonnet et al., 2023) and HSW (Nguyen et al., 2023b).
Theoretically, this paper presents the following results. First, when using a direction sampler that is fully -invariant in Frobenius-orthonormal coordinates, the effective mixed direction can be decomposed as , where is a Frobenius-uniform direction and independent of . Consequently, the population objective of SPDHSW is times that of SPDSW for general , and when , the two squared objectives coincide exactly.
Second, considering that mixed slices sharing the same bottleneck bank are not independent of each other, we decompose the estimator variance into between-bank and within-bank terms: Therefore, even when is increased, a variance floor denoted by remains. The paper further analyzes the contribution of the degree-two components via spherical-harmonic decomposition.
Third, by extending the same variance decomposition to the empirical log-coordinate particle gradient, we show that the accuracy of the scalar discrepancy and the accuracy of the optimization gradient can differ.
Experiments include synthetic SPD distributions, the FPHA hand-action covariance, several EEG domain-adaptation datasets, and SPDNet regularization. The results show that in high-dimensional settings where the matrix logarithm is precomputed or log-coordinates are optimized directly, the projection step and optimization loop can be significantly accelerated. On the other hand, downstream classification accuracy is generally similar to that of SPDSW, and alignment improvements are relatively small when the budget and learning rate for each method are chosen independently.
Critical strengths
-
It clearly analyzes dependencies caused by shared bottlenecks. The strongest contribution of this paper lies in the statistical analysis of the hierarchical estimator rather than in the new SPD projection itself. In the single-bank estimator of the existing HSW, the various final slices share the same bottleneck directions and are therefore not independent Monte Carlo samples. This paper explicitly calculates this covariance and shows that the variance is of the form . This clearly explains why the rate cannot be applied directly and fills an important gap in the previous analysis of HSW (Nguyen et al., 2023b).
-
It clearly distinguishes between the population objective and the finite estimator. While the population objectives of SPDHSW and SPDSW are the same for , it is useful to distinguish that the variance and computational cost of their finite estimators differ. In particular, the paper concisely explains the population equality through the angular-radial factorization of the effective direction and shows that using normalized directions yields the same population scale even for general .
-
It analyzes gradient variance separately, rather than scalar variance. In actual training, the stochastic gradient is more important than the discrepancy value itself. The paper points out that, since directional optimal coupling varies depending on the direction, the scalar cost and the gradient can exhibit different directional variability. Accordingly, it presents a specific criterion showing that hierarchical mixing does little to aid scalar estimation but can reduce gradient noise.
-
The experiments cover a broad range and report various constraints with relative transparency. The paper covers projection-stage timing, total optimization timing, comparisons with equal projection counts, independently selected hyperparameters, fixed-direction controls, and diagnostics of scalar and gradient variance. In particular, it clearly reports that end-to-end speedup is virtually nonexistent in affine-invariant updates, where the matrix logarithm dominates the total runtime. The confirmation experiment using held-out high-gamma subjects is another strength of the experimental design.
-
The computational benefits are clear in high-dimensional log-coordinate settings. When the matrix logarithm is cached or the optimization variable is already , the effect of reducing high-dimensional matrix projections to is substantial. In particular, the time reduction in projection-dominated loops is convincingly demonstrated in large EEG and FPHA settings where .
Critical weaknesses
-
The methodological originality is limited. The algorithm essentially applies Nguyen et al.’s HSW bottleneck mixing to the SPDSW log-space construction proposed by Bonet et al. Since the log-Euclidean SPD space is isometric to via Frobenius-orthonormal coordinate transformation, the proposed estimator can be viewed as Euclidean HSW performed in a symmetric log-space. The population constant-factor result and the shared-bank variance analysis are also results that apply directly to Euclidean HSW rather than being inherently specific to SPD. Therefore, the most important novelty of the paper should be more accurately positioned as the analysis and refinement of the existing HSW estimator rather than a new SPD distance.
-
The gradient variance analysis is limited to diagnostics at a single fixed state. Theorem 3 analyzes the trace variance of the log-coordinate particle gradient at a fixed empirical state. However, it does not show how and change over the entire optimization trajectory, how the directional structure of the gradient covariance changes, or how the actual convergence rate or the range of stable learning rates improves. The descent bound in the Appendix B.9 is also a standard conditional inequality that assumes smoothness, and it does not prove that this condition holds throughout the entire trajectory of the actual sliced-Wasserstein objective. Furthermore, since the analysis focuses on the log-coordinate gradient, the conclusions regarding the affine-invariant Riemannian gradient or the general neural network parameter gradient are limited.
-
Some of the strong experimental results may be influenced by hyperparameter comparisons. In FPHA, the result showing that SPDHSW achieves alignment similar to that of flat in a much shorter time is impressive. However, SPDHSW uses the same learning rate as flat (), and the objective of flat actually deteriorates. This can be explained not only by estimator quality but also by differences in the gradient-noise scale and optimal learning rate resulting from the projection budget. An equal-time FPHA comparison, in which the learning rate is selected independently for each method and budget, is necessary.
-
From the perspective of scalar estimation, hierarchy is not always efficient. With the same number of final slices, flat SPDSW exhibits a lower scalar RMSE than SPDHSW, and in a total-work comparison targeting a 5% relative-RMSE, flat sampling is more efficient in many settings (). This implies that the advantages of the hierarchy are not applicable to general SPDSW estimation but are limited to specific optimization environments where high-dimensional projection costs dominate and the gradient floor is sufficiently low.
-
The presentation of notation and key definitions needs improvement. In Theorem 2, and are first defined, but their notation is then changed to and without further explanation. These quantities actually depend not only on but also on fixed values of and , yet this dependency is omitted. Furthermore, in Theorem 3 is introduced in the main text only as the “directional squared cost,” while the precise definition of optimal coupling does not appear until Appendix B.7. It is preferable to explicitly state in the main text the definitions necessary for understanding key theorems. Also, the sentences are broken up abruptly, making them difficult to read.
-
Claims regarding computational efficiency should more clearly define their scope of application. In affine-invariant updates, the matrix logarithm is a cost shared by both SPDSW and SPDHSW; it is not a problem specific to SPDHSW. However, this also means that the proposed method does not generally accelerate the entire SPD optimization process but only accelerates the post-log projection stage. Therefore, the main computational claim should more prominently highlight the condition of "cached or directly optimized log coordinates."
Questions for authors
-
The proposed SPDHSW estimator appears mathematically equivalent to applying Euclidean HSW in Frobenius-orthonormal log coordinates. What exactly do the authors consider to be the genuinely SPD-specific contribution? Which parts of Theorems 1–3 do not apply to general Euclidean HSW?
-
Table 5 estimates and only at the initial FPHA state, before any alignment update. Do the normalized gradient floor and the advantage of hierarchical over flat persist along the optimization trajectory, particularly near convergence where the gradient magnitude becomes small relative to the stochastic gradient variance?
-
Can you provide an equal wall-clock comparison in the FPHA results, where the learning rate is selected independently for each projection budget and method? In the current shared-rate comparison, it is difficult to distinguish whether the deterioration in the flat case () is due to an underfitting estimator or a learning-rate mismatch.
-
Can the gradient analysis be extended to affine-invariant Riemannian gradients or neural network parameter gradients? At a minimum, please explain how a deterministic Jacobian or metric conversion affects the variance decomposition under fixed conditions.
-
Is it possible to compare independent flat directions, hierarchical mixing, orthogonal directions, or other variance-reduction samplers while maintaining the same computational cost? Based on the current comparison alone, it is difficult to determine whether the benefit is specific to the hierarchy or is a more general effect of structured-direction sampling.
-
In the independently tuned High-Gamma setting, the improvement in alignment is small, and the classification accuracy is nearly identical. Please explain how the authors interpret the practical effect size of these results and in which real-world application scenarios the reduction in the alignment loop leads to a meaningful speedup for the entire pipeline.