- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
The paper studies two-sample testing on compact Riemannian manifolds using diffusion kernels within the martingale MMD framework. It derives a local-efficiency identity comparing diffusion-weighted signal and null-noise frequencies, and extends multi kernel inference to singular covariance through a thresholded pseudo inverse and estimated rank. It provides fixed-grid null and local-alternative limits, an exact finite-feature implementation, and experiments on the circle, torus, sphere, and archived pigeon bearings. The empirical analysis separates equal-sample power from the benefit of processing more observations under a similar computation budget.
Critical strengths
- The spectral efficiency calculation gives a clear interpretation of smoothing: additional diffusion helps when it removes null variation faster than signal. The treatment includes covariance between modes under nonuniform sampling, rather than relying only on an orthogonal uniform-null example.
- The singular-covariance extension is carefully formulated. The null theorem separates covariance-estimation error from the rank threshold, and the local theorem states the additional covariance-convergence and threshold conditions. These conditions matter when some contrast directions vanish or are nearly redundant.
- The experiments are unusually transparent about tradeoffs. They include duplicate and nearby scales, finite-support nulls, truncation controls, permutation calibration of the same rank statistic, and both equal-n and similar-time FUSE comparisons. The paper explicitly reports cases where FUSE or cross-witness testing is more powerful.
- The application is accompanied by relevant checks. The analysis documents independent-of-bearing subsampling, multiplicity correction, conservative small-sample calibration, and sensitivity to ordering/subsampling. It does not interpret the archive comparison causally.
Critical weaknesses
- The methodological advance is narrower than a new general testing framework. Chatterjee and Ramdas (2026, Section 7.1, Theorem 9) already provide multi kernel mMMD with a chi-square reference using covariance inversion. Applying mMMD to a diffusion-kernel family and computing it through finite features are natural specializations, as the paper itself notes in Proposition D.1. The main additions are therefore the frequency-based interpretation and the treatment of singular directions. Both are useful, but the efficiency identity is conditional on an unknown alternative direction and does not provide an adaptive scale-selection or optimality guarantee. The current fixed-grid, known-spectrum setting also limits the breadth of the result.
- The rank-recovery theorem and practical spectral truncation operate in substantially different regimes. Table 4 reports median retained rank two for both the exact-duplicate and nearby-time circle grids. Under a uniform circle law with the stated 100-mode truncation, these grids have population ranks five and six, respectively. Using the paper’s normalization, the population covariance for times a and b is ; at , the default threshold is approximately 0.633, and only two population eigenvalues exceed it in either grid. Thus these experiments demonstrate aggressive finite-sample truncation, not recovery of the population rank. The paper acknowledges this distinction, but the practical connection between its asymptotic theorem and the calibrated test remains incomplete. The lost signal along discarded directions, sensitivity near the threshold, and the recommendation to use different exponents in local experiments deserve a more direct explanation.
- The computation–power advantage is established against a limited set of resource choices. The larger-sample comparison fixes FUSE at 199 permutations; it does not include a similarly efficient finite-feature cross-witness or other inexpensive Gaussian-calibrated baseline under the same budget. Such a comparison is pertinent because Table 5 already shows competitive cross-witness power in some settings. At equal n=5000, FUSE has higher high-frequency power, 0.767 versus 0.613; the similar-time advantage arises when RA processes 5000 pairs and FUSE processes 2500. This is a valid and clearly disclosed comparison, but it establishes an advantage for those particular procedures and budgets. A small extension of the existing comparison would clarify whether rank aggregation itself offers a compelling computational choice. Anirban Chatterjee and Aaditya Ramdas. A Martingale Kernel Two-Sample Test. ALT 2026,PMLR 313,pp. 1–44。
Questions for authors
- Can you report the population/empirical eigenvalue spectra relative to the threshold for the rank-stress examples, and relate discarded directions to power? What single prespecified cutoff would you recommend when it is unknown whether an alternative is local?
- Could the existing larger-sample benchmark include a finite-feature cross-witness baseline on the same predetermined scales, with a valid prespecified aggregation rule, and a modest FUSE permutation-budget sensitivity check? This would directly test the claimed computational use case without introducing a new application.
- How should a practitioner interpret the application result given that the Holm-adjusted rejection fractions over order/subsample reruns are 0.610 and 0.548 for the two olfactory contrasts? In particular, what guidance can be given for a single prespecified analysis without selecting a favorable ordering?
4 comments
W3. We compared RA, MMD-FUSE and a finite-feature cross-witness grid on exactly the same observations, with observations per group for every method. The first distribution is uniform on the torus; the second has density . All methods use the same 400 nonconstant normalized-volume features and six predetermined diffusion times. Frequencies 9 and 11 extend the original frequency-seven comparison within the retained feature basis. The displayed results use an independent new batch of 500 replications per setting after an exploratory frequency comparison, separate from the preceding 300-repetition frequency-seven study. RA uses the recommended cutoff , with the historical exponent shown separately. Cross-witness fits each scale on one half, studentizes on the independent half and aggregates by ; FUSE uses 19, 49 or 199 nested permutations. Fixed bounded features, independent halves and Bonferroni give asymptotic level for the declared cross-witness aggregation.
| Method | Null rejection | Power, | Power, | Power, | Power, | Power, |
|---|---|---|---|---|---|---|
| RA, cutoff exponent | 0.044 | 0.590 | 0.224 | 0.070 | 0.998 | 0.578 |
| RA, cutoff exponent | 0.054 | 0.598 | 0.320 | 0.098 | 0.998 | 0.786 |
| FUSE, 19 permutations | 0.040 | 0.652 | 0.204 | 0.058 | 0.988 | 0.396 |
| FUSE, 49 permutations | 0.040 | 0.672 | 0.184 | 0.052 | 0.988 | 0.410 |
| FUSE, 199 permutations | 0.056 | 0.750 | 0.220 | 0.078 | 0.998 | 0.490 |
| Cross-witness grid | 0.016 | 0.324 | 0.084 | 0.042 | 0.886 | 0.176 |
At frequency 11 and amplitude 0.6, the recommended RA test has power 0.578 versus 0.396/0.410/0.490 for FUSE and 0.176 for cross-witness, gains of 18.2/16.8/8.8 and 40.2 percentage points, respectively. At the original frequency seven and amplitude 0.3, FUSE has higher power (0.652/0.672/0.750 versus 0.590); at frequency nine and amplitude 0.3, RA and 199-permutation FUSE are close (0.224 versus 0.220). At frequency 11 and amplitude 0.3 both methods have low power (0.070 for RA and 0.078 for 199-permutation FUSE); at frequency nine and amplitude 0.6 both reach 0.998. The advantage is clearest at frequency 11 and amplitude 0.6, alongside the frequency-seven setting favoring FUSE. The historical exponent gives power 0.786 at frequency 11 and amplitude 0.6, with null rejection 0.054; the recommended exponent remains the primary comparison.
Median complete times at the same are 0.1717 seconds for RA (including both cutoff calculations), 0.2611/0.2750/0.3491 for FUSE and 0.1145 for cross-witness. These streaming-implementation timings on uniform-null streams include feature evaluation, normalization, calibration and the final p-value, on one BLAS thread after warmup, excluding data generation. They compare methods within this benchmark; the submission reports its original implementation timings. RA uses 34.3%/37.6%/50.8% less time than FUSE with 19/49/199 permutations, respectively. Cross-witness is faster, with lower power in the displayed alternatives. Reducing FUSE's permutation budget leaves its pooled feature Gram normalization cost in place. The displayed power comparison fixes the sample size for all methods, and runtime is measured separately.
Q1. For a new analysis we specify with the normalized-volume kernels. It satisfies both and . The local result additionally assumes . Multiplying every kernel by multiplies covariance eigenvalues by , so the cutoff is multiplied by to preserve the retained directions. Normalization, threshold and retained rank are therefore specified together. The submission already uses exponent in the local spectral experiments and in the non-local experiments. The latter satisfies the null condition but lies on the boundary of the stronger local condition; satisfies both.
At , the population spectra and empirical medians in W2 place the third direction close to the cutoff. Changing the cutoff from 0.63335 to 0.79584 changes null rejection 0.026 to 0.024 and frequency-seven power 0.118 to 0.104 in the additional paired circle experiment (500 replications, amplitude 0.3). The projection formula in W2 quantifies why a small covariance eigenvalue may still carry standardized signal. For small-sample calibration under exchangeability, permutations recompute covariance, retained rank and the entire statistic.
Continued in the next part
Q2. The matched comparison in W3 includes the requested cross-witness baseline with a fixed six-scale Bonferroni aggregate and FUSE budgets 19, 49 and 199. Every method receives the same observations per group. At frequency 11 and amplitude 0.6, recommended-cutoff RA reaches power 0.578 versus 0.176 for cross-witness and 0.396/0.410/0.490 for FUSE; at frequency seven and amplitude 0.3, FUSE reaches 0.652/0.672/0.750 versus 0.590 for RA. Median complete times at the same are 0.1717 seconds for RA (including both cutoff calculations), 0.2611/0.2750/0.3491 for FUSE and 0.1145 for cross-witness. These streaming-implementation timings on uniform-null streams include feature evaluation, normalization, calibration and the final p-value, on one BLAS thread after warmup, excluding data generation. They compare methods within this benchmark; the submission reports its original implementation timings. RA uses 34.3%/37.6%/50.8% less time than FUSE with 19/49/199 permutations, respectively. Cross-witness is faster, with lower power in the displayed alternatives. Reducing FUSE's permutation budget leaves its pooled feature Gram normalization cost in place. This comparison holds the observations and sample size fixed across methods.
Q3. The 0.610 and 0.548 fractions measure sensitivity to ordering and subsampling of the same birds. The primary analysis uses bearing-blind seeded record selection and a fixed order. The 0.610/0.548 rejection frequencies show that the decision depends on the ordering and subsample. For a single analysis, prespecify the bearing-blind record-selection seed, pairing/order, normalization, cutoff, reference distribution and Holm family. The primary asymptotic Holm p-values are 0.0181 and 0.0989 for the two olfactory contrasts. For the same RA statistic, 9999 pooled permutations recomputing covariance and rank give Holm p-values 0.0039 and 0.0222. The latter calculation changes the calibration of the same observed statistic. These results present both the primary large-sample analysis and its calibration sensitivity.
Thank you again for your valuable feedback. We hope our responses address your concerns and welcome any further questions or discussion.
I will maintain my current score
Thank you for the detailed responses and additional experiments. These largely address my main concerns, and I will maintain my current score.
Thank you for your time and valuable comments.
W1. We identify the contributions as the diffusion-frequency efficiency identity and singular covariance inference. Chatterjee and Ramdas provide the martingale and full rank multikernel construction; our row-average convention is asymptotically equivalent to theirs. Our efficiency identity compares signal and null attenuation, including nonuniform mode covariance, and the null and local-alternative limits handle covariance estimation, thresholding and positive-rank recovery. Even two characteristic kernels k and 2k have collinear contrasts. Rank adaptation handles such redundant covariance directions; diffusion times are specified independently of the testing observations. For an unknown direction we aggregate the predetermined grid. An independent pilot may choose among a fixed finite menu while preserving the test-sample level; the efficiency identity describes attenuation for a specified alternative, with optimal scale selection remaining a separate problem. A fixed bounded numerical kernel can supply the represented contrasts when an analytic spectrum is unavailable, subject to its own covariance and rank conditions.
W2. We distinguish population rank from the directions retained at a finite sample size. For the normalized-volume circle basis with 50 sine/cosine pairs, direct expansion gives
Γab=8∑k=150e−(a+b)k2.
The duplicate grid is (0.025,0.05,0.10,0.25,0.25,1); the nearby grid replaces the second 0.25 by 0.251. An additional 500-replication circle experiment uses common observations across the two grids and both cutoffs, independently of the submitted rank-stress experiment. The null is uniform and the alternative has density 1+0.3cos(7θ) relative to normalized volume. At n=500:
Population ranks are five and six. The historical cutoff 0.63335 and prospective cutoff 0.79584 both retain two population directions. The third empirical eigenvalue exceeds the historical cutoff in 0.114 of the null runs for each grid. For a fixed Gaussian signal vector b and covariance eigenpairs (λj,uj), discarding positive directions loses noncentrality
∑0<λj≤κλj(ujTb)2.
For frequency one, three and seven, the fractions of noncentrality lost are 0.178/0.656/0.797 on the duplicate grid and 0.177/0.723/0.825 on the nearby grid. These quantify the Gaussian projection mechanism. The actual frequency seven rejection rates at n=500 are 0.118 with the historical cutoff and 0.104 with the prospective cutoff; their null rates are 0.026 and 0.024. The two grids have the same displayed rejection rates in this paired batch; the submission's independently seeded rank-stress batch gave 0.034 and 0.040 at exponent 1/4. Thus finite sample truncation trades weak-direction retention for stability, while the fixed-law theorem gives eventual rank recovery as threshold and estimation error vanish, for a fixed positive spectrum.
Continued in the next part