- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
The paper asks how to distinguish changes in signal strength from changes in spatial patterns when their norms are identical. It proposes averaging lagged Grassmann distances between estimated subspaces, shows where norm and endpoint statistics fail, and provides tracking and calibration guarantees under specific assumptions. Experiments use synthetic signals in North Atlantic EOF coordinates, additional noise tests, and an unlabelled heatwave case study.
Critical strengths
-
The problem is well motivated. The matched examples clearly show why signal strength alone can miss spatial changes, and why comparing endpoints can miss an excursion.
-
The calibration framework is useful. Under stream-level exchangeability, the paper provides finite-sample false-positive control. The sensitivity and calibration-shift experiments also make the scope of this guarantee clearer.
Critical weaknesses
-
The incremental contribution relative to existing subspace detectors is insufficiently established. The proposed procedure combines windowed SVD, principal-angle distances, temporal averaging, and standard exchangeability-based calibration. The impossibility results concern a restricted feature class, including even-length angle proxies, rather than the complete cited detection procedures. Appendix C.7 adds useful comparisons, but its Grassmann-jump CUSUM is a heuristic inspired by Subspace-CUSUM. Consequently, the evidence does not yet establish what substantial new capability or knowledge the proposed method contributes beyond existing detectors and simple normalized-covariance or spectral statistics.
-
The mechanism behind the empirical separation needs clarification. CORAL uses prescribed three-dimensional paths, with very low projected noise in the main table. For period-two hopping, an even tracking window gives a constant population leading subspace, yet ranking improves from approximately chance under frame-orthogonal residual noise to 100% under isotropic noise. This raises the possibility that differences in eigengaps and estimation fluctuations contribute to separation. The authors acknowledge the window-averaging limitation, but the mechanism behind the successful noisy results remains unclear.
Questions for authors
-
For period-two hopping with an even window length , the population windowed leading subspace is constant. What explains the 100% isotropic-noise ranking? Can you separate genuine trajectory recovery from effects of eigengaps and estimation fluctuations, including the noise-to-zero limit and odd versus even windows?
-
How does the method compare with an appropriate implementation of an existing subspace detector and a simple normalized-covariance or eigenvalue baseline under the same calibration protocol? For the noiseless CORAL hopping construction, the covariance eigenvalues differ between the branches despite equal norms, making this comparison particularly consequential.
-
The main CORAL experiments fix . Does the reported separation still hold when amplitudes vary over time, for example through trends or bursts? With calibration and test streams sharing the same amplitude law, can you report false-positive rates and detection power on amplitude-varying matched pairs?
3 comments
Thank you for updating the paper. The new comparisons are helpful, but motion, covariance, and Subspace-CUSUM all reach 100% power in Table 4. Could you also compare these methods in the harder structured-noise settings in Appendix C.7, using the same calibration protocol? C.7 uses a Grassmann-jump CUSUM method rather than the Subspace-CUSUM baseline added in Table 4.
Follow-up on structured-noise comparisons
RESULTS On hopping, spectral dispersion and projected-energy Subspace-CUSUM have 100% observed power in all eight cells, versus 58.5–96.5% for mean motion and 81.5–100% for covariance. On return at noise 0.25, covariance gives 99.5–100% and CUSUM 93.5–99.5%, versus motion’s 55.5–64.5%; spectral dispersion gives 21.5–26.5%. At noise 0.50, covariance retains 53.5–64% return power while motion falls to 15.5–16.5%. We added all sixteen conditions in Tables 4–5, Appendix C.7, and report these comparative limitations in Section 7. The earlier isotropic/amplitude-law comparison is now Table 6 in C.10.
CALIBRATION AND UNCERTAINTY All methods score the same streams. Each of 20 independent repetitions uses 19 calibration nulls and 10 fresh streams per class, with strict rejection above the 18th ordered calibration score at q=0.10. Ten separate training-null streams set references and CUSUM drift without test tuning. We use the actual projected-energy recurrence from C.10, including its 36-sample reporting delay, rather than the Grassmann-jump heuristic. C.7 specifies the retained paths, windows, lags and noise laws/levels.
Fresh-null rejection ranges from 4.5% to 15%. For each of spectral dispersion and CUSUM, all eight paired hopping-power advantages over motion (3.5–41.5 percentage points) have pointwise repetition-bootstrap 95% intervals excluding zero. Resampling uses the 20 independent repetitions. These are not simultaneous intervals, and observed 100% power, including collapsed empirical intervals, is not a population guarantee.
REFERENCE CONVENTION We fix initial azimuth within a repetition and randomise it between repetitions; original C.7 randomised it per stream. For return nulls, this permits independent training of one fixed null direction. The calibration counts/rule and noise conditions are unchanged, but this is not an exact replay of the original orientation mixture.
SCIENTIFIC IMPLICATION These are power limitations of the motion decision rule in the tested settings. For constant-amplitude hopping, the information difference is explicit: the branches have different covariance spectra but the same noiseless even-window leading line (C.10). Spectral or projected-energy detection can succeed without resolving the alternation; successful noisy motion rejection likewise does not establish trajectory recovery. Our matched constructions identify information lost by specified summaries, the window-parity result identifies loss caused by averaging, and the smooth-drift bounds give sufficient conditions for preserving a motion gap through estimation. Together with exchangeable-null calibration, these results explain which comparisons can conceal change and when estimated motion supports a calibrated fixed-versus-moving decision.
Response to Reviewer ziPA
DIRECT ANSWER — Contribution and existing detectors. We establish what selected summaries discard and when noisy estimates can distinguish fixed and moving patterns. Our matched constructions hide return excursions from endpoints and constant-amplitude alternation from even-block angles. For smooth one-way drift, we bound estimation error relative to the motion gap and calibrate decisions using exchangeable nulls. Our hopping trials show perfect noisy ranking even when windowing erases alternation (Proposition 4/Figure 9). Table 4 shows covariance and Subspace-CUSUM also succeeding. Our analysis distinguishes detecting a change from recovering its spatial path.
CORRECTION TO THE PAPER — We incorrectly grouped Subspace-CUSUM with endpoint/block comparisons in the submitted related work. We now describe its projected-energy mechanism separately and remove that attribution from the angle-proxy descriptions (p. 2, lines 79–87). The submitted remark after Theorem 1 already restricts its lower bounds to specified angle proxies (p. 4, line 215, through p. 5, line 218). Theorem 1(iii) now explicitly states constant amplitude (p. 4, lines 205–208), as its submitted proof required: equal visit counts alone do not balance squared amplitudes. Table 2's hop rows (p. 8) now mark fixed residual as unmatched; projected early/late alone is degenerate because it lacks a population eigengap. This corrects the labels without changing numerical values.
NEW EVIDENCE — Hopping mechanism (Q1). Different noise responses separate mean-motion scores after even windows erase noiseless alternation. In new Proposition 4 we prove the exact boundary at constant amplitude and zero noise: for directions (cos psi, ±sin psi), even windows have a constant leading line; odd windows alternate by arctan(sin(2 psi)/(w cos(2 psi))) for 0<psi<pi/4. Odd lags retain this angle and even lags cancel it. At psi=30 degrees, even-window hopping has spectrum (0.75,0.25,0), versus (1,0,0) on forcing; the gaps are 0.5 and 1. For w=36, we tested 100 pairs and obtained forcing/hopping means 0/0 at zero noise, 9.37e-8/1.51e-7 at noise 1e-6, and 8.34e-4/1.34e-3 at 0.0089. Rankings are 50%, 100%, 100%. Both scores approach zero as noise vanishes; the positive-noise rankings are finite-trial observations, not a trajectory-recovery guarantee. At zero noise, w=35 and 37 retain scores 0.0494 and 0.0468 radians, below the instantaneous hop angle pi/3. These mechanism scores are mean lagged angles in radians, not per-month rates. Section 6 explains the mechanism (p. 8, lines 418–423); Appendix C.10 gives the proof (p. 32, line 1723, through p. 33, line 1736), noise check (p. 33, lines 1739–1746) and Figure 9 (p. 33, caption lines 1762–1769). Figure 9's error bars use 100 independent pairs.
NEW EVIDENCE — Stronger comparisons (Q2). Motion, normalised covariance and projected-energy Subspace-CUSUM each attain 100% observed power across all nine new path–amplitude cells at in-frame noise 0.25. Spectral dispersion reaches 19–33.5% on drift, 32.5–35% on return and 100% on hopping, confirming the spectral distinction you identified. We implemented Subspace-CUSUM's projected-energy recursion with an independently trained null direction, a future window excluding the scored sample and its 36-sample delay. We calibrate the maximum over updates available by the common horizon on the same independent null streams as the other scores. We choose its drift parameter from separate null training data, without transferring Gaussian delay optimality to these different signal laws. The other scores average normalised-covariance deviation or spectral dispersion 1−lambda_max/trace. We fix initial azimuth at zero and train references on that null direction, using the submitted drift/return schedules; the original CORAL trials randomised initial azimuth. Section 6 reports these results (p. 8, lines 423–428), Appendix C.10 gives the implementation (p. 33, line 1770, through p. 34, line 1788), and Table 4 the outcomes (p. 34, lines 1796–1813). Covariance and projected-energy accumulation retain information that the specified angle proxies discard.
NEW EVIDENCE — Amplitudes (Q3). The observed full power for motion, covariance and Subspace-CUSUM holds for constant 1, trend 0.5+1.5t/T and a burst peaking at 4 in Table 4. Both latent branches share the amplitude path, with independent additive noises; calibration and test share the amplitude/noise law. We run 20 disjoint repetitions, each with 19 calibration nulls and 10 fresh streams per class at q=0.10 (p. 34, lines 1789–1794). Fresh-null rejection ranges from 4.5–14.5%; standard errors across the 20 independent repetitions range from 1.4 to 3.8 percentage points. The 200 test streams per class are not 200 calibration replicates. The matched-law design answers Q3 without assuming robustness to amplitude-law shift.