- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
This paper analyzes an RKHS test statistic for online change detection based on the \chi-squared divergence. Given a sequence of observations x_1, x_2, \hdots with an unknown change time \tau, they consider a sliding window approach, where at time t the window x_{t-n,t},...x_{t} is compared with x_{t+1}, \hdots, x_{t+m} using a test statistic that approximates the \chi^2 divergence between the empirical measures defined by the windows. The analysis focuses on deriving explicit bounds on performance criteria for online change detection, including probability of false alarms, average run length to false alarm, and detection delays.
Critical strengths
Kernel based change detection has attracted significant interest in recent years. There are a wide variety of possible statistics and stopping rules that can be derived using RKHS-based approaches including MMD, and the various divergences, and it is of interest to undersand their trade offs. The authors point out that their non-asymptotic performance analyses of \chi^2 divergence for online detection is novel.
Another strength is that the authors consider many different performance criteria in the analysis including several different false alarm measures and multiple delay measures.
Critical weaknesses
Sliding window non-parametric change detection, where a block is compared with as t increases, has the limitation that if detection does not occur within the interval , then the algorithm will fail (unless it detects due to random fluctuations.) This suggests that the approach in this paper is better suited for offline change detection, where the test statistic can be computed at all possible splits of the data, avoiding this problem.
A key question is: Do the operating regimes specified in the results (Theorem 3.1 through Theorem 4.2) lead to non-vacuous performance guarantees? Theorem 3.1 and Corollary 3.2 appear to have some limitations in terms of their operating regimes that should be explicitly stated. Namely, since the kernel is bounded by K and since the operator has $\rho >0 $, we can bound the statistic as . Therefore, if then Theorem 3.1 is vacuously true because detection stops altogether, which of course makes delay go to infinity. Likewise, in Corollary 3.2, for any the eventually get so large that detection stops altogether and delays go to infinity.
Questions for authors
Do the results imply non-trivial guarantees on false alarms and delays simultaneously (that is, under the same operating conditions for the algorithm in terms of \rho, n,m and \nu?)
Secondly, dependence on the constants in the the theoretical results are rather opaquely presented. Is it possible to provide an intuitive explanation for the right hand sides of (27), (25)? Additionally, what specifically does the sample-size condition implying about the size of the windows?
We thank the reviewer for the careful assessment and for highlighting the importance of understanding the operating regimes and scope of the guarantees. These suggestions have led to substantial additions: explicit non-vacuity conditions, a joint-guarantee analysis, clearer interpretations of the miss-bound terms, and an offline extension. We address the four points below.
1. Sliding windows (Weakness 1). With fixed adjacent windows, a change at t0 remains within the monitored data only for split indices t∈[t0−m+1,t0+n−1]. Our detection-delay tail bound concerns the initial transition, while the reference window is still nominal. Crucially, the proof uses set inclusion, not independence between successive overlapping tests. The revised sliding-window remark makes this limitation explicit. We also extend the pointwise analysis to offline samples Z1:T, testing each split with n=t and m=T−t; a union bound controls false alarms over the T−1 splits. We do not claim an offline localization guarantee.
2. Non-vacuity (Weakness 2).
3. Simultaneous guarantees (Q1). The false-alarm and miss bounds apply to the same (n,m,ρ,ν), but a non-vacuous threshold alone does not ensure an informative miss bound. The new joint-guarantees appendix derives the necessary positive-margin condition Sϵt>2ν+4maxc0,d0. It also makes the limitations of our explicit constants transparent: for the derived miss bound even to be non-trivial, one needs both n>128log8,K2(K+ρ)/ρ3≈266K2(K+ρ)/ρ3 and n>256log8,K2K+ρ/ρ5/2; the first is more restrictive when ρ≤K/3. These are necessary conditions for this particular upper bound to be informative, not requirements for the detector to operate or detect changes. The Neumann-series sample-size assumption is likewise specific to the miss-bound proof and is not used for false-alarm control. As a complementary practical insight, our new leading-order delay relationship is supported by 21 non-pilot predictions with a median absolute relative error of 10.8% at n=120; it is a tuning heuristic, not an additional guarantee.
4. Interpreting the operational miss bound (Q2). We have expanded the explanations of its two exponential terms and auxiliary assumptions:
Conclusion. We appreciate the reviewer's questions, which have considerably improved the transparency and practical interpretation of the work. The revised manuscript now delineates the strengths and limitations of the guarantees and provides quantitative guidance for their use, alongside the new notation table and offline extension.