- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper proposes PARC, an online change-point detector that does not require a designated unchanged prefix. Past recent-versus-earlier feature contrasts define candidate change directions. For each direction, PARC constructs an empirical historical response distribution and scores subsequent blocks by their positive excess over its maximum, normalized by historical variability. Frozen short-window neural features support startup and are replaced by native long-window features. The analysis covers historical separation, score stability, startup threshold costs, and a separate finite-horizon false-alarm guarantee under exchangeable normal sequences. Evaluation on TSSB and TCPD reports improved detection coverage and capped delay loss over twelve baselines, supported by component studies and numerical theory checks.
Critical strengths
-
A well-motivated and interpretable comparison mechanism. PARC evaluates each proposed change against historical variability measured in the same direction. This directly addresses why a large recent contrast need not constitute unusual behavior. Freezing each comparison before testing later raw observations makes the construction causally coherent. These properties give the method a clear rationale beyond simply combining multiple window discrepancies.
-
Promising improvements in detection coverage under a reasonably careful evaluation protocol. PARC reports detection rates of 60.0% on TSSB and 51.0% on TCPD, compared with 48.4% and 39.5% for the strongest coverage comparators, respectively, while also reducing capped loss. Family-separated nested selection, retention of early changes in evaluation, and explicit accounting for missed and early alarms strengthen this evidence. The paper appropriately acknowledges that the gains primarily concern coverage rather than faster detection on jointly detected cases.
Critical weaknesses
-
The submitted artifacts do not support independent verification of the central comparison. Appendix C.2 states that the complete per-method configuration grids are included with the artifact. However, the released reproducibility documentation explicitly states that baseline implementations, baseline configuration grids, and baseline-specific tests are absent. Several component and statistical analyses also lack complete reproduction workflows, and the nine recovered TCPD records lack a pinned acquisition/recovery recipe. This discrepancy is consequential because the main empirical claim depends on the correctness and fairness of these baseline implementations and operating points.
-
The evidence does not yet sufficiently isolate the contribution of the central mechanism or establish generalization beyond the development archives. Both archives informed architecture development, so nested family splits do not eliminate prior design-selection effects. Moreover, the reported paired-family interval for the historical-maximum versus historical-mean comparison on TSSB includes zero, and a dedicated mature-gate ablation is missing. The neural startup advantage is primarily supported by TCPD, since all TSSB changes occur after the mature transition. Together, these limitations leave uncertainty about which components account for the gains and how reliably those gains transfer.
Questions for authors
-
Can you reconcile Appendix C.2’s statement that the complete baseline configuration grids are included with the release documentation stating that they are absent? Can you provide the exact baseline implementations or pinned upstream versions, the selected configurations and thresholds for each outer fold, and the per-record first alarms underlying Table 1, together with the provenance and recovery procedure for the nine recovered TCPD records?
-
Can you provide paired effect estimates and uncertainty intervals for the historical-maximum versus historical-mean controls on both archives? How sensitive is the 16-block historical reference to increasing history length, outliers, or previous changes, and are comparisons with a robust quantile reference available at comparable empirical false-alarm levels? Separately, is there an existing ablation that removes the mature coordinate gate while keeping the other components and fitting protocol unchanged?
-
Please clarify which architectural choices were informed by results on TSSB or TCPD, and whether any evaluation families or collections were entirely excluded from architecture development. If independent or cross-archive transfer results already exist, can you provide them to help assess generalization beyond the development archives?
-
The mature representation concatenates per-channel spectral and moment features without explicitly encoding cross-channel dependence. To what extent can it detect changes that preserve individual-channel statistics but alter cross-channel correlation or relative phase? If controlled results for such changes are available, can you provide them and clarify whether detection relies mainly on transient effects near the change boundary?
3 comments
Thank you for addressing my comments, but I still have a few questions.
The mean has fewer no-change alarms on TSSB, and the 95th percentile is slightly better on TCPD.
Given these mixed results, could you report paired uncertainty intervals for maximum versus percentile and compare their detection rates and delays after a large past fluctuation?
We added a simulation study to include two-channel time series change in both correlation and either mean or scale.
Could you test a correlation-only change while keeping each channel’s marginal distribution unchanged, so we can see whether PARC detects the dependence change itself?
We found and evaluated the frozen detector on a new dataset, WEAR, which is a separate wearable-sensor dataset.
Given that WEAR only evaluates startup and the advantage over Online RFF-MMD disappears at the 5% target, could you clarify how far these results support generalization of the full method?
Thank you so much for your quick reply and very helpful suggestions. There were some insufficient explanations from the previous revision. We have revised the paper again, with the changes highlighted.
Maximum versus percentile. We added paired 95% family-bootstrap intervals for maximum minus 95th percentile. We first average the predictions for each record across five splits. The results are:
| Archive | Detection difference (percentage points) | Capped-loss difference |
|---|---|---|
| TSSB | +2.03 [0.00, 4.69] | -0.028 [-0.056, -0.008] |
| TCPD | -0.50 [-1.67, 0.00] | +0.007 [-0.003, 0.019] |
In the above table, the maximum gives lower capped loss on TSSB, and the percentile performs similarly on TCPD. These intervals use saved predictions. We do not repeat model fitting.
We also added a simulation of mean/scale changes to include 200 time series for each case in the below table. Both versions use the same features and change directions to ensure a fair comparison. The results are:
| Past fluctuation | Maximum: detected / median delay | Percentile: detected / median delay |
|---|---|---|
| None | 64 / 206.5 | 79 / 221 |
| Spike | 53 / 219 | 71 / 254 |
| Brief excursion | 52 / 254 | 63 / 254 |
We can see past fluctuations reduce detection, but percentile detects more changes. For changes detected by both, the median paired delay difference is zero. Therefore, overall they are similar and percentile a reasonable alternative (Appendix F, Tables 11–12).
Correlation-only change. Thank you so much for this suggestion. Our earlier simulation changed both correlation and mean or scale, so it did not answer your question. We added a test that changes only Gaussian innovation correlation from +0.8 to -0.8. Each channel's marginal distribution and temporal dependence remain unchanged. PARC detects 19/200 changes versus 198/200 for a targeted correlation diagnostic, with 14/200 versus 10/200 normal alarms.
Thank you for this helpful suggestion. This test shows a limitation of PARC’s channel-wise features. We now state it clearly in the revised paper and identify efficient cross-channel features as future work, and this future work must overcome detection delay and computational cost as well. The discussion is in the main paper and Appendix I.
WEAR and the full method (new experiments). The earlier inputs tested only startup. Sorry for this confusion. We now added the fair experiments to evaluate all 22 people at 5, 10, 15, and native 50 Hz. Each person's thresholds are fitted from the other 21 people. We keep other settings all fixed.
D/E/C are detection / early-alarm / control-alarm percentages over 211 transitions and 128 controls.
| Rate | Target | PARC: D / E / C (%) | RFF-MMD: D / E / C (%) |
|---|---|---|---|
| 5 Hz | 5% | 23.7 / 0.5 / 5.5 | 16.6 / 2.4 / 5.5 |
| 5 Hz | 10% | 27.0 / 0.9 / 10.9 | 22.7 / 3.8 / 10.9 |
| 10 Hz | 5% | 27.5 / 0.0 / 5.5 | 11.8 / 1.9 / 6.3 |
| 10 Hz | 10% | 28.4 / 0.5 / 10.2 | 22.3 / 3.3 / 10.2 |
| 15 Hz | 5% | 22.3 / 0.5 / 5.5 | 10.4 / 1.4 / 3.1 |
| 15 Hz | 10% | 23.7 / 2.8 / 11.7 | 17.1 / 3.8 / 10.2 |
| Native 50 Hz | 5% | 11.4 / 0.5 / 5.5 | 15.2 / 2.4 / 5.5 |
| Native 50 Hz | 10% | 18.5 / 1.9 / 13.3 | 18.0 / 4.3 / 9.4 |
We can conclude the following lessons:
- External transfer: At 5–15 Hz, PARC has the highest detection rate among all eight methods at both targets, with fewer early alarms than RFF-MMD. At 10 Hz, it also achieves lower capped loss with the same or fewer control alarms.
- Mature-stage detection: Almost all detections at 10 Hz and all detections at native 50 Hz occur in the mature stage. These new results show that PARC transfers effectively.
- Sampling resolution: The same frozen model detects changes at all four rates. At native 50 Hz, PARC raises fewer early alarms at both targets and achieves similar detection to RFF-MMD at the 10% target.
Therefore, overall PARC show postive transfer and beats other sota methods on this separate dataset.
Archive ablations (existing Table 2):
| Variant | TSSB detection / early (%) | TCPD detection / early (%) |
|---|---|---|
| PARC | 60.0 / 2.9 | 51.0 / 4.0 |
| Native short features | 59.1 / 3.5 | 43.0 / 7.0 |
| Fixed 64-point context | 59.1 / 3.5 | 40.0 / 5.5 |
| No startup retirement | 25.5 / 7.0 | 52.0 / 7.5 |
The TCPD results show why we use learned startup comparisons. The TSSB results show why we retire them after startup. WEAR adds mature-stage transfer without retraining. These tests use the same recordings and are not an untouched confirmation. We describe the design in Sections 4.5 and 6.3. The new results are in Section 6.4, Table 4, and Appendix G, Tables 13–14.
Thank you again for helping us clarify these results. We truly appreciate the time you have spent reviewing our work!!
Thank you for your review. We updated the code package, added controls, and evaluated PARC on one additional dataset, WEAR. We updated the paper with revisions highlighted.
Reproducibility. Thank you for pointing out the missing grids. The updated ZIP now includes reproducibility details such as the detector, baseline grids, fitted configurations, recovery scripts, simulations, ablations and results.
Historical endpoint (New comparison). We expanded the ablation using the same proposals, preprocessing, and budget, with separate thresholds. D and E are detection and early-alarm percentages. N is no-change alarms and L is capped loss.
N averages five splits, with six TSSB and two TCPD no-change records. The maximum detects more changes with fewer early alarms than the mean. The mean has fewer no-change alarms on TSSB, and the 95th percentile is slightly better on TCPD. See Section 6.4 and Appendix F.
In addition, we added an ablation for the variability term based on your feedback.
PARC detects more changes with fewer early alarms than these controls. Each variant fits its own threshold, so the alarm rates differ.
History (new sensitivity experiment). Thank you for this point. We tested different numbers of historical blocks:
TCPD series are shorter and have earlier changes than TSSB. The 8- and 16-block variants detect mostly the same TCPD records, although their thresholds are fitted separately.
Cross-channel sensitivity (New simulations). Thank you for this point. We added a simulation study to include two-channel time series change in both correlation and either mean or scale. Each setting contains 100 time series (length 512). Delay is the median among detected changes.
PARC detects more changes as the mean shift increases, although many alarms are late. The diagnostic detects more of the smaller mean and scale changes.
For 16 channels, feature extraction takes 6.78 ms for PARC and 19.81 ms with pairwise correlations. These are feature-extraction times, not full-detector times. PARC’s channel-wise features are simpler to compute, but may miss changes that affect only relationships between channels. The full analysis is in Appendix H. Channel-wise extraction is also used by WEASEL+MUSE (Schäfer and Leser, 2018) in the related field of time-series classification.
Nonetheless, extending PARC to model cross-channel relationships is an important direction for future work, and we must address the computational cost as well. Thank you again for your suggestion.
Transfer to WEAR (The new dataset added for revision). We found and evaluated the frozen detector on a new dataset, WEAR, which is a separate wearable-sensor dataset. We used 14 participants to fit thresholds and 8 other participants for evaluation. We set a target threshold level of 10%. The results are:
In the above table, PARC has the highest detection rate and fewest early alarms. Some methods have fewer control alarms, but PARC is still the best when considering all three aspects. We selected 64-second windows to obtain enough shifts and same-activity control windows. The eight test participants were not used to fit thresholds. The detection difference remains uncertain across participants. We include the full protocol and results in Appendix G in the revised paper.
Thank you again for your valuable time, and we hope this can address most of your concerns.