- Rating
- 3
- Confidence
- 3
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper addresses the problem of imbalanced support in the distribution of normal training data for anomaly detection. The authors propose RestoreAD, which uses uniform regularization to prevent concentration on a single candidate, retrieves normal references for anomaly scoring and support-difficulty estimation, and applies CSNC to align the scores. Experiments on TSB-AD-M examine detection performance and support-dependent false-positive disparities. A controlled experiment varies training support while keeping the representation geometry symmetric.
Critical strengths
- Considering support imbalance: Anomaly detection performance can be significantly affected by how well different regions of the normal-data distribution are represented in the training data. Explicitly considering this imbalance when calculating anomaly scores is a strength of this paper.
- Theoretical analysis of CSNC: The authors provide theoretical bounds for both population and empirical medians, which strengthen the theoretical support for the proposed approach.
Critical weaknesses
- Implications of reduced disparity: Proposition 1 provides a theoretical interpretation of cross-support comparability, and Figure 3 shows a smaller false-positive rate (FPR) gap together with a slight increase in the mean anomaly true-positive rate (TPR). However, the practical implications of reduced disparity are not sufficiently clear. Reporting the overall held-out FPR and TPRs across support bins at the existing training-Q95 thresholds would help clarify where the improvements occur.
- Temporal-overlap exclusion: Temporal-overlap exclusions are specified for relation regularization and reference-isolation estimation, but it is not clear whether the same exclusion applies to query-to-reference retrieval in Equation (6).
Questions for authors
- Can the authors explain more clearly the practical implications of reduced disparity, including how the improvements are distributed across different support regimes?
- Temporal-overlap exclusion is specified for in Equation (5). Does the same exclusion apply when constructing in Equation (6), particularly when training windows are scored for calibration? Please clarify how identical and overlapping query–reference pairs are handled, since these scores determine the CSNC medians and Q95 thresholds.
6 comments
Response to Reviewer RRjw (Part 2/2; Q2)
Response to Reviewer RRjw (Part 2/2; Q2)
Q2. Temporal exclusion in query-to-reference retrieval
In the submitted implementation, the temporal exclusion used for reference-isolation estimation in Eq. (5) is not applied to query-to-reference retrieval in Eq. (6). Retrieval uses the full reference bank, including during training-window scoring. Identical source windows and temporally overlapping references can therefore enter the scores used to fit CSNC medians and Q95 thresholds.
To assess this sensitivity, we froze the encoder and reference bank and compared three query-retrieval policies: unmasked retrieval, exclusion of identical source windows, and exclusion of all temporally overlapping windows. Under the supplemental strict non-overlap policy, admissible reference slots satisfy
where and are source-window start indices within the same series. This excludes every overlapping reference slot, including duplicate slots representing the query window. We retain without relaxing the mask.
For each policy, we recompute query scores and support estimates, then refit CSNC bin boundaries, medians and score-version-specific Q95 thresholds using only training-prefix scores. For comparable group-wise reporting, evaluation support-group membership remains fixed to the original unmasked setting, independently of the refitted CSNC calibration bins.
Across all 180 M180 series and three seeds, strict non-overlap retrieval gives the following results at the separately refitted training-Q95 thresholds:
| Measure | Raw | CSNC |
|---|---|---|
| Overall FPR ↓ | 20.37% | 18.87% |
| Worst-bin FPR ↓ | 49.23% | 41.49% |
| Cross-bin FPR gap ↓ | 43.82 pp | 35.08 pp |
Metrics are computed within each series–seed pair, then averaged over valid paired seeds within each series and equally across series. The gap has 179 valid series; other measures have 180. Mean worst-bin FPR decreases by 7.74 pp, with a descriptive 95% paired series-bootstrap interval of [6.30, 9.28] pp. The reductions also persist after excluding the first 95 test points to remove boundary-crossing input windows.
We will explicitly document the stage-specific exclusion rules and include complete FPR/TPR results for all three retrieval policies in the revised appendix. CSNC’s reductions in mean worst-bin FPR and cross-bin disparity persist under strict non-overlap retrieval.
Thank you again for your thoughtful feedback. We will incorporate these additional analyses and clarifications into the revised manuscript to strengthen the empirical evidence and clarify the implementation details.
Best regards,
Authors.
Thank you for the clarification and additional analyses. The matched-FPR comparison and the strict non-overlap results substantially address my concerns. Please incorporate these results and the retrieval exclusion rules into the revised manuscript. Please also report the overall held-out FPR at the original training-Q95 thresholds and the complete FPR/TPR comparisons across the three retrieval policies.
Response to Reviewer RRjw (Part 1/2; Q1)
Response to Reviewer RRjw (Part 1/2; Q1)
Dear Reviewer RRjw,
Thank you for your careful review and constructive suggestions. We appreciate your recognition of the importance of normal support imbalance and the theoretical support for CSNC. Your questions have helped us strengthen the decision-level evaluation and clarify the retrieval and calibration protocol. We address your comments below.
Thank you for highlighting the need to connect reduced disparity to decision-level outcomes. The practical benefit is lower mean worst-bin FPR at the same overall false-positive budget. We added analyses on all 180 M180 series and three seeds, comparing the same RU detector before and after CSNC with identical estimated support-group membership. We report support-wise training-Q95 results, followed by a matched-budget comparison.
Support-wise results at training-Q95 thresholds. Raw and CSNC each use the 95th percentile of their respective training-score distributions. The table reports mean anomaly TPR (%) ↑ for the ten-bin evaluation subset (392/540 series–seed cases; 141 distinct series). Bins increase in estimated support difficulty; n denotes valid paired series with anomalies in each bin.
Across all 180 series, training-Q95 mean overall anomaly TPR is 85.73% → 86.45% (Raw → CSNC). Mean TPR gains are largest in the lower-difficulty bins, with the lowest-difficulty bin increasing from 66.36% to 81.13% (+14.77 pp). At the highest-difficulty end, mean TPR remains around 97%, while FPR decreases from 89.78% to 88.76% (135 valid series). These results describe how sensitivity and false positives change across support groups under training-Q95 thresholding. Since Raw and CSNC yield different overall test FPRs at these thresholds, we additionally compare them at the same overall false-positive budget below.
Comparison at the same overall false-positive budget. At an offline matched expected overall test FPR of 5%, we obtain:
Using unrounded paired results, the mean worst-bin FPR reduction is 1.71 pp, with a descriptive 95% paired series-bootstrap interval of [1.33, 2.11] pp. Worst-bin FPR and FPR gaps are computed within each series–seed pair, then averaged over seeds within each series and equally across series. The gap has 179 valid series; other measures have 180. Matched working points use test-normal labels only for offline evaluation, not CSNC fitting or deployment threshold selection.
At the same overall false-positive budget, CSNC lowers mean worst-bin FPR and cross-bin disparity, with similar mean overall anomaly TPR. These results support its practical benefit in reducing the concentration of false positives across estimated support groups. We will include additional bin-wise analyses and sample counts in the revision.