acceptodds
Under review as a conference paper at ICLR 2027

When Early Is Not Early: The Precursor Illusion in Time-Series Anomaly Detection

Abstract

Early anomaly warning requires both detectable evidence before event onset and an alarm that exploits that evidence. Yet common evaluations define pre-onset targets by temporal proximity and apply point adjustment, allowing reactive detections to receive credit as apparent predictions. We term this failure mode the precursor illusion. We introduce a model-agnostic, placebo-calibrated framework that first tests whether pre-onset intervals are distinguishable from matched normal operation, then tests whether frozen causal detector scores exploit the certified evidence, and finally isolates evaluation effects while preserving scores and thresholds. The test detects evidence in 43 of 47 independently annotated transitions in the 3W dataset, yet across five benchmarks it identifies pre-onset deviations in only 70 of 500 evaluable events, revealing sparse and dataset-dependent evidence within the specified descriptor family. Mean detector scores achieve above-chance discrimination between precursor and matched normal windows in none of the 35 model–dataset combinations after multiple-testing correction, and only the near-onset score peaks of LSTM-VAE separate SMD precursor windows beyond a selection control, although all seven evaluated models distinguish the 3W transitions from normal operation. Point adjustment raises mean pre-onset Shifted-warning F1 from 0.036 to 0.177 without changing any alarm time, and input-blind alarms at matched rates obtain higher adjusted F1 than every detector. These results show that benchmark warnability must be established independently before early-warning models are ranked.

Primary area:
datasets and benchmarks
Keywords:
Time-series anomaly detection, Early anomaly warning, Precursor evidence, Placebo calibration, Benchmark evaluation bias

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_TeryOpenReview ↗
Rating
3
Confidence
1
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

I deliberately didn't use an LLM to understand the paper, since it would still camouflage my educated guess. I hope the authors understand that this review will be less useful than the others from a technical standpoint. I, however, want to praise the readability of this paper: it's the best in my batch.

The authors identify the "precursor illusion" in time-series anomaly detection, where standard evaluation protocols extend target windows into pre-onset periods and use Point Adjustment (PA). This allows late alarms to falsely receive "early warning" credit without verifying whether pre-onset evidence actually exists. Then, they propose a placebo-calibrated (what is this?) framework to test for genuine precursor evidence using a descriptor family, introduce Matched-window Precursor AUC (MPA) to evaluate whether frozen detector scores exploit this evidence, and isolate evaluation effects while keeping detector outputs fixed.

Critical strengths

The paper is well-structured, and I love that you framed your research questions right from the get-go. It provides the reader with structure and immediately manages expectations. I'm in no position, however, to review this paper in confidence, as already stated. Hence, I'm giving it a weak accept, and leave it to my fellow reviewers to adjust the scoring. I appreciate the authors addressing the point adjustment problem in anomaly detection.

Critical weaknesses

None that I could identify.

Questions for authors

Is ∑~i−1\mathbf{\tilde{\sum}_i}^{-1} in Eq. 1 a covariance matrix? I'm not sure I followed the notation carefully, but I haven't seen it defined anywhere in the section.

3 comments
AuthorsOpenReview ↗

Brief Update on Our Response to Reviewer Tery’s Comments

Thank you very much for your careful review and your positive comments on the paper’s readability, structure, and presentation of the research questions. We appreciate your encouragement and your questions about concepts and notation that would benefit from further clarification.

Regarding Eq. (1), Σ~i\widetilde{\Sigma}_i denotes the shrinkage-regularized covariance matrix of statistical descriptors estimated from normal-operation data, and Σ~i−1\widetilde{\Sigma}_i^{-1} is its inverse. Intuitively, this term helps us measure how far a window departs from normal operation while accounting for the typical variability of the descriptors and their correlations. We will explain the meaning and role of these symbols more explicitly next to the equation.

Regarding “placebo calibration,” the “placebos” are control windows drawn from normal-operation records of the same device. We select time points within these records as hypothetical anomaly onsets and apply exactly the same calculations used for windows preceding actual anomalies. These time points simply serve to align the analysis windows; the underlying data remain normal-operation records. This provides a reference for how much change can arise from normal variation alone. We then compare the changes observed before an actual anomaly against this reference to assess whether they exceed what is typically observed in matched normal data. This is the basic idea of placebo calibration.

In addition, motivated by the limitations identified in this paper regarding precursor evidence and its validation in existing benchmarks, we have been developing a new real-world multivariate time series dataset. Data preparation was still underway at submission and is now complete. The dataset comprises multivariate time series collected from 48 electromechanical devices operating under real-world conditions, together with the corresponding system fault information. We will introduce this dataset for anomaly prediction and detection in our forthcoming response, with the aim of providing additional real-world data for precursor analysis and anomaly prediction evaluation. We will present this dataset in the appendix as a supplementary resource; its inclusion will not alter the paper’s main experiments or conclusions.

We will revise the main text to explain these concepts, the notation, and the overall research approach more clearly, and provide you with a more complete response. We need a few more days to complete this process and wanted to share this progress update in the meantime. We will submit our detailed response as soon as possible, and in any case before the deadline. Thank you again for your careful review, encouragement, and patience.

AuthorsOpenReview ↗

Response to Reviewer Tery

Thank you again for your patience and encouragement. As promised, we have now completed our detailed responses and revised the manuscript accordingly. We have addressed all concerns raised by the reviewers and further improved the manuscript through additional experiments, analyses, and revisions. We sincerely appreciate your positive feedback and the time you have taken to review our work.

Thank you, guys, for your response. I'll maintain my score since I also flagged to the AC that my review is just an educated guess. I'll leave it to my fellow reviewers for the decision.

Reviewer_wxGKOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary
  • This paper investigates the precursor illusion in time-series anomaly detection, where temporally shifted labels and point adjustment may credit reactive detections as early warnings without establishing detectable pre-onset evidence.

  • The authors propose a model-agnostic, placebo-calibrated framework that separates precursor detectability, model use of precursor evidence, and evaluation-protocol effects. The framework tests pre-onset deviations against matched normal operation, evaluates frozen causal detector scores using matched-window precursor AUC (MPA), and compares evaluation protocols while preserving scores and thresholds.

  • Experiments detect evidence in 43 of 47 annotated 3W transitions, but pre-onset deviations in only 70 of 500 evaluable events across five benchmarks, about 49 above the placebo expectation. None of the 35 benchmark mean-score MPA comparisons is significant after multiple-testing correction, whereas all seven detectors distinguish the 3W transitions; only the near-onset score peaks of LSTM-VAE separate SMD precursor windows beyond a selection control. Point adjustment substantially inflates warning F1 without changing alarm timing, and input-blind alarms at matched rates obtain higher adjusted F1 than every detector.

Critical strengths

1. Separation of precursor detectability from alarm timing

  • The paper separates three questions often conflated in early-warning evaluation: whether pre-onset intervals contain detectable evidence, whether detector scores exploit it, and how evaluation protocols credit alarms. This exposes a limitation of benchmarks that define pre-onset targets by temporal proximity without verifying their distinguishability from normal operation.

2. Controlled isolation of credit transfer across onset

  • Through trace- and threshold-preserving comparisons, the paper isolates the effects of evaluation targets and point adjustment while keeping detector scores and normalization fixed. It demonstrates that post-onset alarms can receive pre-onset warning credit without earlier detection, and that input-blind alarms at matched rates can outperform the evaluated detectors under point adjustment.
Critical weaknesses

1. Limited predictive interpretation and robustness of precursor evidence

  • The test detects evidence in 43 of 47 annotated 3W transitions (§4.1), but 29.2% of pre-transition verification windows are also rejected under a separate chronological split (Appendix B.1.2). Since the test identifies deviations rather than necessarily fault-specific precursors (§5), the positive-control results may partly reflect operating-state drift.

  • A five-step delayed onset produces 93% apparent precursors in precursor-free synthetic events (§4.1; Table 6). Although an onset-continuity check is reported for SMD (Table 8), corresponding benchmark checks are not reported for PSM or SWaT, whose dataset-level significance is also sensitive to placebo-pool assumptions (§4.2; Appendix B.2.2).

2. Incomplete interpretation of negative model results

  • None of the 35 benchmark mean-score MPA comparisons is significant after correction (§4.3; Table 3), yet near-onset LSTM-VAE scores and per-machine iForest exhibit stronger discrimination (§4.3; Tables 18–19). This leaves the effects of temporal aggregation and experimental configuration insufficiently isolated.

  • Statistical power further limits interpretation: the minimum detectable MPA on SMD is approximately 0.60–0.66 (§4.3), while PSM and SWaT have few precursor-positive events (Tables 2 and 14). Non-significance therefore does not establish that detectors cannot exploit precursor evidence.

Questions for authors

Q1. Predictive interpretation of precursor evidence (Weakness 1)

  • Under the chronological split, 29.2% of pre-transition verification windows are rejected (Appendix B.1.2). Does the 43/47 recovery on 3W hold when the placebo null is constructed from normal operation immediately preceding each transition? Could this analysis be extended from the 16-record subset to all 47 evaluable instances?

Q2. Robustness of benchmark precursor classifications (Weakness 1)

  • Could the authors apply the onset-continuity check (Table 8) to precursor-positive events in PSM and SWaT and report how many are classified as boundary shifts?

Q3. Attribution of negative MPA results (Weakness 2)

  • Although none of the 35 benchmark mean-score MPA comparisons is significant after correction (§4.3; Table 3), stronger discrimination appears in near-onset LSTM-VAE and per-machine iForest analyses (Tables 18–19). Could the authors isolate the effects of temporal score aggregation and training configuration while keeping the precursor event set and normal reference fixed?

Q4. Statistical power and interpretation of non-significance (Weakness 2)

  • Given the minimum detectable MPA of approximately 0.60–0.66 on SMD and the small precursor-positive sets for PSM and SWaT (Tables 2 and 14), could the authors clarify which effect sizes are excluded by the reported bounds, including the upper 95% bounds of at most 0.656 on SMD, and distinguish non-significance from evidence of negligible precursor discrimination (Abstract; §4.3)?
7 comments
AuthorsOpenReview ↗

Brief Update on Our Response to Reviewer wxGK’s Comments

Thank you very much for your careful review and your interest in the research question we address. Your specific questions have provided valuable guidance for further refining our analyses and clarifying our conclusions.

We have been working to address the four questions you raised and are now consolidating the relevant analyses and explanations into a detailed, point-by-point response. We need a few more days to complete this process and wanted to share this progress update in the meantime. We will submit our detailed response as soon as possible, and in any case before the deadline.

Since submitting the paper, we have continued to refine this work and investigate time series anomaly prediction in greater depth. Motivated by the benchmark limitations identified in this paper, we have also been developing a real-world multivariate time series dataset for anomaly detection and prediction. Collecting and preparing data from real devices requires substantial time, and the data preparation was not yet complete when we submitted the paper. This preparation is now complete, and in our forthcoming response, we will introduce this dataset, comprising multivariate time series collected from 48 electromechanical devices under real-world operating conditions, together with corresponding system fault information. We hope this resource will facilitate the investigation of precursor evidence and the evaluation of anomaly prediction methods. We will present this dataset in the appendix as a supplementary resource; its inclusion will not alter the paper’s main experiments or conclusions.

Thank you again for your careful review, constructive feedback, and patience.

AuthorsOpenReview ↗

Response to Reviewer wxGK (1)

W1:

Thank you for highlighting the effects of drift during normal operation and uncertainty in onset annotations on the interpretation of precursors. We have added two analyses: an assessment of detection robustness using local normal references across all 3W instances, described in our response to Q1 below, and an extension of the onset-continuity check to PSM and SWaT to distinguish boundary-shift patterns from other positive events, described in our response to Q2 below.

W2:

Thank you for pointing out that temporal aggregation, experimental configuration, and statistical power must be considered separately. We have added paired experiments with fixed event sets and normal controls to examine the effects of aggregation, training configuration, and the normal reference separately, as detailed in our response to Q3 below. We also clarify the effect sizes excluded and still permitted by the existing confidence bounds and provide an additional equivalence sensitivity analysis, as detailed in our response to Q4 below.

Our detailed responses to your questions are provided below.

Q1:

We have added two analyses: (1) an examination of the source of the original 29.2% rejection rate for normal windows; (2) a reconstruction of the reference using normal data close to the transition across all 47 instances in the original paper.

First, the 29.2% rejection rate is not distributed uniformly across records. This value is obtained by calculating the proportion of rejected normal windows within each record and then averaging equally across 16 records; it does not mean that 29.2% of records contain false alarms. The record with the highest rejection rate reaches 100%, indicating that a small number of records with high rejection rates substantially influence the overall result.

To explain these cases, we have presented the corresponding raw sequences in Appendix B.2.1, marking the normal reference intervals, annotated fault-transition intervals, anomaly intervals, and windows flagged by the precursor test. We focus on whether some channels begin to change before the annotated transition and whether these changes temporally coincide with the precursor test rejecting the null hypothesis within the normal validation interval (i.e., identifying a significant difference from normal windows). This visualization shows the signal changes to which the test responds, rather than reporting only an average rejection rate.

In the records examined, signals within the windows flagged by the precursor test show clear changes relative to the normal reference interval. This indicates that these rejections correspond to actual signal deviations, although those deviations are not reflected in the original dataset labels. The corresponding raw-sequence plots are provided in Figure 6 of Appendix B.2.1.

Second, to address your question, “Do the 3W detection results still hold when the placebo null distribution is constructed from normal operation immediately before the transition, and can this analysis be extended to all 47 instances?”, we retain all 47 records from the original paper and examine whether detection remains robust after changing the normal reference. We fix the original pre-fault target windows, test statistic, and significance threshold, and construct local null distributions using up to 300, 900, and 1800 seconds of normal data close to the start of the transition.

Normal-reference settingInstances includedInstances detectedDetection proportion
Original paper setting474391.5%
Local normal reference: 300 seconds474595.7%
Local normal reference: 900 seconds474595.7%
Local normal reference: 1800 seconds474391.5%

For the 900-second setting, 42 of the original 43 positive instances remain positive and 1 becomes negative; 3 of the original 4 negative instances become positive. Thus, the high detection rate is retained across all 47 instances when normal references close to the transition are used, and is stable across the reference lengths compared.

(To be continued in the next comment.)

AuthorsOpenReview ↗

Response to Reviewer wxGK (2)

(Continued from the previous comment.)

Q2:

We have extended the continuity check in Table 8 (now Table 11 in Appendix B.2.3) to all originally precursor-positive events in PSM and SWaT; 5/7 and 7/9, respectively, are classified as boundary-shift patterns.

Specifically, an event is classified as an “onset jump” if a jump at the annotated onset exceeds the normal threshold. If there is no significant jump at the onset but an earlier jump exceeds the threshold and has the same direction as the post-annotation change, the event is classified as a “boundary-shift pattern.” Events satisfying neither criterion are categorized as “unclassified.” The original positive-event sets are held fixed, with the following results:

DatasetOriginally precursor-positive eventsOnset jumpBoundary-shift patternUnclassified
SMD (verification control)513867
PSM7151
SWaT9270

The 38/6/7 counts for the SMD positive group match the original Table 8. Boundary-shift patterns account for 71.4% and 77.8% of the original positive events in PSM and SWaT, respectively, with the corresponding jumps occurring 4–41 minutes and 8–27 seconds before the annotated onset. The “unclassified” events also exhibit detected pre-onset deviations; they simply do not satisfy either of the two morphological criteria above.

Here, “boundary-shift” is a diagnostic classification based on signal morphology and does not imply a confirmed annotation error. If the annotated onset occurs later than the actual fault onset, alarms before the annotated onset may still constitute post-onset detection. If the signal changes genuinely precede the actual fault onset, they may provide evidence for early warning. Thus, changes occurring before the annotated onset may have different implications for early warning. This further indicates that onset labels used for anomaly detection do not automatically provide a reliable basis for evaluating early warning. When used for anomaly prediction, the actual meaning of the annotated onset and the relationship between pre-onset changes and fault development must also be verified.

(To be continued in the next comment.)

Reviewer_pSK8OpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
1
Soundness
3
Read review
Paper summary

Time series anomaly detection is reactive: it can only be detected once the anomaly appears. This paper instead looks at precursor detection, where the goal is to detect the anomaly before it actually happens. To this end, the paper first introduces a test to see whether an anomaly has an actual detectable precursor, and show that this is often not the case in many real world benchmarks. Additionally, the authors introduce MPA, a new evaluation metric for precursor detection.

Critical strengths
  1. Interesting problem. The tackled problem is very interesting, and understudied in the general literature, which mostly focuses on anomaly detection, after the event has already happened. While I believe that such reactive anomaly detection is already useful in many applications, since it helps flagging events of interest which do not necessarily break the system, adding an additional focus on precursors shifts the problem in an interesting direction.
Critical weaknesses
  1. Technical contribution. The technical contribution of this paper is quite limited. A statistical test is proposed to check whether there is a precursor, but this test contains many choices which were not validated. Similarly, the proposed evaluation metric MPA was only compared to pointwise and point-adjusted F1 scores, while neglecting the potential to adapt more recent anomaly detection metrics (range-based, affiliation, VUS) for evaluating precursors.

  2. Presentation. It was very hard to understand the paper and it took several iterations for me to be able to understand and evaluate the quality of the work. In general, there were two types of issues which prevented me understanding the paper.

    1. There were many terms and concepts introduced that are non-standard and non-trivial, but assumed to be. The text then continues to use these without giving a clear interpretation or formal definition of the concept. Only by thoroughly reading the text, you understand the concept, and then you can re-read the text to actually know what is happening. An academic paper should be understandable after one read, with a second read only being necessary to verify the deeper technical details.
    2. Sometimes decisions were made without clear argumentation why it was made. This only rises questions to a reader, which can be prevented easily. In some cases, a (partial) answer is in the appendix, but then there is no clear reference to the appendix in the main text (e.g., line 299, why only 5 morphologies, all results are in appendix, and there is an appendix reference but only at the end of the results of section 4.1 without any details). While more explicit references would help, argumentation in the main body would be better since it should be understandable by itself even without appendix.
Questions for authors

Detailed review: In the current state, I would reject this paper. Although an interesting problem is investigated, the limited technical contribution and poor presentation make me believe that the paper is not yet ready. If the presentation is substantially improved during rebuttal, I would update my rating to a weak reject, since I feel like the technical contribution can not be changed substantially within a limited time frame. I do believe, however, that this is interesting work that could benefit from additional work and rigour.

Below, I first list the main questions I had while reading your paper, after which I give some small comments and suggestions that could be useful for a camera-ready version/next iteration.

Q1: Your test measures whether there is a precursor in the existing benchmarks. For an anomalous event Ae=[te,ue]A_e = [t_e, u_e] and width WW, would it be possible to set Pe(W)=[te,te+W)P_e(W) = [t_e, t_e + W) and the remaining anomaly is then Ae′=[te+W,ue]A_e' = [t_e + W, u_e] (where you deal with anomalous intervals that are shorter than WW)?

Q2: The precursor test (section 3.2) in some sense boils down to computing a shift in features/sequence characteristics of two consecutive windows. How do you guarantee this test catches all the precursors, since it can also be interpreted as a model to detect precursors? Additionally, why did you not use any other model (change point detection, self-supervised model, ...) to verify whether the pre-onset window is a visible precursor?

Q3 [smaller question]: is it possible that some information is missing from table 2 and figure 4? Section 4.3 refers to these results for the discussion, but mentions things like "On SMD, the rejection rate halves within 10 minutes" and "On 3W, it stays above 0.91 at every level up to 200 seconds". However, these tables and figures do not show absolute rejection rates at different time intervals of the onset (and no results at all about 3W). I quickly scanned the appendix B2 but could also not find this information.

Q4: The related work section mentions that "Pointwise F1 poorly represents interval-valued anomalies, whereas point adjustment may substantially inflate performance by counting an entire segment as detected after a single hit" (line131). To deal with this, MPA is introduced to better measure whether the precursor is detected. I have several questions regarding this: Q4a: Did you evaluate whether MPA is more reliable than other metrics? Now, you only show the results using MPA without actually evaluating it. Q4b: Why did you not evaluate using existing anomaly detection metrics (such as range-based, affiliation-based, VUS, which were referenced)? You could also apply minor adjustments to these to not take into account late detections (e.g., for affiliation, you could make sure that the affiliation regions always and at an anomaly). In addition, the #todo Tatbul. Q4c: to avoid MPA being a pointwise metric, you average the anomaly scores across an anomalous interval. Could you also do this for F1, in which case it could also be a reliable metric for precursor detection?

Q5: Can you show the precursors on real time series? Now you only show some trivial synthetic examples. It would be interesting to see some of the samples in which there is a precursor according to the test and some samples in which the test fails.

Minor comments/suggestions:

  • The abstract and introduction (especially first paragraph) assume a precise definition of the anomaly onset and pre-onset and near-onset, without actually defining it. A short description of what the "onset" means would be helpful. You could, for example, also interpret it as the complete anomalous event. The tet_e notation also does not necessarily help, because it is not clear that tet_e is the start of the anomalous event.
  • Throughout the text, it was several times mentioned that the detectors are causal without any further explanations. However, most of the anomaly detectors do not leverage causal relations. I suggest dropping causality from the text, to avoid this ambiguity and simplify the text.
  • The research questions are very nicely posed and come back through the methodology. This unfortunately stops at the experimental evaluation (section 4), where you would expect the questions to be answered. I believe this part can be restructured. The main thing would be to formulate a clear and explicit answer to the research questions, beyond simply mentioning the numbers in the tables.
  • Several non-trivial concepts (event-placebo exchangeability, paired window exchangeability) are mentioned in passing, followed by some theoretical claim. There is no additional explanation as to why the claim follows the concept, nor is there a reference for the concept. Adding this would substantially improve understanding.
  • Line 213: a simple intuition about what the rejection rate is (i.e., the percentage of R and Q that are rejected from showing a difference) would help the understanding. I could also be useful to add an (inline) formula of this.
  • In fig. 3: there are only 5 out of 8 morphologies tested, without argumentation or clear reference to section B1.1 where all morphologies are shown. It would be useful to already explicitly mention in the setup of section 4.2 that this information is available in appendix, but also an argumentation as to why the specific subset of morphologies is shown and not everything.
  • Line 355: discuss why are only 500 out of 537 events eligible. It is not clear what the criteria for being eligible is. It is not having a precursor, since otherwise all 500 events would be in PP.
11 comments
AuthorsOpenReview ↗

Response to Reviewer pSK8 (1)

W1: We sincerely thank you for raising these questions, which help us provide a more complete technical explanation of the study. In response, we have expanded the rationale for the design and choices underlying our methodology, as follows:

  1. We compare the proposed metric, MPA, with adapted range-based and affiliation-based metrics and VUS (see our detailed response to Q4 below).

  2. We have clarified the overall logic of our study to better explain the rationale behind our methodological choices and how we validate the precursor test (Q2). Specifically, our reasoning proceeds as follows:

First, we introduce our precursor test statistic T in the methodology section. Given the space constraints, we have added the rationale for constructing this precursor-detection module in Appendices A.2 and B.1.3, addressing your question in Q2 about why existing change-point detection or self-supervised models were not used (see our detailed response to Q2 below). We also clarify your question in Q1 about whether a segment of data after an anomaly has already begun can be treated as a precursor for predicting subsequent anomalous behavior (see our detailed response to Q1 below).

Second, how can we demonstrate that the proposed statistic identifies “precursors”? In the experimental evaluation (RQ1), we show that the precursor test statistic T identifies precursors in the data. Datasets with ground-truth “precursor” annotations are extremely scarce, and 3W is one such dataset that we were able to find. On 3W, where precursor ground truth is available, T detects precursors in 43/47 cases, demonstrating that it can identify the annotated precursors in a substantial majority of cases.

Third (newly added), how can we demonstrate that the identified “precursors” are useful for predicting faults? Because of space constraints, we have added counterfactual ablation experiments in Appendix B.1.4 to examine whether the detected deviation information contributes to identifying pre-fault windows. Specifically, using normal operation from the same oil well in 3W as a reference, we replace the mean level and linear trend of the pre-fault sequence with those of the normal state, attenuating the detected deviation while preserving other fluctuations as far as possible. Across the same 47 3W events used in the original paper, the number of positive T tests decreases from 43 to 0 after ablation. Positive examples are the windows comprising the final 100 sampling points before the stable-fault stage for these 47 events. For each event, we select 8 equal-length normal windows from the same record as controls, yielding 47 positive and 376 negative examples (423 windows in total) for computing model performance. All metrics are window-level metrics, and no point adjustment is applied. The table below reports metrics for commonly used anomaly-detection, anomaly-prediction, and supervised-classification methods on the original and ablated inputs, where “→” indicates the change from before to after ablation.

ModelAUROC ↑AUPRC ↑Recall ↑Precision ↑F1 ↑MPA ↑
IForest0.795 → 0.5870.453 → 0.2160.787 → 0.3190.230 → 0.1080.356 → 0.1610.823 → 0.600
DAGMM0.783 → 0.5970.594 → 0.2690.830 → 0.5110.163 → 0.1070.273 → 0.1770.806 → 0.630
USAD0.972 → 0.6260.885 → 0.3301.000 → 0.5320.168 → 0.0970.287 → 0.1640.989 → 0.689
LSTM-VAE0.967 → 0.6570.875 → 0.3421.000 → 0.5530.162 → 0.0970.279 → 0.1650.976 → 0.689
Anomaly Transformer0.623 → 0.5790.314 → 0.2210.553 → 0.2340.271 → 0.1360.364 → 0.1720.672 → 0.585
FCM0.949 → 0.6640.822 → 0.3180.979 → 0.5740.175 → 0.1110.297 → 0.1860.973 → 0.673
A2P0.895 → 0.5880.768 → 0.2880.894 → 0.2340.300 → 0.1010.449 → 0.1410.934 → 0.590
HGB0.874 → 0.5680.399 → 0.1430.660 → 0.1490.408 → 0.1350.504 → 0.1410.880 → 0.573
RF0.883 → 0.6380.474 → 0.1580.511 → 0.0210.421 → 0.0290.462 → 0.0250.896 → 0.681

After replacing the level and linear trend with the normal reference, the models' AUROC values generally move toward the random-ranking level of 0.5. MPA also decreases, indicating a reduction in the difference between pre-fault windows and normal controls. AUPRC, Recall, and F1 decrease for every model as well. These consistent changes support the contribution of the ablated components to identifying pre-fault windows and their value for fault prediction. We provide a detailed explanation of MPA's reliability in our response to Q4.

(To be continued in the next comment.)

AuthorsOpenReview ↗

Response to Reviewer pSK8 (2)

(Continued from the previous comment.)

Fourth, on public datasets without precursor ground-truth annotations that are nevertheless commonly used for anomaly prediction (some were originally designed for anomaly detection and later adapted by shifting labels backward), we find relatively few precursors: 70/500.

Fifth (newly added), given the scarcity of real-world multivariate event-sequence datasets for anomaly prediction, we introduce a new benchmark dataset. We recognize that the rebuttal stage should not introduce material that changes the main results of the original paper. We therefore include the dataset in the supplementary materials and describe it in Appendix C, without affecting the main conclusions of the submitted paper. We hope that this data resource will support subsequent research in this field.

W2:
We particularly appreciate your comments regarding the difficulty of following the paper. The comments have helped us improve readability. In response, we have revised the manuscript comprehensively: we have improved the writing throughout and added formal definitions and explanations of the proposed concepts to the abstract and introduction. Wherever appendix content is discussed, the corresponding references are now provided in the main text. Further details can be found in the revised manuscript PDF.

Our detailed responses to your questions are provided below.

Q1: Technically, the first W time points after an anomaly begins can be used to predict its subsequent evolution, but this defines a different task from the anomaly early-warning problem studied here.

Our study is motivated by early-warning requirements in real industrial systems. In complex production systems, such as advanced equipment and precision manufacturing, anomalies can reduce product yield, damage equipment, and cause unplanned downtime. The central objective of early warning is therefore to identify signs of an anomaly's development before it occurs, allowing time for intervention before onset, reducing the resulting losses, and, where feasible, preventing the anomaly.

As suggested, defining $ P_e(W)=[t_e,t_e+W) $ as a precursor window would mean using data already labeled as anomalous to predict the subsequent portion of the same anomalous event. This is useful for studying anomaly-evolution prediction or warnings of fault deterioration and has practical value. However, it does not answer the central question of this paper: before an anomaly has begun, is there detectable precursor evidence, and can a model exploit it to raise an alarm in advance? We therefore retain the definition of candidate precursor windows before anomaly onset to distinguish anomaly early warning from post-onset detection and evolution prediction.

Q2: We agree that no method based on a finite set of features or a particular detection mechanism can guarantee the discovery of every possible precursor. Accordingly, the precursor-test results should be interpreted as detectable evidence under a specific detection mechanism; a negative test does not imply that no precursor exists.

To assess the effectiveness of the original method and the influence of different detection mechanisms, we add comparisons with three representative methods: Online Kernel CUSUM, a modern kernel-based CUSUM change-point detector (Wei & Xie, 2026); MMDEW, an efficient MMD-based online change-point detector using exponential windows (Kalinke et al., 2025); and TS2Vec, a self-supervised time-series representation-learning method (Yue et al., 2022). All methods use the same candidate precursor windows and the same normal-data splits for fitting, calibration, and validation. Their test thresholds are calibrated consistently using normal controls.

The results on 3W are as follows:

MethodDetected instances, overlap with our methodDetection proportionNormal FPR
Our method43/4791.49%5.033%
Online Kernel CUSUM40/47, 40/4385.11%4.810%
MMDEW33/47, 31/4370.21%4.4983%
TS2Vec42/47, 39/4389.36%7.5291%

(To be continued in the next comment.)

AuthorsOpenReview ↗

Response to Reviewer pSK8 (3)

(Continued from the previous comment.)

The results across the 500 evaluable events in the five benchmark datasets are as follows:

MethodDetected events, overlap with our methodDetection proportionNormal false-positive rate (FPR)
Our method70/50014.0%4.58%
Online Kernel CUSUM42/500, 35/708.4%5.02%
MMDEW32/500, 28/706.4%4.73%
TS2Vec63/500, 57/7012.6%4.72%

At comparable normal false-positive rates, our method detects more pre-anomaly deviations. This result can be explained by the methods' detection mechanisms. Online Kernel CUSUM accumulates evidence of distributional changes through kernel statistics and is suited to identifying newly emerging changes during continuous monitoring. MMDEW performs multiscale distributional comparisons using exponential windows and is suited to detecting abrupt changes and distributional drift at different timescales in data streams. TS2Vec learns complex temporal representations through self-supervision and can capture changes beyond predefined statistical features, but its detection performance depends on whether the learned representations preserve the relevant anomaly information and on the subsequent scoring mechanism.

In contrast, the advantage of our method is that it jointly tests local changes and overall deviations from normal operating conditions, accommodating uncertainty in the form and duration of pre-anomaly changes. The local-change branch captures newly emerging changes in features such as the mean, variability, trend, and relationships among variables. The normal-state-deviation branch identifies abnormal states that have already developed and persist, potentially detecting differences from historical normal patterns even when no clear new change point occurs within the candidate window. These complementary sources of evidence reduce the detection blind spots that can arise from relying solely on local changes or a single anomaly-scoring mechanism. In addition, explicit multiscale statistical descriptors make the data changes underlying a detection interpretable, while independent normal controls calibrate statistical significance. The method therefore provides a default mechanism for evaluating precursor detectability that accommodates different forms of change and has an explicit statistical interpretation.

At the same time, we do not claim that the proposed statistic covers all precursors or outperforms existing methods for every type of change. Change-point detection and self-supervised representation methods can also serve as interchangeable evidence-detection components, allowing researchers to select methods according to the characteristics of their data.

Q3. Thank you for identifying the information gap between the figures and tables and the accompanying text. We have revised Table 2 to add event rejection rates at lead times of 10, 50, 100, and 200 sampling steps for all five benchmarks, alongside the original zero-lead results. The table note specifies each dataset's time unit and how rejection rates are calculated. The graphical content of Figure 4 remains unchanged: it continues to show the excess of the event rejection rate over the normal rejection rate and its statistical significance. We have revised the accompanying text and caption to distinguish the event rejection rates in Table 2 from the excess rejection rates in Figure 4, and have explicitly linked the numerical comparisons in the main text to Table 2. For 3W, we add the following details to the main text: among the ten reported lead times, the rejection rate is 0.936 at leads of 10 and 20 seconds and 0.915 at the other eight positions, covering the range of 0–200 seconds before onset.

(To be continued in the next comment.)

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.