- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper studies open-set time-series anomaly detection in a weakly supervised setting where the nominally normal training data contain a small fraction of anomalies, a few labeled anomalies are available only for some (seen) classes, and anomalies of unseen classes must also be detected at test time. The authors argue that existing open-set TSAD methods generate pseudo-anomalies heuristically without assessing their quality and overfit the seen anomalies. TURPLE has three parts: (1) deviation-based pre-training in which every nominally normal window is probed with a fixed Gaussian perturbation and the trajectory of its response over epochs (mean, Mann-Kendall trend, late shift) selects a bounded fraction of windows for relabeling as anomalies, (2) a learnable frequency-domain amplitude and phase augmenter that pushes labeled anomaly seeds away from the normal and anomaly prototypes to synthesize pseudo-unseen anomalies, (3) joint training of an unseen head with a sample-level gate that down-weights pseudo-anomalies flagged as harmful, and score fusion at inference. Three propositions motivate the design. Validation is window-level AUC over five runs on eight datasets against fifteen baselines under the general and hard protocols of prior work, with ablations, a sensitivity analysis, a contamination-rate sweep and a wall-clock comparison in the main text and further controls in the appendix.
Critical strengths
-
The label-refinement module is scored against known contaminants, not only through downstream AUC. Because the contamination is injected by the authors, Appendix H.7 can compare the flipped set with the known contaminants, reaching precision and recall of 0.83 against a precision of 0.32 for the last-epoch response alone (Table 9, p. 25 L1314-1319). It shows that the perturbation-response trajectory carries a contamination signal independently of the detection results, which I expect to be reusable by other weakly supervised detectors.
-
The empirical case is broad and the largest margins are not marginal. Table 1 (p. 7 L338-339) covers eight datasets and fifteen baselines in the general setting, and the reported margins over IMPACT on SAD (80.91 vs 68.13), PSM (72.42 vs 64.24) and CT (96.93 vs 91.96) are several times the reported standard deviations.
Critical weaknesses
-
The main concern is how the reported numbers were produced. The protocol cannot be reconstructed from the main text, and the selection of hyperparameters and runs is nowhere described. Section 4.1 states the protocol in one sentence (p. 7 L356-358) and leaves unstated what the 2% contamination is a fraction of, where the labeled anomalies come from, and what the validation split is used for, although any label-based use of a chronological validation split in the hard setting would expose the unseen-class labels that the problem definition withholds from training (p. 3 L142-143). Hyperparameter values and sensitivity curves are reported, but not the split or criterion that selected them, and the five runs behind each entry are not characterized (p. 7 L371-372, p. 30 L1615-1616). If test performance entered the selection of hyperparameters or runs, or if validation AUC over all classes drove selection in the hard setting, the reported margins would not measure what the paper claims, and the text rules out neither.
-
The claims outrun the evidence shown in the main text. Table 2 pools seen-class and unseen-class test anomalies into one AUC and the main text reports no unseen-class results, yet it reads Table 2 as "remarkable generalization capability" beyond the seen classes (p. 8 L420-427) and promises "consistent and robust superiority" on both (p. 2 L093). Several Table 1 margins over IMPACT lie within one standard deviation (UCR, SMD, p. 7 L338-339), four Table 2 rows go to IMPACT without mention (p. 8 L382-392), no test or interval on the differences is reported, and Table 3 prints no dispersion while the text says the gate "consistently contributes" (p. 9 L449-451), next to "remarkable superiority" (p. 8 L417). The question whether the claims are supported is thus answered in the main text by adjectives rather than by the reported variance.
Questions for authors
-
As noted under Weakness 1, Tables 1 and 2 report the mean and standard deviation over five independent runs (p. 7 L371-372), the hyperparameters are given as defaults, ranges and sensitivity curves without a stated selection split or criterion (p. 9 L454-459, p. 30 L1615-1616), and the main text describes no validation procedure beyond the 60/20/20 split, nor which anomaly-class labels that split exposes in the hard setting (p. 3 L142-143). Were the five seeds and the per-dataset hyperparameters behind each reported entry fixed before any test-set evaluation, for TURPLE and for every baseline alike?
-
As noted under Weakness 2, Table 2 reports one AUC per seen class, and the text reads the averages as "remarkable generalization capability" to unseen anomalies (p. 8 L420-422, L424-427), but the main text neither defines how seen-class and unseen-class test anomalies enter that AUC nor reports unseen-class performance separately. Is each hard-setting AUC in Table 2 computed on the pooled test set, or as an equal-weight average of a seen-class AUC and an unseen-class AUC?
-
Related to Strength 1, Tables 9 and 10 measure the label refinement against the injected contaminants on CT only, without stating the number of runs behind the TLR row, although the contaminants are known by construction on every dataset and at every contamination rate of Figure 5, and the main text claims the elimination of contamination across datasets (p. 8 L417-419). For each dataset and each contamination rate, what were the number of flipped windows and their precision and recall against the injected contaminants?
6 comments
Response to Reviewer 9nUB[2/2]
For Table 3, we now provide the ablation results for 5 runs as follows:
| Method | SAD (General) | SAD (Hard) | CT (General) | CT (Hard) |
|---|---|---|---|---|
| TURPLE w/o TLR | 78.12±0.35 | 67.30±0.45 | 94.90±0.22 | 81.38±0.52 |
| TURPLE w/o Gate | 79.31±0.42 | 70.52±0.31 | 94.18±0.35 | 80.76±0.41 |
| TURPLE w/o Aug | 79.79±0.51 | 69.05±0.62 | 93.61±0.48 | 79.82±0.65 |
| TURPLE w/o Pre | 78.95±0.38 | 68.41±0.40 | 95.20±0.25 | 80.55±0.39 |
| TURPLE (Ours) | 80.88±0.24 | 70.68±0.28 | 96.90±0.15 | 83.18±0.26 |
As shown in the table, adding the gate yields better results than removing it, which aligns with the claim in our original paper. We will update these results in our revision.
(3) Clarification of hard-setting: For the hard setting, we follow the experimental setup of IMPACT. Specifically, the hard-setting AUC is computed on the pooled remaining test set, where every test sample is equally weighted. In the original paper, we also provide the separate results for both seen and unseen anomalies in Appendix H (refer to Table 5) and our model consistently outperforms other baselines across all datasets on both seen and unseen anomaly detection.
Q3: For each dataset and contamination rate in Figure 5, how many windows did TLR flip, and what precision and recall did it achieve in identifying injected contaminants?
A3: Thanks for your comment! To better address your concerns, we provide detailed clarifications below.
(1) Number of runs: As mentioned on page 25, line 1309 of the original paper, our runs are 3.
(2) Clarification on contamination and elimination: The amount of contamination we introduced is a percentage of the training dataset. In the original paper, we referred to the ability to robustly and efficiently eliminate contamination, not to remove it 100 percent.
(3) Additional flipping results: To further address your concerns, we further conducted additional flipping experiments for other datasets; these calculations were all performed with a contamination rate of 2% in the training set.
| Dataset | Method | Precision | Recall | F1 | FFR (%) | AUROC | AUPRC |
|---|---|---|---|---|---|---|---|
| SAD | TLR (Ours) | 0.89±0.05 | 0.87±0.04 | 0.88±0.04 | 0.18±0.06 | 0.92±0.03 | 0.72±0.04 |
| PTBXL | TLR (Ours) | 0.91±0.03 | 0.89±0.04 | 0.90±0.03 | 0.12±0.04 | 0.95±0.02 | 0.78±0.03 |
| TUSZ | TLR (Ours) | 0.88±0.04 | 0.86±0.05 | 0.87±0.04 | 0.10±0.03 | 0.93±0.02 | 0.74±0.04 |
As reported in the table, our flipping results are close to 90% for both precision and recall, demonstrating the robustness and effectiveness of our identification and flipping capabilities. We will update the complete table in the revised version.
In light of these responses, we hope we have addressed your concerns, and hope you will consider raising your score. If there are any additional notable points of concern that we have not yet addressed, please do not hesitate to share them, and we will promptly attend to those points.
Reference:
[1]Durani, Walid, et al. "Weakly Supervised Anomaly Detection via Dual-Tailed Kernel." ICML. 2025.
[2]Zhou, X., Wang, Y., et al. "IMPACT: Influence Modeling for Open-Set Time Series Anomaly Detection." ICML 2026.
Follow-up questions on the author response
Thank you for the response. Since it refers to the IMPACT setting throughout, I read the IMPACT paper (ICML 2026) and its released code and compared them with the submission. The points below remain open.
On A1 (W1, Q1)
1. Seeds and selection. The response states that the hyperparameters were selected on the validation set and that the seeds were fixed before testing. IMPACT has no selection step. It reports default values, and its validation set is a random 20% of the training set used for influence estimation (Appendix E). Its code runs seeds 42 to 46, and the baseline entries of your Tables 1, 2 and 6 match the IMPACT paper (399 of 400 cells). Which seeds were used for TURPLE on each dataset? Could you point to the part of your released code that builds the validation set and performs the selection?
On A2 (W2, Q2)
2. Table 2. My Q2 asked how each hard-setting AUC in Table 2 is computed. The response states that it is computed on the pooled test set. A pooled AUC equals the mean of the seen and unseen AUCs weighted by their shares of the test anomalies. IMPACT's tables behave this way. On SAD, with one seen class of four, the four methods there satisfy Table 2 = 0.25 × seen + 0.75 × unseen in 15 of 16 cells (IMPACT on Digit-Zero, 0.25 × 77.08 + 0.75 × 58.11 = 62.85). TURPLE's Table 2 entries instead equal the unweighted mean of its seen and unseen AUCs in your Table 5 in all 20 rows (Digit-Eight, (84.21 + 54.39) / 2 = 69.30). The pooled weights would give about 61.8 there, below IMPACT's 63.86. The standard deviations do not fit either. Letter-G has ±3.46 in Table 2 against ±0.15 and ±1.26 in Table 5, and a mean of two values cannot vary more than the more variable one. Which definition produced the TURPLE column of Table 2?
3. Table 5. For the four baselines the unseen entries match IMPACT's Table 5 in 79 of 80 cells, but the seen entries differ in all 80. In the 16 class rows (the Mean rows left aside) the new mean equals 2 × Table 2 − unseen in all 64 cells, and 63 of them keep IMPACT's standard deviation (IMPACT on Letter-G is 99.60±0.22 there and 86.27±0.22 here). How were these entries obtained? With the published values IMPACT's seen AUC is above TURPLE's in 12 of 16 class rows, and your own Table 5 puts IMPACT ahead on the PTBXL unseen mean (62.83 vs 61.76). This does not agree with the statement that TURPLE outperforms the baselines on both.
Response to Reviewer 9nUB[1/2]
We are truly grateful for the time you have taken to review our paper and your insightful review. Here we address your comments in the following.
A1: Thanks for your comment! To better address your concerns, we provide detailed clarifications below.
(1) Selection of runs and hyperparameters: Following IMPACT[2] and WSAD-DT[1], we conducted 5 independent runs to demonstrate the robustness of our model rather than relying on a single random result. Furthermore, all hyperparameters were selected strictly on the validation set, with the train/validation/test splits also strictly following the IMPACT settings to prevent any test-set leakage. All hyperparameters and random seeds are fixed before testing.
(2) Contamination injection and data leakage: Following IMPACT[2] and WSAD-DT[1]'s settings, the 2% contamination is a fraction of the training set (i.e., we inject a number of anomalies equal to 2% of the training data size). These contaminated samples are selected from the test set and are strictly excluded from the final evaluation phase, thereby preventing any potential data leakage.
(3) Hard-setting label exposure: Regarding the exposure of anomaly types under the hard setting, we have added a column ("Seen anomaly") in Table 2 to explicitly describe the specific visible classes for each dataset.
A2: Thanks for your comment! To better address your concerns, we provide detailed clarifications below.
(1) Clarification of our claims: In the original text, the phrase "remarkable generalization capability" was followed by the statement that our method "achieved the highest average AUC across the four multi-class datasets and outperformed the strongest competing open-set method" (L421-423). Our intention was to highlight the strong overall performance under the hard setting, which reflects the model's good generalization ability. Similarly, the phrase "consistent and robust superiority" was meant to express that our method performs well in both the general setting (without labeled seen anomalies) and the hard setting (with seen anomalies).
(2) Marginal improvement: In Table 1, our framework outperformed the baseline IMPACT on all datasets, leading by at least one percentage point on each, with an average improvement of 4.58 percentage points. For UCR, our model outperformed IMPACT by more than one standard deviation (L338–L339). For SMD, this is because IMPACT performs well on certain seeds, thereby improving its mean performance. We conducted a significant test with 10 runs to investigate this:
As reported in the table, with 10 runs, our model significantly outperforms IMPACT and IMPACT's mean and standard deviation decreased, while our model remained consistent.
Regarding Table 2, our slight underperformance in a few cases occurs because IMPACT perturbs high-influence normal samples at the sample level, which naturally adapts to highly idiosyncratic morphological variations. In contrast, TURPLE performs prototype-repulsive extrapolation at the class level, designed to capture the broader, shared manifold of unseen anomalies rather than isolated sample fluctuations. Despite this variance in specific classes, TURPLE achieves the highest mean AUC across the five CT classes under both the hard setting and the general setting. This indicates that the prototype-guided extrapolation provides more consistent and generalizable coverage over the entire anomaly distribution, rather than overfitting to specific class-level advantages. To further address your concern, we report below the results for the datasets with a statistical significance analysis.
As shown in the results, our model significantly outperforms IMPACT on other seen anomaly classes and achieves the highest AUC across all datasets.