- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
IDEAS applies a base detector independently to each channel and to pairwise differences of highly correlated channels, centers the resulting scores, and aggregates them with a selected operator. The largest improvements appear on six synthetic benchmarks. On 20 real-world datasets, the method is competitive with existing detectors rather than clearly superior.
Critical strengths
- The procedure is fully specified: a correlation threshold, independent scoring, mean centering, and aggregation by minimum, maximum, median, or a tail-based weight. The dimension score is the correlation between a channel score and the aggregated score. Section 3 is sufficient for reimplementation.
- When anomalies occupy few channels, the gain in VUS-PR is large. The reported factor of 3.8 corresponds to dataset B with 20 dimensions, and the one- and two-channel groups of dataset A show a comparable margin.
- Dataset F isolates cross-channel anomalies. Lowering the correlation threshold from 1.0 to 0.9 introduces difference channels and produces a clear increase in VUS-PR, which supports their contribution in this setting.
- The real-world results are reported in full, and the runtime comparison separates efficient univariate detectors from joint multivariate models.
Critical weaknesses
- In datasets A–E, the anomalous channels are designated in advance, and the anomaly class is sampled independently for each channel. This protocol favors detectors that score channels separately. The abstract claim of consistent improvement is therefore largely a consequence of the data-generating process.
- The default aggregation operator is chosen by the mean ratio-to-winner on datasets A–F, which are also the reported evaluation sets. The correlation threshold is selected in the same setting. Selection on the evaluation collections is not separated from the final comparison.
- On the 20 real-world datasets, EGI-IDEAS ties PCA and OmniAnomaly under the reported mean ratio-to-winner. The authors attribute this outcome to label noise. The conclusion is more qualified than the abstract, while the abstract states the stronger claim.
- Anomalous-dimension identification is substantially weaker than temporal detection. Top-1 accuracy on dataset B is 0.376, compared with 0.076 for random selection. Dimension-level F1 on dataset A is 0.534, compared with 0.352 for a random predictor.
Questions for authors
- If the aggregation operator and the correlation threshold are fixed using data disjoint from datasets A–F, do EGI-IDEAS and PCA-IDEAS remain the leading methods?
- On dataset F with λ = 0.8 and d = 20, how many difference channels are included, and how often do they receive high scores when the per-channel marginals are left unchanged?
- For the semi-supervised CNN experiments, which real-world datasets provide an anomaly-free training series, and were EGI and MP evaluated on that series or only on the test series?
6 comments
W3. Real-world results and the abstract.
On the 20 real-world datasets, EGI-IDEAS performs on par with the strongest baselines (mean RTW: 0.59 vs. 0.60 for PCA and 0.59 for OmniAnomaly) and outperforms the other eight baselines, while requiring no training data, unlike OmniAnomaly, and being much more computationally efficient than PCA.To investigate the performance difference, we analyzed the 498 real-world test series by the number of labeled anomalous segments:
- 1–3 segments (248 series), as in our synthetic datasets: EGI-IDEAS ranks first among all 14 detectors (mean RTW: 0.57 vs. 0.50 for PCA and OmniAnomaly).
- 4 or more segments (250 series): EGI-IDEAS ranks fifth, with PCA and OmniAnomaly performing significantly better (paired Wilcoxon, ).
This trend also holds within datasets. Regression with dataset fixed effects shows that EGI-IDEAS's VUS-PR decreases by 0.074 relative to PCA and OmniAnomaly per tenfold increase in anomalous segments (), while the fraction of anomalous time points has no significant association (). This is consistent with EGI's reliance on rare subsequence patterns, which may become less distinguishable when anomalous patterns recur frequently.
Real-world benchmarks also present possible annotation and evaluation challenges (Appendix A.10 and Figure A17): 30.5% of series have more than 10% anomalous time points (up to 98.5%); in 20.3%, a channel's largest deviation occurs outside labeled anomalies and exceeds its largest deviation within them by at least threefold; and in 11.8%, none of the 14 detectors achieves VUS-PR above 0.2.
We revised the abstract, Section 1, and the conclusion to distinguish the consistent improvements on the synthetic data from the competitive real-world results, and added this analysis to Appendix A.10.
W4. Anomalous-dimension localization.
Anomalous-dimension localization is substantially better than random selection. For the revision, we re-evaluated localization with a single procedure for IDEAS and the multivariate baselines below: the dimension anomaly scores of Eq. (7) are computed only for the original dimensions, from the same runs as the reported detection results. The submitted evaluation incorrectly treated the constructed pairwise-difference channels as additional dimensions. EGI-IDEAS achieves a dimension-level F1 of 0.546 versus 0.347 for random selection on dataset A, and a top-1 accuracy (HR@1) of 0.410 versus 0.076 on dataset B, more than five times higher. It also exceeds random selection on datasets C–E, and in dimension-level AUC-PR on all six datasets (Table 2).
We further compared EGI-IDEAS with three natively multivariate localization baselines: ALoRa-Loc (ICLR 2026) and the per-channel reconstruction errors of TranAD and OmniAnomaly. EGI-IDEAS outperforms all three on datasets A–E.
Mean dimension-level F1 over the groups of each dataset:
| Method | A | B | C | D | E | F |
|---|---|---|---|---|---|---|
| EGI-IDEAS | 0.546 | 0.274 | 0.673 | 0.237 | 0.669 | 0.729 |
| ALoRa-Loc | 0.298 | 0.132 | 0.399 | 0.104 | 0.394 | 0.693 |
| TranAD | 0.283 | 0.127 | 0.360 | 0.098 | 0.356 | 0.523 |
| OmniAnomaly | 0.348 | 0.134 | 0.431 | 0.106 | 0.431 | 0.713 |
| Random | 0.347 | 0.116 | 0.451 | 0.090 | 0.461 | 0.550 |
Mean HR@1 over the groups of each dataset:
| Method | A | B | C | D | E | F |
|---|---|---|---|---|---|---|
| EGI-IDEAS | 0.678 | 0.410 | 0.882 | 0.434 | 0.912 | 0.883 |
| ALoRa-Loc | 0.268 | 0.068 | 0.464 | 0.082 | 0.597 | 0.677 |
| TranAD | 0.375 | 0.102 | 0.634 | 0.086 | 0.648 | 0.664 |
| OmniAnomaly | 0.385 | 0.094 | 0.606 | 0.076 | 0.620 | 0.586 |
| Random | 0.225 | 0.076 | 0.469 | 0.050 | 0.491 | 0.661 |
Table 2 in the revised paper reports dimension-level AUC-PR. EGI-IDEAS exceeds the strongest localization baseline by up to 2.2× in dimension-level F1 and 5.0× in HR@1, whereas all three baselines fall below random selection in F1 on C and E and are often close to random. Although localization remains more challenging than temporal detection, these results demonstrate the effectiveness of IDEAS in identifying anomalous dimensions.
We also evaluated localization on series with strong temporal detection (series-level AUC-PR > 0.5):
| Series with AUC-PR > 0.5 | EGI-IDEAS | MP-IDEAS | CNN-IDEAS | PCA-IDEAS |
|---|---|---|---|---|
| B: HR@1 | 0.77 | 0.90 | 0.95 | 0.26 |
| D: HR@1 | 0.83 | 0.86 | 0.99 | 0.41 |
| A: F1 | 0.67 | 0.60 | 0.56 | 0.53 |
| C: F1 | 0.72 | 0.62 | 0.55 | 0.55 |
| E: F1 | 0.72 | 0.55 | 0.54 | 0.58 |
Localization is more accurate on series with strong temporal detection, particularly HR@1 on datasets B and D. We included the baseline comparison in Section 5.3 (Table 2) and Appendix A.13, which reports dimension-level F1, HR@1, and AUC-PR for every group.
Q2. Difference channels on dataset F (, ).
On average, 150.8 of 190 possible channel pairs satisfy (range: 100–190), resulting in approximately 171 channels per series, including the 20 original channels.
We further analyzed channel scores across 195 labeled anomaly intervals, where cross-channel dependencies are disrupted while individual-channel distributions are largely preserved:
| Measurement | EGI-IDEAS | MP-IDEAS | CNN-IDEAS | PCA-IDEAS | Reference |
|---|---|---|---|---|---|
| Intervals where a difference channel has the highest peak score | 96.4% | 99.0% | 99.5% | 99.5% | 88% |
| Flagged points that fall within anomalies (difference channels) | 31.8% | 18.4% | 24.3% | 27.3% | 5.8% |
| Flagged points that fall within anomalies (original channels) | 14.4% | 11.2% | 8.6% | 7.2% | 5.8% |
For the first row, we compare each channel's peak score within an anomaly interval. For the remaining rows, each channel flags its top 7.5% highest-scoring time points. The reference values are the proportion of difference channels (88%) and the anomaly rate (5.8%).
Paired difference channels produce the highest peak scores in 96.4–99.5% of labeled anomaly intervals, and their flagged points are 1.6–3.8× as likely to fall within anomalies as those flagged by original channels. This is consistent with Figure A3, where adding paired difference channels raises VUS-PR on the validation set of dataset F from 0.09–0.47 () to 0.50–1.00 ().
Q3. Training series on real-world datasets.
Every real-world test series is paired with a training series:
- For the 16 datasets from mTSBench, we use the official training files, which contain no labeled anomalies.
- For SWaT A2/A5 and WADI A2, we use normal-operation recordings from the testbeds.
- For TSB-AD-M, we use the benchmark's original split, with the first part of each series used for training and the remainder for testing.
Only semi-supervised detectors use these training series: CNN-IDEAS, CNN, LSTMAD, OmniAnomaly, TranAD, ALLM4TS, and OFA. EGI-IDEAS, MP-IDEAS, and PCA-IDEAS, along with the unsupervised baselines, operate exclusively on the test series without using training data. We clarified this protocol in Appendix A.3. We appreciate the reviewer’s constructive suggestions, which have helped us strengthen both the evaluation and the scope of our claims. Independent validation reproduces the original configuration choices, while the new localization comparisons demonstrate gains over established alternatives. The additional real-world analysis clarifies the conditions under which IDEAS performs well and where challenges remain. We hope these results address the concerns about evaluation rigor and empirical scope and support a more favorable assessment of our work.
We thank the reviewer for recognizing the clear methodology, strong performance on sparse-channel anomalies, the evaluation of cross-channel anomalies, and the comprehensive real-world and runtime comparisons.
W1. Synthetic data generation and channel-wise scoring.
Datasets A–E are designed to evaluate anomalies affecting different numbers of channels. We acknowledge that independently generated channel anomalies may favor channel-wise methods. However, this setting is practically relevant, as Pinet et al. (2026) show that many anomalies in existing multivariate benchmarks exhibit strong univariate evidence. Importantly, IDEAS also performs well beyond sparse-channel anomalies:
Dataset F incorporates cross-channel anomalies while largely preserving individual channel behavior. PCA-IDEAS achieves VUS-PR of 0.90–0.99 across the five groups, compared with 0.70–0.81 for the strongest multivariate baseline in each group (OFA or CBLOF at d=100).
Dataset A (danom=20) evaluates anomalies affecting all channels. EGI-IDEAS achieves VUS-PR of 0.95, compared with 0.74 for CNN and 0.72 for OmniAnomaly.
The 20 real-world datasets provide evaluation on naturally occurring anomalies beyond our synthetic generation process. EGI-IDEAS nearly matches PCA (mean RTW: 0.59 vs. 0.60), equals OmniAnomaly (0.59), and outperforms the other eight baselines by 0.05–0.26. It also achieves the highest VUS-PR on 3 of the 20 datasets, more than any other detector (Figure 4).
We revised the abstract and Section 1 to distinguish the consistent improvements on synthetic benchmarks from the competitive performance against the strongest baselines on real-world data.
W2 / Q1. Hyperparameter selection on held-out data.
We repeated the selection procedure in Appendix A.7 using independently generated validation sets with different random seeds, disjoint from the test series in datasets A–F (20 series per group, 600 total). The held-out validation selects the same default aggregation operators for all four detectors as those originally used in the paper. The table reports mean RTW for the best and second-best operators:
The original operator choices are robust: each default outperforms the runner-up by at least 0.07 in mean RTW and is the best operator in 23 of 24 detector–dataset combinations, with MP on dataset F as the only exception. Thus, independent validation reproduces the reported configurations without changing the test results.
Role of λ. The correlation threshold λ controls which channel pairs receive pairwise-difference channels. Lowering λ includes more pairs, while λ=1 excludes all pairwise-difference channels.
Dataset F. The held-out validation selects λ=0.8 for all four IDEAS detectors, consistent with Figure A.3. The table reports mean VUS-PR:
Dataset F introduces cross-channel anomalies while largely preserving individual-channel behavior. Without difference channels (λ=1), these anomalies are difficult to detect, yielding VUS-PR of only 0.12–0.39. Difference channels expose disrupted relationships, substantially improving performance at λ=0.9. Lowering λ to 0.8 increases the proportion of selected pairs from approximately 55% to 80%, providing further gains, particularly for EGI-IDEAS (0.80 to 0.86) and MP-IDEAS (0.60 to 0.73).
Datasets A–E. These datasets are not designed to introduce cross-channel anomalies. On the held-out validation sets, only 0.8% of channel pairs exceed a correlation of 0.9, compared with 0.9% at a threshold of 0.8. Consequently, lowering λ adds fewer than one difference channel per series on average, and mean VUS-PR differs by at most 0.018 across all detectors and datasets. We therefore retain λ=0.9 as the default for A–E.
The held-out validation reproduces the original configurations, leaving the reported test results unchanged, with EGI-IDEAS and PCA-IDEAS remaining the leading methods. We added the validation results to the Appendix A.7 and clarified the selection protocol in Section 4.2.