- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper identifies the "predictive risk–decision gap" in time series anomaly detection: scoring directly by prediction/reconstruction error may cause the same deviation to be anomalous under one task definition and normal under another. To address this, the paper proposes HieraNTP: it first self-supervisedly pretrains a "tokenizer + next-token predictor," then freezes them, and uses a labeled adaptive detector (XGBoost, comprising a shared base detector plus dataset-specific extensions) to learn evidence combination. On TSB-AD-U/M, VUS-PR improves by 12.0%/15.5% respectively over the strongest baseline, and it achieves the highest overall score on both tracks of TAB.
Critical strengths
- The problem framing is insightful: the discussion of the "predictive risk–decision gap" is clear and well-supported
- The experiments are extremely solid: dual benchmarks and experiment designs. And the results are leading.
Critical weaknesses
- Imbalance between complexity and interpretability: the thousands-of-dimensions handcrafted evidence representation + XGBoost make the engineering implementation complex。
- Strong dependence on labels: the task-adaptive detector requires task labels, and when comparing with unsupervised/zero-shot baselines, the differences in setup need to be clarified.
- Generalization is not sufficiently validated: pretraining uses QUALS and evaluation uses TSB-AD/TAB, with no independent validation on completely unseen/cross-domain datasets; on some datasets (TODS, SMD), it still underperforms TimeRCD.
Questions for authors
- What are the computational/storage/inference overheads of the thousands-of-dimensions evidence representation? Could you provide a runtime and GPU memory comparison, or validate the performance after feature dimensionality reduction (e.g., top-k features, PCA)?
- The task-adaptive detector relies on labels. Is the comparison with unsupervised/zero-shot baselines (TimeRCD, MOMENT, etc.) fair?
- What are the trade-offs between XGBoost and an end-to-end differentiable detector? Have you considered replacing it with a lightweight MLP/graph model?
- On completely cross-domain distribution data, how does the framework perform?
8 comments
To W2Q2: Fairness of Comparisons Between Supervised Task Adaptation and Unsupervised or Zero-Shot Anomaly Detection
Thank you for raising the question about supervision and comparison fairness. The TSB-AD Tuning sets and TAB training data provide labels that support supervised anomaly detection. HieraNTP follows these training and test splits, uses training labels to fit the task-adaptive detector, and evaluates on the official test sets. The manuscript also identifies the zero-shot and task-adapted settings of the methods.
Our use of supervision is motivated by the predictive risk–decision gap. Predictive discrepancies identify violations of temporal regularity, while anomaly labels specify which violations require detection. The Yahoo case illustrates this distinction. Missing peaks and disrupted cycles both produce risk responses, but the task labels only the latter as anomalous.
To compare performance under matched supervision, we organized the available predictive discrepancies, reconstruction discrepancies, and representations from TSPulse, Chronos-Bolt, and MOMENT, then applied the same anomaly labels and XGBoost training pipeline as HieraNTP. The table reports VUS-PR on TSB-AD-U/M.
| Method | Zero-shot detection | Task-adaptive detection |
|---|---|---|
| TSPulse | 0.48 / 0.36 | 0.57 / 0.49 |
| MOMENT | 0.38 / — | 0.52 / 0.49 |
| Chronos-Bolt | 0.29 / — | 0.57 / 0.49 |
| Multiscale statistics | — | 0.46 / 0.41 |
| HieraNTP | — | 0.66 / 0.50 |
With task adaptation, the three pretrained models reach 0.52–0.57 on U and approximately 0.49 on M, competitive with leading detectors in the main comparison. These results show that retaining multiview risk evidence and learning task criteria from training labels benefits different pretrained models. Under matched supervision, HieraNTP achieves the highest VUS-PR among the evaluated representations on both tracks. This matched-supervision comparison has been added to Table 4 of the revision.
Our motivation is that forecasting accuracy and anomaly-detection accuracy differ in their task semantics. Anomaly annotations express the events that matter to the application. HieraNTP learns which risks and combinations should trigger anomaly decisions, connecting structured predictive evidence to the target task’s criteria.
The revision clarifies the supervision protocol in Section 4.1 and reports the matched-supervision comparison in Section 4.3 (Table 4), with feature construction and complete results documented in Appendix E.1 (Tables 23–24).
To Q3: Trade-Offs Between Gradient-Boosted Decision Trees and End-to-End Differentiable Neural Network Detectors
Thank you for this suggestion. We replaced XGBoost with a lightweight multilayer perceptron (MLP) and a graph attention network (GAT), using the same frozen predictor, structured risk representation, and anomaly-label supervision. Both alternatives retain shared training and dataset-specific adaptation. The GAT uses risk-evidence channels as nodes to learn interactions among evidence.
| Detector | TSB-AD-U VUS-PR (%) | TSB-AD-M VUS-PR (%) |
|---|---|---|
| XGBoost | 65.56 | 50.26 |
| MLP | 53.99 | 48.72 |
| GAT | 54.69 | 43.70 |
XGBoost achieves the best performance on both tracks. Its tree splits select evidence according to input conditions, allowing different samples to use different evidence combinations. This matches HieraNTP’s objective of learning task-specific criteria from multiview predictive risk.
Feature masking is applied during detector training. We subsequently evaluate evidence dependence on a held-out univariate probe through changes in the anomaly logit under feature-wise masking. On the U held-out probe, the median share of normalized logit sensitivity carried by the top 256 features is 96.36% for XGBoost, compared with 34.51% for MLP and 39.32% for GAT. Thus, the trained XGBoost detector concentrates its decisions on a small subset of evidence, while the trained neural alternatives remain sensitive to more channels. Together with detection performance, these results support XGBoost’s effectiveness in selecting and combining task-relevant evidence under the current supervision.
The trade-off is that XGBoost learns conditional evidence selection through a separately trained detector, whereas MLP and GAT provide differentiable architectures for joint optimization with upstream representations. The replacement experiments support our use of XGBoost, while jointly training the predictor and a differentiable detector remains a promising direction. We have reported the detector comparison and feature-masking sensitivity analysis in Appendix E.7 (Table 34), with the connection to the task-adaptive detector described in Section 3.4.
To W3Q4: Cross-Dataset Generalization and Performance on Unseen Source Distributions
Thank you for raising the concerns about cross-domain generalization and dataset-level differences. Cross-domain anomaly detection involves transferring temporal regularities and accommodating changes in anomaly criteria. The same deviation can be normal or anomalous under different task definitions, as emphasized by the predictive risk–decision gap. HieraNTP learns reusable temporal knowledge through pretraining and uses downstream supervision to learn what the resulting risks mean for the target task.
Pretraining uses QUALS, derived from GIFT-Eval Pretrain, with no data leakage from TSB-AD or TAB. It trains the tokenizer and predictor, while downstream training labels supervise anomaly decisions.
To evaluate cross-dataset performance, we added leave-one-source-dataset-out evaluation. Each target source is excluded entirely from detector training and hyperparameter selection. The shared base detector is trained using the remaining sources and evaluated on the held-out dataset. The table reports mean VUS-PR over the complete TSB-AD Eval sets.
| Method | Training setting | TSB-AD-U (%) | TSB-AD-M (%) |
|---|---|---|---|
| HieraNTP | Standard task adaptation | 65.56 | 50.28 |
| HieraNTP | Shared base detector only | 61.48 | 49.91 |
| HieraNTP | Leave-one-source-dataset-out | 53.70 | 36.89 |
| TimeRCD | Zero-shot | 52.24 | 31.68 |
| TSPulse | Zero-shot | 48.01 | 36.10 |
With the target source excluded, HieraNTP achieves 53.70%/36.89%, exceeding the zero-shot TimeRCD and TSPulse results. This shows that evidence–label relationships learned from other datasets support detection on held-out sources. The further gains from target-source training data demonstrate the value of task supervision in interpreting predictive risk.
Under standard evaluation, HieraNTP improves VUS-PR by 12.0% on TSB-AD-U and 15.5% on TSB-AD-M relative to the strongest baselines. TimeRCD remains stronger on TODS and SMD, showing that gains vary across datasets and suggesting differences in how learned anomaly criteria transfer from training data to test sequences. We have added the held-out source results to Section 4.3 (Table 3) and detailed the LOSO protocol and per-source results in Appendix E.3 (Tables 25–27), while retaining the dataset-level comparisons in Section 4.2 (Table 1).
To W1Q1: Representation Dimensionality and the Computational, Storage, and Inference Costs of Evidence Processing
Thank you for raising the concerns about representation complexity and computational overhead. We added top-k feature selection, PCA experiments, and runtime measurements.
HieraNTP expands 64 base discrepancy channels through three processing paths and 13 temporal summaries, yielding 2496 univariate features. Retaining variable means and maxima gives 4992 multivariate features. Each feature has an identifiable evidence source and processing configuration.
Top-k performs post-hoc selection using split gain from the Tuning-trained detector. PCA is fitted on training data, followed by detector retraining on the projected features. The table reports all completed settings using mean VUS-PR over the complete Eval sets.
VUS-PR is reported in percent. With 1024 retained features, top-k achieves 65.70% on U and 49.79% on M, showing that the supervised detector concentrates its decisions on a small subset of task-relevant evidence. Retraining with 1024 PCA components achieves only 55.62% and 46.55%, indicating that a common PCA projection loses task-relevant risk information distributed across channels. Different tasks require different evidence, motivating HieraNTP to preserve these channels for supervised selection and combination.
For computation and inference, complete U350 and M180 evaluations take approximately 2.8 and 20.4 minutes using seven RTX 4090 GPUs and 48 workers. These evaluations cover 530 test sequences and approximately 264 million scalar observations. Single-sequence inference uses approximately 890 MiB of GPU memory. Under the concurrent configuration, peak aggregate memory usage per GPU is approximately 20.74 GiB.
For storage, the XGBoost model files occupy approximately 11.13 MiB for U and 8.50 MiB for M. Intermediate evidence is passed through memory, and the final FP32 anomaly-score arrays occupy approximately 69.28 MiB for U350 and 74.73 MiB for M180. We describe the representation dimensions and reference feature selection in Section 3.3, report the top-k and PCA comparisons in Appendix E.6 (Table 33), and document runtime, GPU memory, and storage costs in Appendix C.3.