- Rating
- 3
- Confidence
- 1
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper examines whether complex temporal graph anomaly detectors provide enough benefit over simple features. It introduces SimpleCount, which uses training labels to select one of 14 historical features. Across five public datasets and one synthetic benchmark, it compares detection performance and runtime against SLADE, MIDAS-F, and Isolation Forest. The results suggest that simple features can be competitive on some datasets at substantially lower runtime.
Critical strengths
-
The paper addresses a useful evaluation question. Comparing complex models against a strong simple baseline helps assess whether their additional complexity produces meaningful gains. This is practically relevant for streaming anomaly detection.
-
The proposed baseline is simple and interpretable. Each score comes from a single count, recency, or first-occurrence feature. This makes it easy to understand which basic signals are predictive in each dataset.
-
The evaluation considers both performance and computational cost. Runtime measurements make the trade-offs clearer. Five-seed SLADE results, per-feature analysis, and supplementary AUPRC results also provide useful context beyond the main ROC-AUC comparison.
Critical weaknesses
-
A single Isolation Forest result is not enough to assess whether combining features helps. The experiment shows that the tested Isolation Forest configuration does not outperform the selected single feature. Other simple models, such as a shallow decision tree, may capture useful interactions between features. Moreover, SimpleCount uses labels for feature selection, while Isolation Forest does not. A lightweight multi-feature baseline using the same training labels would make this comparison more informative.
-
The experimental coverage could be narrow, and the relation to existing benchmarks needs clarification. The main experiments include only one neural anomaly detector, SLADE, while GraphMixer is evaluated on only two datasets. This limits how broadly the findings on the benefits of model complexity can be generalized. The paper should also discuss and evaluate on BAG [a], which evaluates 25 models on 10 datasets covering dynamic edge- and node-level anomaly detection, and clarify what additional insights this study provides.
[a] Hua, F., Qi, Y., Wei, Z., Tian, Y., Xu, C., Wu, X., Li, J. and Guo, J., 2026, March. Bag: Benchmarking anomaly detection on dynamic graphs. In Proceedings of the AAAI Conference on Artificial Intelligence (Vol. 40, No. 17, pp. 14892-14900).
Questions for authors
N/A
3 comments
I would like to thank the authors for their detailed response. The rebuttal has addressed my concerns.
Thank you for taking the time to review and read the rebuttal!
We thank the reviewer for the suggestions and the pointer to BAG. All changes are highlighted in blue in the revised PDF.
W1: We agree, and have added exactly this comparison (Appendix H, summarised in Section 5.1, highlighted in blue). Logistic regression, histogram gradient boosting, and an MLP are fitted to the same 14 features using training labels, with configurations selected on validation AUROC. These are multi-feature baselines that use the same labels as SimpleCount, and boosting can capture interactions between features. They recover additional signal on some datasets. The unsupervised IsoForest remains in the main text as the label-free control, and the label-usage table (Section 4.3) states how each method uses labels. For example, on Bitcoin-Alpha logistic regression raises AP from 11.06% for the single feature to 19.01%, above DyGFormer’s 15.83% (Table A4).
W2: We have broadened the evaluation (Appendix H, summarised in Section 5.1, highlighted in blue). GraphMixer, DyGFormer, TGN, and CAWN are now evaluated on all five interaction datasets, alongside SLADE, and DyGFormer also on Theia and TRACE. The extended evaluation also adds Theia, TRACE, MIDAS DARPA, Synth-Quad, and two email streams. Across 27 comparisons with neural methods, the single feature matches or exceeds neural AUROC in 20, while neural models lead on the Bitcoin graphs and the email streams.
We have also added BAG to Related Work (highlighted in blue). All five public datasets we use are among BAG’s ten, although BAG evaluates injected rather than organic anomalies on Bitcoin-Alpha and Bitcoin-OTC. Our study adds what BAG does not cover: