- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 1
Read review
Paper summary
This paper introduces GGADBench, a benchmark for generalist graph anomaly detection (GGAD). It evaluates 30 methods: 15 GGAD methods, 10 conventional GAD baselines, and 5 graph foundation models on 22 graphs. The benchmark studies effectiveness, cross-graph transferability, anomaly retrieval via AUPRC, and computational efficiency. It also introduces Anomaly Residual Polarity to characterize whether anomalous nodes have larger or smaller feature residuals relative to their neighborhoods.
Critical strengths
- The breadth of the benchmark is substantial.
- The paper is well organized.
Critical weaknesses
- The benchmark tunes hyperparameters on graphs that are reported as unseen test targets. The main evaluation calls Reddit, Tolokers, Weibo, and Questions "unseen" target graphs. However, Appendix C states that all tuned methods receive 100 trials, with configurations selected according to performance on Reddit, Tolokers, Weibo, and Questions.
- The conventional GAD baselines are not comprehensive. They should include more baselines such as CONSISGAD [1] and SpaceGNN [2].
- ARP has less novelty than the paper suggests. Thus, ARP is simply a rescaled signed AUROC of the handcrafted neighborhood-residual statistic, i.e., ARP=2AUC-1.
Reference
[1] Nan Chen, Zemin Liu, Bryan Hooi, Bingsheng He, Rizal Fathony, Jun Hu, Jia Chen. Consistency Training with Learnable Data Augmentation for Graph Anomaly Detection with Limited Supervision. ICLR 2024.
[2] Xiangyu Dong, Xingyi Zhang, Lei Chen, Mingxuan Yuan, Sibo Wang. SpaceGNN: Multi-Space Graph Neural Network for Node Anomaly Detection with Extremely Limited Labels. ICLR 2025.
Questions for authors
- Can the entire benchmark be rerun with completely disjoint validation domains?
- What additional information does ARP provide beyond reporting the AUROC of the simple neighborhood-residual score?
2 comments
Friendly Reminder
Dear Reviewer TRfo,
This is a friendly reminder that we greatly value your feedback on our rebuttal. Your constructive comments have been very helpful in improving our paper.
As the discussion phase is drawing to a close, we wish to extend a respectful request for your feedback about our rebuttal.
We highly value your insights and are eagerly awaiting your revised evaluation. Additionally, if you have any further questions or need additional clarification, please feel free to contact us.
Thank you again for your time and consideration.
Best regards,
Authors of Submission #38993.
Response
W1: Hyperparameter Tuning
Thank you for raising this concern. The original hyperparameter search was conducted separately from the benchmark, using Reddit, Tolokers, Weibo, and Questions for training and Facebook for validation. To ensure complete independence between hyperparameter tuning and benchmark testing, we have rerun the optimization with the following protocol:
All affected benchmark experiments have been rerun using the newly selected hyperparameters. The main findings and conclusions remain consistent with the original results.
Revised manuscript: Appendix C.2 and all Table.
W2: Additional GAD Baselines
Thank you for this suggestion. We have added representative ConsisGAD and SpaceGNN to our benchmark, increasing the number of conventional baselines from 10 to 12 and the total number of evaluated methods from 30 to 32.
Revised manuscript: Method descriptions 3.1, Appendix A
W3: Anomaly Residual Polarity (ARP)
ARP is mathematically equivalent to 2AUROCresidual−1. Its purpose is to quantify the applicability of neighborhood residuals as an anomaly signal across different graphs, rather than introduce a new detection metric.
ARP provides a unified measure of the strength and direction of residual-based anomaly discrimination. Our analysis shows that 11 of 12 residual-based GGAD methods exhibit positive ARP–AUROC correlations across 19 target graphs, supporting its use as a diagnostic measure of residual-signal applicability. We have revised the manuscript to clarify the role of ARP and avoid overstating its novelty.
Revised manuscript: Section 2, Appendix D.5.
Thank you again for your time and valuable insights, and hope our responses can address all your concerns!