acceptodds
Under review as a conference paper at ICLR 2027

GGADBench: A Comprehensive Benchmark for Generalist Graph Anomaly Detection

Abstract

Generalist Graph Anomaly Detection (GGAD) aims to detect anomalies across different graphs with a unified model and has developed rapidly in recent years. However, existing methods are evaluated on different datasets and under different transfer settings and protocols, making their performance difficult to compare reliably. We present GGADBench, the first comprehensive benchmark for GGAD, covering 30 methods and 22 graphs with both organic and injected anomalies under a unified evaluation protocol. GGADBench evaluates methods in terms of overall effectiveness, cross-graph transferability, anomaly retrieval, and efficiency. We further introduce anomaly residual polarity, a dataset-level measure that characterizes how anomalies differ from their local neighborhoods. Our evaluation reveals three key findings: (i) prompt-tuned graph foundation models are competitive with specialized GGAD methods on organic targets, whereas strong AUROC does not necessarily translate into effective anomaly retrieval; (ii) cross-graph performance is driven more by the target graph than by the source anomaly type or source quantity; and (iii) residual-based rankings can reverse on targets with negative ARP. These findings highlight the importance of transferable feature construction and target-adaptive anomaly modeling for future GGAD.

Primary area:
datasets and benchmarks
Keywords:
graph anomaly detection, comprehensive benchmark

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_TRfoOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

This paper introduces GGADBench, a benchmark for generalist graph anomaly detection (GGAD). It evaluates 30 methods: 15 GGAD methods, 10 conventional GAD baselines, and 5 graph foundation models on 22 graphs. The benchmark studies effectiveness, cross-graph transferability, anomaly retrieval via AUPRC, and computational efficiency. It also introduces Anomaly Residual Polarity to characterize whether anomalous nodes have larger or smaller feature residuals relative to their neighborhoods.

Critical strengths
  1. The breadth of the benchmark is substantial.
  2. The paper is well organized.
Critical weaknesses
  1. The benchmark tunes hyperparameters on graphs that are reported as unseen test targets. The main evaluation calls Reddit, Tolokers, Weibo, and Questions "unseen" target graphs. However, Appendix C states that all tuned methods receive 100 trials, with configurations selected according to performance on Reddit, Tolokers, Weibo, and Questions.
  2. The conventional GAD baselines are not comprehensive. They should include more baselines such as CONSISGAD [1] and SpaceGNN [2].
  3. ARP has less novelty than the paper suggests. Thus, ARP is simply a rescaled signed AUROC of the handcrafted neighborhood-residual statistic, i.e., ARP=2AUC-1.

Reference

[1] Nan Chen, Zemin Liu, Bryan Hooi, Bingsheng He, Rizal Fathony, Jun Hu, Jia Chen. Consistency Training with Learnable Data Augmentation for Graph Anomaly Detection with Limited Supervision. ICLR 2024.

[2] Xiangyu Dong, Xingyi Zhang, Lei Chen, Mingxuan Yuan, Sibo Wang. SpaceGNN: Multi-Space Graph Neural Network for Node Anomaly Detection with Extremely Limited Labels. ICLR 2025.

Questions for authors
  1. Can the entire benchmark be rerun with completely disjoint validation domains?
  2. What additional information does ARP provide beyond reporting the AUROC of the simple neighborhood-residual score?
2 comments

W1: Hyperparameter Tuning

Thank you for raising this concern. The original hyperparameter search was conducted separately from the benchmark, using Reddit, Tolokers, Weibo, and Questions for training and Facebook for validation. To ensure complete independence between hyperparameter tuning and benchmark testing, we have rerun the optimization with the following protocol:

SettingTraining GraphsTest Graphs
Hyperparameter TuningFacebookYelpNYC, Books
GGADBenchFacebook (main setting)Benchmark targets (excluding YelpNYC and Books)

All affected benchmark experiments have been rerun using the newly selected hyperparameters. The main findings and conclusions remain consistent with the original results.

Revised manuscript: Appendix C.2 and all Table.


W2: Additional GAD Baselines

Thank you for this suggestion. We have added representative ConsisGAD and SpaceGNN to our benchmark, increasing the number of conventional baselines from 10 to 12 and the total number of evaluated methods from 30 to 32.

Revised manuscript: Method descriptions 3.1, Appendix A


W3: Anomaly Residual Polarity (ARP)

ARP is mathematically equivalent to 2AUROCresidual−12\mathrm{AUROC}_{\mathrm{residual}}-1. Its purpose is to quantify the applicability of neighborhood residuals as an anomaly signal across different graphs, rather than introduce a new detection metric.

ARP provides a unified measure of the strength and direction of residual-based anomaly discrimination. Our analysis shows that 11 of 12 residual-based GGAD methods exhibit positive ARP–AUROC correlations across 19 target graphs, supporting its use as a diagnostic measure of residual-signal applicability. We have revised the manuscript to clarify the role of ARP and avoid overstating its novelty.

Revised manuscript: Section 2, Appendix D.5.


Thank you again for your time and valuable insights, and hope our responses can address all your concerns!

AuthorsOpenReview ↗

Friendly Reminder

Dear Reviewer TRfo,

This is a friendly reminder that we greatly value your feedback on our rebuttal. Your constructive comments have been very helpful in improving our paper.

As the discussion phase is drawing to a close, we wish to extend a respectful request for your feedback about our rebuttal.

We highly value your insights and are eagerly awaiting your revised evaluation. Additionally, if you have any further questions or need additional clarification, please feel free to contact us.

Thank you again for your time and consideration.

Best regards,

Authors of Submission #38993.

Reviewer_YEAgOpenReview ↗
Rating
3
Confidence
4
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

This paper proposes the first benchmark for generalist graph anomaly detection (GGAD), covering 22 graph datasets and designs a comprehensive evaluation protocol. The authors conduct an extensive assessment of 30 existing algorithms using this benchmark, covering metrics such as AUROC performance, anomaly retrieval, and computational efficiency on large-scale graphs. Furthermore, based on their experimental findings, the authors offer a series of insightful observations that provide valuable takeaways for future research in the field of GGAD.

Critical strengths
  1. This paper presents the first benchmark for GGAD. The authors have compiled 22 graph datasets of varying scales and designed a comprehensive evaluation framework.
  2. The authors evaluated 30 existing methods on the proposed benchmark and made a series of valuable findings based on the experimental results, such as cross-graph feature semantics may deserve more attention than individual dimensional compatibility.
  3. The paper is well-structured and logically sound, allowing readers to easily grasp the research content and key findings.
  4. A well-planned arrangement of figures and tables further enhances the readability of the paper, allowing readers to efficiently grasp the main points.
Critical weaknesses
  1. In RQ2, the authors point out that increasing the number of source domains does not significantly improve the performance of the GGAD method. However, the experiments only compare single-source training with multi-source training using 3–4 source domains; there is a lack of systematic evaluation regarding larger numbers of training domains, and larger training data scales.
  2. Appendix C.2 indicates that the authors used four datasets—Reddit, Tolokers, Weibo, and Questions—for hyperparameter search; however, these same datasets also appear in the final performance evaluation. This implies that the hyperparameter selection for some methods may have leveraged label information from the target graphs used for testing, thereby compromising the independence and fairness of the cross-graph generalization evaluation.
Questions for authors

Please see the Weaknesses.

3 comments

W1: Impact of Source Quantity and Scale

Thank you for this suggestion. We have extended RQ2 with additional experiments using 1, 3, 5, and 7 source graphs.

ConfigurationSource DatasetsSource NodesMean AUROC (%)
1 sourceFacebook1.1K51.66
3A sourcesFacebook, T-Finance, Amazon50.7K53.57
3B sourcesAmazon, Questions, Reddit70.1K53.60
5 sources3A + YelpChi, Tolokers86.3K52.72
7A sources5 + Weibo, Questions143.6K52.89
7B sources7A, replacing Tolokers with DGraph-Fin3.83M52.88

7B includes DGraph-Fin (3,700,550 nodes and 4,300,999 edges) as a training source. Its total source size is 26.7 times that of 7A. The results show that increasing source quantity and scale does not consistently improve GGAD generalization.

Revised manuscript: Section 4.2.4.


W2: Hyperparameter Tuning

Thank you for raising this concern. The original hyperparameter search was conducted separately from the benchmark, using Reddit, Tolokers, Weibo, and Questions for training and Facebook for validation. To ensure complete independence between hyperparameter tuning and benchmark testing, we have rerun the optimization with the following protocol:

SettingTraining GraphsTest Graphs
Hyperparameter TuningFacebookYelpNYC, Books
GGADBenchFacebook (main setting)Benchmark targets (excluding YelpNYC and Books)

All affected benchmark experiments have been rerun using the newly selected hyperparameters. The main findings and conclusions remain consistent with the original results.

Revised manuscript: Appendix C.2 and all Table.


Thank you again for your time and valuable insights, and hope our responses can address all your concerns!

Thank you for the detailed response and additional experiments. The authors have adequately addressed all my concerns. I am satisfied with the clarifications and will maintain my original score.

AuthorsOpenReview ↗

Official Comment by Authors

We would like to thank Reviewer YEAg again for taking the time to carefully consider our response. We are glad that our response have addressed your concerns and sincerely appreciate your constructive feedback.

Reviewer_jBzYOpenReview ↗
Rating
3
Confidence
4
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

This work presents GGADBench, claimed to be the first comprehensive benchmark for Generalist Graph Anomaly Detection (GGAD), which seeks a one-for-all detector that learns general anomaly knowledge from source graphs and can be applied to unseen target graphs. The benchmark code and data-processing scripts are released.

Critical strengths
  1. Given that GGAD methods have proliferated since 2024 under heterogeneous datasets, source-target splits, and evaluation protocols, a unified benchmark is genuinely needed. The benchmark fills a clear and timely gap.
  2. The four research questions systematically cover effectiveness, cross-graph transferability, anomaly retrieval, and efficiency, explicitly measuring aspects that are often overlooked, such as AUPRC for the rare anomaly class, the effect of source anomaly type and source quantity, and both GPU and CPU memory on the target side.
Critical weaknesses
  1. The organic sources consist of only three graphs and the injected sources of four graphs, many of the 22 graphs are classical small-scale benchmarks, and the genuinely large industrial graphs appear only as targets and frequently trigger out-of-memory failures. The representativeness of the chosen source graphs and domains therefore deserves more discussion.
  2. Zero-shot and target-adaptive methods are ranked together without sufficiently stratified reporting. Methods that may perform target adaptation even without labels could enjoy an advantage over strictly training-free zero-shot methods, and this distinction should be made explicit when interpreting the shared tables.
  3. Although results are reported as mean plus standard deviation over 11 seeds, there are no paired significance tests or confidence intervals for the average ranks, so it is hard to judge whether small gaps between closely ranked methods are meaningful.
Questions for authors
  1. Can you provide a quantitative analysis of the correlation between ARP and AUROC/AUPRC across all graphs and methods, and consider a structural counterpart to ARP based on ego-network or degree-based deviations?
  2. Could you report the results in a stratified manner for strictly zero-shot methods versus methods allowed target adaptation, and quantify the gain attributable to target-side adaptation without labels?
  3. Are the differences between closely ranked methods statistically significant under paired tests over the 11 seeds, and can you provide confidence intervals for the average ranks?
  4. Can you supplement the efficiency analysis with source-side training cost and the total cost of source training plus inference over many targets, and report alternative scalable configurations for methods that currently run out of memory?
2 comments

W1. Source Graph Representativeness

Thank you for this suggestion. We have expanded the source configurations from 3 to 5 and 7 graphs, including an alternative three-source configuration.

ConfigurationSource DatasetsNodesMean AUROC (%)
1 sourceFacebook1.1K51.66
3A sourcesFacebook, T-Finance, Amazon50.7K53.57
3B sourcesAmazon, Questions, Reddit70.1K53.60
5 sources3A + YelpChi, Tolokers86.3K52.72
7A sources5 + Weibo, Questions143.6K52.89
7B sources7A − Tolokers + DGraph-Fin3.83M52.88

7B includes DGraph-Fin (3,700,550 nodes and 4,300,999 edges) as a training source. Its total source size is 26.7 times that of 7A. The results show that increasing source quantity and scale does not consistently improve generalization.

Revised manuscript: Section 4.2.4.


W2. Stratified Reporting of Evaluation Protocols

Thank you for this suggestion. We have revised all performance tables. Each method is also annotated with its target-side protocol: Zero-shot, Target-adaptive, In-context, and Few-shot.

Revised manuscript: All Tables.


W3. Statistical Significance of Method Rankings

Thank you for this suggestion. We have added Figure 2 to compare 12 pairs of adjacent GGAD methods across 11 target graphs. Each target contributes its mean AUROC over 11 random seeds.

We report Hodges–Lehmann estimates, 95% confidence intervals, and exact two-sided Wilcoxon signed-rank tests with Holm correction. All 12 pairs exhibit ranking reversals across targets, and none shows a statistically significant difference.

Revised manuscript: Figure 2 and Section 4.1.


W4. Correlation Between ARP and AUROC/AUPRC

Thank you for this suggestion. We have added Spearman correlation analyses across 15 GGAD methods and 19 target graphs, covering both attribute and structural residual polarity.

MethodAttribute ARP–AUROCAttribute ARP–AUPRCDegree ARP–AUROCDegree ARP–AUPRC
Residual-based methods
ARC0.550.51-0.42-0.46
UNPrompt0.590.35-0.63-0.39
AnomalyGFM-0.12-0.07-0.16-0.10
IA-GGAD0.610.63-0.40-0.43
DR-GGAD0.630.64-0.20-0.27
OWLEYE0.850.74-0.25-0.32
TA-GGAD0.610.61-0.44-0.46
SAARCS0.610.58-0.44-0.36
GAD-MoRE0.800.64-0.22-0.27
ProMoS0.760.73-0.38-0.38
TPCA-GAD0.25-0.03-0.10-0.06
REFIGAD0.320.11-0.10-0.03
Other methods
ZeroGAD-0.32-0.58-0.07-0.31
NeighborDiv0.150.00-0.10-0.06
TFM4GAD-0.26-0.25-0.010.01

Among 12 residual-based methods, 11 show positive attribute ARP–AUROC correlations and 10 show positive attribute ARP–AUPRC correlations. For degree-based ARP, all 12 methods show negative correlations with both metrics.

Revised manuscript: Appendix D.5.


W5. Efficiency and Scalability Analysis

We have extended RQ4 to report total training-plus-inference time for one random seed under the 3-source and 7B configurations.

Method3 Sources: Training3 Sources: Total7B: Training7B: Total
ARC0.89.41.73.9
OWLEYE0.53.52.14.4
AnomalyGFM5.16.010.811.5
IA-GGAD1.010.720.723.7
ProMoS7.317.126.428.6
REFIGAD16.929.419.830.6
UNPrompt46.047.895.496.9
GAD-MoRE37.353.32401.92413.8
Time in minutes.

The results show substantial differences in computational costs.

For OOM cases, we use chunked processing where applicable to reduce peak memory usage while preserving the original computations as closely as possible. Remaining infeasible configurations are explicitly reported.

Revised manuscript: Section 4.4 and Figure 5


Thank you again for your time and valuable insights, and hope our responses can address all your concerns!

AuthorsOpenReview ↗

Friendly Reminder

Dear Reviewer jBzY,

This is a friendly reminder that we greatly value your input on our rebuttal. Your comments have been extremely helpful in improving our paper.

As the discussion phase is drawing to a close, we wish to extend a respectful request for your feedback about our rebuttal.

We highly value your insights and are eagerly awaiting your revised evaluation. Additionally, if you have any further questions or need additional clarification, please feel free to contact us.

Thanks for your time again.

Best regards,

Authors of Submission #38993.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.