- Rating
- 3
- Confidence
- 4
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The main novelty lies in exploring fully synthetic training for generalist graph anomaly detection and demonstrating its feasibility on real-world graphs. Specifically, AG-FORGE provides a configurable pipeline for generating diverse graph structures, node features, and anomaly patterns with controllable difficulty, while TS-GGAD combines topology-aware and semantic representations with curriculum learning to exploit large-scale synthetic data. The contribution is therefore not solely a new anomaly detector, but a unified framework for synthetic data generation and generalist GAD training. The experiments further investigate whether synthetic training can achieve competitive performance without relying on real-world training graphs.
Critical strengths
-
The paper investigates whether synthetic data can address the data scarcity that limits the scalability of generalist graph anomaly detection.
-
AG-FORGE generates diverse anomalous graphs, while TS-GGAD combines topological and semantic evidence with curriculum learning. Ablation studies provide evidence for the effectiveness of these components.
-
Experiments on 14 real-world datasets demonstrate strong AUROC performance, while comparisons with real-world-trained generalist methods provide preliminary evidence for the feasibility of synthetic-data-driven training.
Critical weaknesses
-
AG-FORGE relies on predefined graph structures and anomaly injection rules, but the paper does not systematically evaluate how synthetic data diversity and generation scale affect generalization or distinguish performance on naturally occurring versus injected anomalies.
-
As shown in Table 1, synthetic training causes notable performance drops for some methods on certain datasets, such as ARC on Amazon and ReFi-GAD on Flickr. The paper does not analyze the underlying causes or clarify when synthetic training benefits or harms generalization.
Questions for authors
See Weaknesses.
2 comments
Thank you for the detailed response and additional experiments. The authors have adequately addressed all my concerns. I am satisfied with the clarifications and will maintain my original score.
Author Response to Reviewer LihF (Revisions Included in the Updated Manuscript)
To Q1: We have evaluated the impact of synthetic data diversity through detailed ablations in Appendix Table 6. Removing any structure, feature, or anomaly generation strategy reduces average AUROC by 0.95–3.54 percentage points. Specifically, removing individual strategies from these three generation stages leads to average drops of 2.33, 1.80, and 1.07 percentage points, respectively, demonstrating the importance of diverse graph environments and anomaly patterns. To further systematically investigate the impact of synthetic data diversity, we conducted additional experiments. we investigate how the diversity of generation strategies affects cross-domain generalization by jointly changing the number of topology, feature, and anomaly generation strategies from 1 to 3. To ensure a fair comparison, we maintain the same number of synthetic graphs and report the average results over five random seeds.
The average AUROC consistently improves from 79.04% to 84.56% as the number of generation strategies increases, demonstrating that greater synthetic data diversity leads to better cross-domain generalization. We further investigate the contributions of graph diversity and anomaly pattern diversity. For anomaly diversity, the ablations in Appendix Table 6 show that removing each of the four basic anomaly generation strategies reduces average AUROC by 0.95–1.28 percentage points, with an average decrease of 1.07 percentage points. This indicates that diverse anomaly patterns provide complementary information for generalist GAD training. For graph diversity, we conduct additional experiments by varying the numbers of topology and feature generation strategies while keeping all five anomaly strategies available. The results are summarized below.
The results show that increasing graph diversity improves average AUROC from 83.62% to 84.82% (+1.20 percentage points) when the anomaly strategy pool is fixed, further supporting the importance of diverse structural and feature environments. Together, these experiments demonstrate that both graph diversity and anomaly pattern diversity contribute to effective cross-domain generalization. Regarding training scale, Fig. 5 systematically evaluates synthetic training data scales from 5K to 1M nodes under different model capacities. The results show that increasing training data generally improves cross-domain generalization when sufficient model capacity is available, while smaller models may struggle to fully exploit large-scale synthetic data. This highlights the importance of jointly scaling training data and model capacity, further supporting the motivation behind AG-FORGE and TS-GGAD.
Regarding anomaly origins, our 14 benchmarks include 8 datasets with injected anomalies (e.g., Cora, ACM, BlogCatalog) and 6 with naturally occurring anomalies (e.g., Weibo, Reddit, Amazon). TS-GGAD achieves the best AUROC on all datasets, with average results of 91.40% and 80.24% for the two groups, respectively. Importantly, TS-GGAD is pretrained entirely on synthetic graphs, yet achieves strong performance on datasets containing naturally occurring anomalies. This provides further evidence that AG-FORGE supports learning transferable anomaly knowledge rather than merely fitting predefined synthetic anomaly patterns.
To W2: We believe these performance variations are mainly related to the limited capacity of existing generalist GAD models, which were originally designed for relatively limited real-world training data**. When faced with large-scale and diverse synthetic data, their limited capacity may prevent them from fully exploiting the rich anomaly information**, leading to performance differences across target domains.
Nevertheless, synthetic training also brings substantial improvements, such as +10.98% for ARC on Facebook and +4.75% for ReFi-GAD on YelpChi. Overall, existing methods still achieve competitive generalization, with ReFi-GAD even improving its average rank from 4.50 to 3.86**. These results suggest that the benefits of synthetic training depend on model capacity**. While existing models may struggle to fully exploit diverse synthetic data, more expressive models can better leverage such data for generalization. This further motivates TS-GGAD, which achieves the best AUROC on all 14 datasets through effective topology-semantic anomaly modeling.