- Rating
- 3(+1)
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper addresses few‑shot cross‑domain adaptation for graph foundation models (GFMs). The authors propose COLA, a retrieval‑augmented framework that builds two complementary databases: graph semantic database and graph structural database. Besides, the paper provides theoretical bound analysis for routing generalization, and conducts extensive node‑classification & ego‑graph classification experiments under limited settings. Ablations, hyperparameter sensitivity and efficiency analysis are also provided. Overall this is a technically complete work for retrieval‑augmented graph foundation model adaptation, though several practical and conceptual questions remain to be clarified.
Critical strengths
- The paper clearly identifies two key limitations for existing GFM adaptation: semantic‑structural heterogeneity, and conflating retrieval similarity with downstream utility. Distinguishing retrieval relevance and prediction utility is a meaningful conceptual contribution for graph RAG (Sec 1).
2.Theorem 3.1 provides generalization bound for adaptive routing under source‑target distribution shift, formally analyzing oracle gain, source routing loss and total‑variation domain shift; complementarity condition for semantic and structural branches is derived in appendix A.
Critical weaknesses
- The benchmark used in the article is a widely accepted historical standard for graph communities. However, though the n-shot setting increases the difficulty, in practical applications, such performance (such as accuracy around 50%) is almost means unacceptable, and not mature for usage in a large number of scenarios which require a robust systems.
- Although I agree with the intelligence of GFM in terms of generality, more extensive application scenarios should be used to verify its "generality", such as graph-related QA, even for graph-related agentic workflows. However, what has been confirmed in this article is limited to node classification and ego-graph classification on several limited datasets, and even key traditional tasks like link prediction are missing. I hope to see graph-based models that truly match the meaning of "foundation", to be honest, can complete a large number of different types of tasks, rather than being limited to the mature tasks in the field of graph research that have been studied for decades. This is what I enjoy about reading GFM's article. This is not required for specific score-target methods, they should focus on specific task and increase points. But for GFMs, the story should be wider to be okay.
- The main baseline RAG‑GFM also builds semantic‑structural memory but without utility re‑ranking. However, the paper does not add a straightforward ablation variant: applying simple supervised ranking / cross‑attention re‑ranker replacing the proposed RL group‑relative policy, to quantify actual gain brought by reinforcement‑learning policy versus simpler supervised re‑ranking alternatives (Sec4.3 Fig2(b)).
- The core conception GFM, is not commonly defined. And the GFMs used are limited. Which makes the scope of this paper are not spread. There are many useful and accptable GFMs including GraphLLMs like GraphGPt, LLaGA, Graph Pattern Machines, etc.
- COLA contains structural‑kernel training, dual database construction, RL‑based group‑relative utility policy, null‑aware router. Multiple hyper‑parameters are introduced (walk depth L, candidate pool size M, top‑K selection, RL clipping coefficient, KL‑entropy regularization weights). While hyper‑parameter sensitivity is tested for K and L, sensitivity for RL regularization hyper‑parameters () is not reported. It is unclear whether good performance heavily relies on careful hyper‑parameter tuning (Sec3.4, Fig3).
- Experiments are conducted on moderate‑size public citation/web/e‑commerce benchmarks. The paper does not analyze memory cost for building large‑scale semantic/structural database when source domains contain millions of graph/node instances. For real‑world large‑scale source corpus, storing embedding for every source instance will bring non‑trivial memory overhead, and approximate nearest‑neighbor retrieval may degrade utility‑calibration performance (Sec4).
Questions for authors
Please address my questions in weakness part.
5 comments
Part II
W3. The main baseline RAG-GFM also builds semantic-structural memory but without utility re-ranking. However, the paper does not add a straightforward ablation variant: applying simple supervised ranking / cross-attention re-ranker replacing the proposed RL group-relative policy, to quantify actual gain brought by reinforcement-learning policy versus simpler supervised re-ranking alternatives (Sec4.3 Fig2(b)).
R3. Thank you for your valuable feedback. To directly quantify the benefits of group-relative policy optimization over simpler supervised re-ranking methods, we conducted an additional controlled comparison with Pairwise MLP and Cross-attention re-rankers on OGBN-Products.
| Setting | Pairwise MLP | Cross-attention | GRPO (Ours) |
|---|---|---|---|
| Node 1-shot | 54.9 | 55.9 | 57.4 |
| Node 5-shot | 62.7 | 64.0 | 66.5 |
| Graph 1-shot | 62.1 | 59.5 | 62.5 |
| Graph 5-shot | 69.6 | 69.4 | 70.7 |
The results show that GRPO achieves the highest accuracy in all four settings, outperforming Pairwise MLP by 0.4–3.8 percentage points and Cross-attention by 1.3–3.0 points. Since Pairwise MLP and GRPO share the same scorer architecture and training budget, their comparison provides direct evidence of the additional benefit of group-relative policy optimization over supervised re-ranking. Together with the results in Fig. 2(b), these findings support the effectiveness of optimizing context selection through relative downstream rewards rather than relying solely on supervised utility ranking.
We have updated the utility-learning analysis in Section 4.3 and reported the controlled Pairwise MLP and Cross-attention comparisons in Appendix D.4.
W4. The core conception GFM, is not commonly defined. And the GFMs used are limited. Which makes the scope of this paper are not spread. There are many useful and accptable GFMs including GraphLLMs like GraphGPt, LLaGA, Graph Pattern Machines, etc.
R4. Thank you for your valuable feedback. We clarify the definition of GFMs adopted in this work and provide an additional backbone comparison to examine COLA's applicability across different graph modeling architectures.
-
Definition and scope of GFMs. We consider GFMs as pretrained graph encoders that learn transferable representations for adaptation to unseen domains. As formulated in Eq. (1), COLA operates on representations extracted from a frozen backbone, with its retrieval and routing mechanisms designed for cross-domain adaptation.
-
Generalization across different backbones. To directly evaluate architectural compatibility, we additionally applied COLA to the Graph Pattern Machine (GPM)[2], which differs substantially from the original backbone by explicitly modeling graph substructure patterns. The results are summarized below.
Backbone Task Frozen (%) + COLA (%) Gain (pp) Original Node 57.4 ± 4.9 66.2 ± 4.3 +8.8 Original Graph 62.1 ± 3.9 68.9 ± 2.2 +6.8 GPM Node 78.3 ± 4.0 81.5 ± 3.4 +3.2 GPM Graph 81.7 ± 3.2 83.7 ± 3.1 +2.0 COLA consistently improves performance with both backbones. In particular, it provides additional gains of 3.2 and 2.0 percentage points on GPM for node and ego-graph classification, respectively. These results provide direct evidence that COLA's retrieval-based adaptation is not restricted to the original backbone architecture.
-
Coverage of broader GFM architectures. Table 1 further compares COLA against representative graph foundation models, including GraphCLIP, UniGraph, and RAG-GFM. Graph-language models such as GraphGPT and LLaGA involve additional graph-language interfaces that are not evaluated in the current work. We therefore distinguish the demonstrated compatibility across representation-based backbones from broader applicability to graph-language generation architectures.
We have reported the GPM backbone comparison in Section 4.3 and Appendix D.5.
Part III
W5. COLA contains structural-kernel training, dual database construction, RL-based group-relative utility policy, null-aware router. Multiple hyper-parameters are introduced (walk depth L, candidate pool size M, top-K selection, RL clipping coefficient, KL-entropy regularization weights). While hyper-parameter sensitivity is tested for K and L, sensitivity for RL regularization hyper-parameters () is not reported. It is unclear whether good performance heavily relies on careful hyper-parameter tuning (Sec3.4, Fig3).
R5. Thank you for your valuable feedback. We assessed sensitivity to the GRPO regularization parameters , , and (Eq. 17) on OGBN-Products 1-shot node classification. We varied one parameter at a time across 13 configurations, using identical source-side training budgets and 10 fixed evaluation splits. The default is .
| Parameter | Value | Accuracy (%) |
|---|---|---|
| Default | (0.01, 0.001, 0.2) | 57.4 ± 6.4 |
| 0 | 56.3 ± 7.4 | |
| 0.005 | 56.3 ± 7.7 | |
| 0.02 | 56.1 ± 7.6 | |
| 0.04 | 55.8 ± 8.6 | |
| 0 | 55.3 ± 8.7 | |
| 0.0005 | 56.2 ± 7.4 | |
| 0.002 | 55.7 ± 7.7 | |
| 0.004 | 55.6 ± 5.6 | |
| 0.05 | 54.0 ± 7.1 | |
| 0.1 | 56.7 ± 7.4 | |
| 0.3 | 56.4 ± 7.6 | |
| 0.4 | 54.8 ± 11.2 |
Relative to the default, accuracy decreases by at most 1.6 pp when varying and 2.1 pp when varying . Clipping values of 0.1–0.3 yield 56.4–57.4% accuracy, while 0.05 and 0.4 perform worse. Thus, performance is relatively stable across the tested regularization settings but more sensitive to extreme clipping values.
We have extended the hyperparameter discussion in Section 4.3 and reported the RL regularization and clipping sensitivity analysis in Appendix D.7.
W6. Experiments are conducted on moderate-size public citation/web/e-commerce benchmarks. The paper does not analyze memory cost for building large-scale semantic/structural database when source domains contain millions of graph/node instances. For real-world large-scale source corpus, storing embedding for every source instance will bring non-trivial memory overhead, and approximate nearest-neighbor retrieval may degrade utility-calibration performance (Sec4).
R6. Thank you for your valuable feedback. We evaluated database storage, million-instance retrieval efficiency, and ANN effects on utility calibration.
1. Storage and Retrieval Scalability. COLA uses 256D semantic embeddings and 32D structural keys (Eqs. 3 and 7). Storage grows linearly with indexed instances. Retained artifacts occupy 0.160 GB (node/ego-graph representations, Exact/IVF indices, and metadata; excluding models and raw graphs), extrapolating to 6.23 GB for one million instances under the same multi-index configuration. We benchmarked FAISS IVF-Flat [3] against exact retrieval on one million synthetic vectors per database, with matching dimensions, the same CPU, and four threads.
| Database (1M) | Exact (ms) | IVF (ms) | Recall@M (%) | Speedup |
|---|---|---|---|---|
| Semantic (256D) | 43.28 | 12.11 | 53.91 | 3.58× |
| Structural (32D) | 5.40 | 1.80 | 95.22 | 3.00× |
IVF accelerates retrieval by 3.00–3.58×, with lower recall, especially for semantic vectors. Synthetic benchmarks do not measure downstream accuracy at million-instance scale.
2. ANN Impact on Calibration. We compared exact and IVF retrieval on OGBN-Products Home/Tech (5-shot node classification, 10 support splits).
| Calibration / Inference | Sem. Recall@32 (%) | Str. Recall@16 (%) | Accuracy (%) |
|---|---|---|---|
| Exact / Exact | 100.00 | 100.00 | 66.5 ± 2.6 |
| Exact / IVF | 92.26 | 99.99 | 66.6 ± 3.5 |
| IVF / IVF | 92.26 | 99.99 | 63.0 ± 3.6 |
Inference-only IVF preserves accuracy (66.6% vs. 66.5%), whereas IVF during calibration and inference reduces it to 63.0%. This suggests greater calibration sensitivity; however, policy retraining prevents attributing the entire decrease solely to ANN.
We have reported these analyses in Appendix D.8.
Reference
[1] Haonan Yuan, et al. "Rag-gfm: Overcoming in-memory bottlenecks in graph foundation models via retrieval-augmented generation." In Proceedings of the ACM Web Conference 2026, pp. 626–637, 2026b.
[2] Wang, Zehong, et al. "Beyond message passing: Neural graph pattern machine." arXiv preprint arXiv:2501.18739 (2025).
[3] Douze, Matthijs, et al. "The faiss library." IEEE Transactions on Big Data (2025).
In light of these responses, we hope we have addressed your concerns and that you will consider raising your score. If any concern remains, we would be grateful for the opportunity to clarify it further.
Part I
We are truly grateful for the time you have taken to review our paper and for your insightful comments. We address each concern below.
R1. Thank you for your valuable feedback. COLA focuses on cross-domain few-shot graph adaptation, where a pretrained graph model must transfer to unseen target datasets or semantic domains with very limited labeled data. We follow the LODO-Dataset and LODO-Domain evaluation protocols adopted by RAG-GFM [1], where only one or five labeled instances per class are available for adaptation to unseen target datasets or semantic domains. We clarify the reported results from two perspectives:
Performance under limited supervision and domain shift. Under the LODO protocols, the target dataset or entire semantic domain is excluded from source pretraining, and only one or five labeled examples per class are available for target adaptation. These conditions substantially constrain the target classifier, especially for multiclass prediction under distribution shifts. The difficulty is also reflected in existing methods evaluated under the same protocol. Thus, the absolute accuracies should be interpreted in the context of the limited supervision and cross-domain transfer setting.
Improvements over existing methods. Under identical evaluation protocols, COLA consistently achieves competitive or superior performance compared with representative GFMs. On OGBN-Products, COLA improves upon RAG-GFM from 55.4% to 57.4% in 1-shot node classification and from 64.2% to 66.5% in 5-shot classification. Moreover, increasing supervision from one to five examples per class improves COLA's accuracy from 59.7% to 77.0% on Cora, 41.8% to 57.8% on CiteSeer, and 47.9% to 62.0% on Wiki-CS. The broader label-efficiency analysis in Fig. 3(a–b) further demonstrates that COLA effectively exploits additional target labels while maintaining its source-derived components frozen.
We have clarified the interpretation of the LODO few-shot setting in Section 4.1 and strengthened the discussion of label efficiency and cross-domain performance in Section 4.2.
R2. Thank you for your valuable feedback. COLA focuses on improving the cross-domain adaptability of pretrained graph foundation models through utility-calibrated retrieval. Beyond the node and ego-graph classification tasks evaluated in Table 1, we conducted an additional link prediction experiment to examine its applicability to a different graph learning task.
Under three independent edge splits with identical negative samples and downstream training budgets, COLA achieves the best performance on both metrics, outperforming the frozen backbone by 5.53 percentage points in ROC-AUC and 5.41 points in AP. Compared with retrieval without utility calibration, COLA further improves these metrics by 2.09 and 1.94 points, respectively. Notably, the target domain is excluded from source-side training, and the classification-calibrated utility scorers and null-aware router are directly transferred to link prediction without task-specific recalibration. These results provide additional evidence that COLA's retrieval-enhancement mechanisms can benefit graph prediction tasks beyond classification, complementing the cross-domain evaluation in Table 1.
We have reported the link-prediction protocol and results in Appendix D.6.