- Rating
- 3(+1)
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
This paper proposes UniCG, a single model for both graph comprehension and generation. Graphs are serialized into discrete tokens using canonical BFS ordering for structure and residual vector quantization for node features. A small GPT-style decoder generates graphs via next-token prediction and performs comprehension via pooled hidden states, with separate LoRA adapters for each task to reduce interference. Experiments cover node, graph, and link-level prediction on about 20 datasets, plus generation on a subset of them.
Critical strengths
- The problem is clearly motivated: evaluating comprehension and generation jointly on the same corpora is a useful perspective that existing graph models lack.
- The tokenization is simple and invertible, with a decode-and-check step, and a compact shared vocabulary of a few hundred tokens serves all domains.
- The experiments cover a wide range of tasks and settings, including seen, unseen, and few-shot adaptation.
- The ablations and appendix are thorough, and the authors openly report unfavorable results, such as the frozen-backbone diagnostic and the weak ZINC250k generation.
Critical weaknesses
- The ablation does not support the core design. UniCG is outperformed by Naive on most comprehension tasks and by Gen.Only on several generation tasks, and Naive has half the trainable parameters. The benefit of task-decoupled experts is therefore not established.
- The main results are weak and hard to interpret. The largest comprehension gains come from the knowledge-graph tasks, yet on the same datasets generation falls well behind the state of the art (e.g. on WN18RR nearly 20 points lower). Meanwhile, some baselines appear misconfigured (e.g. GCN at 26.7% on Cora), and G2PT is missing from the main table.
Questions for authors
- Can the authors provide evidence that joint training outperforms single-task training, for example, whether the generation objective improves few-shot comprehension?
- How are negative samples constructed for knowledge-graph link prediction? Can the authors report MRR and Hits@k?
- How were the baselines tuned? Why does GCN reach only 26.7% on Cora?
- Can the authors evaluate the semantic quality of the generated node features?
- Where are the results for TextGraph and Molecule (QM9-derived)?
I am open to raising my score if the authors can convince me of the effectiveness of UniCG and demonstrate that combining generation and comprehension in one model brings a clear benefit.
5 comments
Rebuttal
Response to Question #1
Question #1: Does joint training improve few-shot comprehension over Comp.Only?
A: Thank you for suggesting a direct test of whether joint training helps few-shot comprehension. Under full-data Uni-Seen, UniCG has higher submitted comprehension means than Comp.Only on all seven corpora. The submitted few-shot Comp.Only reference starts from scratch, so we added the source-pretrained control below.
The new Comp.Only control pre-trains on the same source domains and uses the same five support sets as UniCG, each with five labeled examples per class. The last column counts how many support sets favor UniCG. Mean differences are in percentage points. The UniCG means for Cora, Arxiv, and BBBP come from the runs for this comparison and differ from those in the submission table. The archived configurations leave the adaptation freeze flag unset.
| Corpus | Source-pretrained Comp.Only | UniCG | Mean difference | Support sets |
|---|---|---|---|---|
| Cora | 31.53 | 40.20 | +8.66 | 5 / 5 |
| CiteSeer | 42.09 | 39.66 | -2.43 | 1 / 5 |
| PubMed | 57.00 | 56.73 | -0.27 | 2 / 5 |
| WikiCS | 35.84 | 40.96 | +5.12 | 5 / 5 |
| Amazon-Computers | 51.93 | 54.51 | +2.58 | 4 / 5 |
| Amazon-Photo | 71.62 | 71.49 | -0.13 | 3 / 5 |
| Arxiv | 11.47 | 14.47 | +3.00 | 5 / 5 |
| BACE | 56.19 | 61.13 | +4.94 | 5 / 5 |
| BBBP | 51.71 | 54.94 | +3.23 | 4 / 5 |
| HIV | 65.38 | 63.48 | -1.90 | 2 / 5 |
| WN18RR | 66.63 | 63.33 | -3.30 | 1 / 5 |
| FB15k-237 | 56.28 | 60.55 | +4.27 | 5 / 5 |
UniCG leads on all five draws on Cora (+8.66 points on average), WikiCS (+5.12), Arxiv (+3.00), BACE (+4.94), and FB15k-237 (+4.27). CiteSeer favors the comprehension-only control on four draws. BBBP illustrates support-set sensitivity, with differences from −9.55 to +12.13 points. The observed benefit varies by corpus.
Both objectives can shape the shared backbone during pre-training. During frozen-backbone adaptation, the generation loss has no direct gradient path into the comprehension adapter. The observed effects vary by corpus.
Response to Question #2
Question #2: How are negative triples sampled? Can you report MRR and Hits@k?
A: Thank you for asking how we sample negative examples and how the resulting task relates to entity ranking. The submitted binary task replaces the tail with a sampled entity from the training graph and rejects candidates that equal the head or form an observed training triple. It aims for one negative per positive, yielding about 3,000 of each class per corpus. Each input is an enclosing subgraph with the candidate edge omitted from its serialized edges and the relation supplied as a prefix.
The published ROC-AUC therefore measures discrimination of constructed candidates. A separate sampled-tail evaluation can rank the true tail against corruptions for a fixed head and relation using the trained prediction head. The diagnostic available to us initializes a new head, so its scores cannot support a ranking claim about trained UniCG. For 51 candidates, the expected MRR of a uniform random ordering is approximately 0.0886. Appendix J specifies the binary protocol and what is required for a comparable MRR/Hits@k evaluation.
Response to Question #3
Question #3: How were baselines tuned, and why is Cora GCN accuracy 26.72%?
A: Thank you for flagging the low Cora result and asking about the training settings. The common-input runner uses quantized node features, graph pooling, 60 downstream epochs, Adam at learning rate 0.001, weight decay 0.0005, and batch size 128. Graph foundation model pre-training uses 30 epochs where applicable. These settings evaluate center-node labels on sampled ego-graphs.
GCN's 26.72% on Cora is below the 32.3% majority rate. PubMed shows the same issue (36.00% versus 45.3%). The experiment does not isolate which setting caused these deficits. We retain the original scores and exclude these sub-majority results and the diagnosed GFT configuration from the conclusions.
A separate GCN probe on the same quantized ego-graph corpus. We trained a supervised multi-domain encoder for 30 epochs and fit a target read-out with the encoder frozen for 60 epochs. Across five seeds, this gives 62.13 ± 0.93% on Cora, 77.20 ± 0.65% on PubMed, and 58.50 ± 0.60% on Arxiv. The Cora score is 2.34 points below the submitted UniCG score, while the PubMed and Arxiv probe scores exceed it by 5.13 and 5.90 points. The probe shows that GCN can perform substantially better on these inputs. Appendix J reports the probe separately and Section 5.2 qualifies the original comparison.
Rebuttal
Response to Question #4
Question #4: How well do generated node features reflect the graph structure?
A: Thank you for asking us to examine generated node features directly. The RVQ sensitivity study evaluates predictive utility. To assess generated semantics, we need to measure the distribution of generated features and their alignment with graph structure. Code occupancy describes token usage. A structural test must group residual codes by node before comparing edge-wise association against a within-graph permutation. Our current diagnostic indexes a flattened list of residual codes by node ID, so its reported association does not measure node-level alignment. Appendix J explains the limitation. The evidence in this paper supports the utility of the semantic codes for prediction. The fidelity of generated attributes remains an open empirical question.
Response to Question #5
Question #5: Where are the TextGraph and QM9-derived Molecule results?
A: Thank you for noticing that the two regression corpora were missing from the results. Appendix J now includes the five-seed Uni-Seen regression results.
| Corpus | Targets | Comp.Only | Naive | UniCG |
|---|---|---|---|---|
| TextGraph | 4 | 0.3903 ± 0.0266 | 0.4687 ± 0.0041 | 0.3615 ± 0.0167 |
| Molecule (QM9-derived) | 5 | 0.7539 ± 0.0045 | 0.7792 ± 0.0094 | 0.7743 ± 0.0078 |
TextGraph has four prediction targets and the QM9-derived Molecule corpus has five. The separate QM9 de novo benchmark evaluates sampled molecules. UniCG trails both references on TextGraph, and its Molecule regression mean falls between Comp.Only and Naive.
Changes in the revision (shown in blue):
- Section 5 and Appendix J. We add the capacity comparison, the source-pretrained comprehension control, and the knowledge-graph generation breakdown.
- Appendices C and J. We document the baseline inputs, the GCN probe, and the negative sampling protocol.
- Appendix J. We report TextGraph and Molecule regression results and clarify the scope of the generated attributes evaluation.
Conclusion
Thank you again for the detailed suggestions. The capacity comparison at a common seed and the source-pretrained comprehension control address the two main ablation questions. We also added the omitted regression results and clarified the scope of the knowledge-graph and semantic evaluations. We hope the revised evidence addresses your concerns about what the framework achieves.
Rebuttal
Thank you for your detailed review and for identifying the comparisons that would make the results easier to interpret. Your questions about expert capacity and the comprehension-only control helped us test two explanations that the original ablations could not distinguish. We discuss those results and the scope of the knowledge-graph and semantic evaluations below.
Preview
Response to Weakness #1
A: Thank you for pointing out that the original ablation compares adapters with different parameter counts. Naive's 1.18M adapter parameters are half of UniCG's standard 2.36M adapter budget. Both methods also train the backbone during pre-training. We added one larger shared adapter and two smaller decoupled adapters.
Each corpus reports ROC-AUC / V.U.N. in percent. The standard rows reproduce five-run submission means, and the new rows are seed-0 runs.
Decoupled routing leads by 48.85 and 46.35 V.U.N. points on BACE and BBBP at 1.18M total adapter parameters and by 37.09 and 39.45 points at 2.36M. Doubling the shared adapter adds 8.51 and 8.10 points. Total adapter count alone cannot account for the observed gap. The configurations also differ in active rank per objective. WN18RR favors the shared configuration on V.U.N. at both counts.
Naive leads on five of seven submitted Uni-Seen comprehension tasks, and the frozen-backbone diagnostic does not favor UniCG. On WN18RR the standard UniCG variant has lower structural MMD (0.011 versus 0.055), while Naive has higher V.U.N. The clearest routing effect is the molecular generation recovery under the tested conditions.
With seed 0 for all four configurations, the V.U.N. gains on BACE and BBBP are 50.37 and 44.67 points at 1.18M parameters and 37.46 and 39.86 points at 2.36M.
Response to Weakness #2
A: Thank you for drawing attention to the gap between prediction and generation on WN18RR. On WN18RR, the submitted UniCG ROC-AUC is 95.66%. Its generation V.U.N. is 54.37% versus 73.67% for reproduced DeFoG, a gap of 19.30 points. Structural MMD favors UniCG (0.011 versus 0.069). The descriptors are degree, clustering, and spectrum. Appendix C now documents the different reference construction in the two evaluation paths.
On WN18RR UniCG has higher validity and lower uniqueness and novelty. The decomposition explains how a favorable structural MMD can coexist with lower V.U.N. The Cora GCN comparison is limited by its sub-majority result under the common quantized-input protocol, discussed in Q3. G2PT has a published prediction pathway. Our implemented comparison uses its generation results.