acceptodds
Under review as a conference paper at ICLR 2027

UniCG: A Framework for Unifying Graph Comprehension and Generation

Abstract

The pursuit of general-purpose intelligence has motivated unified architectures that both comprehend and generate content, with notable progress in language and vision. However, prevailing graph learning paradigms still treat these capabilities separately: comprehension models map graphs to predictions, whereas generative models synthesize graphs without being evaluated for comprehension. Consequently, enabling a single model to comprehend and generate graphs through a shared representation remains a major challenge. In this paper, we propose UniCG, a novel framework for Unifying graph Comprehension and Generation that addresses both the representational incompatibility and optimization interference between these capabilities. Specifically, to establish a common graph interface, UniCG combines lossless graph serialization with residual quantization of node semantics to construct a shared discrete graph vocabulary. To support both capabilities within one model, UniCG formulates generation as next-token prediction and comprehension as a pooled read-out over a single autoregressive backbone. To alleviate the asymmetric interference exposed by joint training, UniCG routes the two objectives through task-decoupled experts while retaining the shared backbone. Extensive experiments across 20 graph domains, four comprehension tasks, and three generation regimes against 13 baselines demonstrate that UniCG jointly supports both capabilities, achieving competitive comprehension performance while generating valid and novel graphs.

Primary area:
learning on graphs and other geometries & topologies
Keywords:
Unified Graph Comprehension and Generation, Graph Representation Learning, Graph Foundation Models, Graph Generation

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_2fTcOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper proposes UniCG, a single model for both graph comprehension and generation. Graphs are serialized into discrete tokens using canonical BFS ordering for structure and residual vector quantization for node features. A small GPT-style decoder generates graphs via next-token prediction and performs comprehension via pooled hidden states, with separate LoRA adapters for each task to reduce interference. Experiments cover node, graph, and link-level prediction on about 20 datasets, plus generation on a subset of them.

Critical strengths
  1. The problem is clearly motivated: evaluating comprehension and generation jointly on the same corpora is a useful perspective that existing graph models lack.
  2. The tokenization is simple and invertible, with a decode-and-check step, and a compact shared vocabulary of a few hundred tokens serves all domains.
  3. The experiments cover a wide range of tasks and settings, including seen, unseen, and few-shot adaptation.
  4. The ablations and appendix are thorough, and the authors openly report unfavorable results, such as the frozen-backbone diagnostic and the weak ZINC250k generation.
Critical weaknesses
  1. The ablation does not support the core design. UniCG is outperformed by Naive on most comprehension tasks and by Gen.Only on several generation tasks, and Naive has half the trainable parameters. The benefit of task-decoupled experts is therefore not established.
  2. The main results are weak and hard to interpret. The largest comprehension gains come from the knowledge-graph tasks, yet on the same datasets generation falls well behind the state of the art (e.g. on WN18RR nearly 20 points lower). Meanwhile, some baselines appear misconfigured (e.g. GCN at 26.7% on Cora), and G2PT is missing from the main table.
Questions for authors
  1. Can the authors provide evidence that joint training outperforms single-task training, for example, whether the generation objective improves few-shot comprehension?
  2. How are negative samples constructed for knowledge-graph link prediction? Can the authors report MRR and Hits@k?
  3. How were the baselines tuned? Why does GCN reach only 26.7% on Cora?
  4. Can the authors evaluate the semantic quality of the generated node features?
  5. Where are the results for TextGraph and Molecule (QM9-derived)?

I am open to raising my score if the authors can convince me of the effectiveness of UniCG and demonstrate that combining generation and comprehension in one model brings a clear benefit.

5 comments

Thank you for your detailed review and for identifying the comparisons that would make the results easier to interpret. Your questions about expert capacity and the comprehension-only control helped us test two explanations that the original ablations could not distinguish. We discuss those results and the scope of the knowledge-graph and semantic evaluations below.

Preview

  • Weakness #1: Capacity-controlled expert ablation
  • Weakness #2: Knowledge-graph results and baseline comparisons
  • Question #1: Joint training and few-shot transfer
  • Question #2: KG negative sampling and ranking
  • Question #3: Cora GCN and tuning
  • Question #4: Generated node attributes
  • Question #5: TextGraph and Molecule regression
  • Changes in the revision
  • Conclusion

Response to Weakness #1

Weakness #1: Do separate experts help at equal adapter capacity, given that Naive and Gen.Only lead on some tasks?

A: Thank you for pointing out that the original ablation compares adapters with different parameter counts. Naive's 1.18M adapter parameters are half of UniCG's standard 2.36M adapter budget. Both methods also train the backbone during pre-training. We added one larger shared adapter and two smaller decoupled adapters.

Each corpus reports ROC-AUC / V.U.N. in percent. The standard rows reproduce five-run submission means, and the new rows are seed-0 runs.

ConfigurationAdaptersBACEBBBPWN18RR
Naive, rank 321.18M61.64 / 12.8967.54 / 12.3797.37 / 85.55
Naive, rank 642.36M64.97 / 21.4070.63 / 20.4796.77 / 81.61
UniCG, 2 × rank 161.18M63.28 / 61.7468.33 / 58.7294.46 / 52.00
UniCG, 2 × rank 322.36M65.13 / 58.4967.87 / 59.9295.66 / 54.37

Decoupled routing leads by 48.85 and 46.35 V.U.N. points on BACE and BBBP at 1.18M total adapter parameters and by 37.09 and 39.45 points at 2.36M. Doubling the shared adapter adds 8.51 and 8.10 points. Total adapter count alone cannot account for the observed gap. The configurations also differ in active rank per objective. WN18RR favors the shared configuration on V.U.N. at both counts.

Naive leads on five of seven submitted Uni-Seen comprehension tasks, and the frozen-backbone diagnostic does not favor UniCG. On WN18RR the standard UniCG variant has lower structural MMD (0.011 versus 0.055), while Naive has higher V.U.N. The clearest routing effect is the molecular generation recovery under the tested conditions.

With seed 0 for all four configurations, the V.U.N. gains on BACE and BBBP are 50.37 and 44.67 points at 1.18M parameters and 37.46 and 39.86 points at 2.36M.


Response to Weakness #2

Weakness #2: Explain the WN18RR generation gap, the low Cora GCN score, and the absence of G2PT from the main table.

A: Thank you for drawing attention to the gap between prediction and generation on WN18RR. On WN18RR, the submitted UniCG ROC-AUC is 95.66%. Its generation V.U.N. is 54.37% versus 73.67% for reproduced DeFoG, a gap of 19.30 points. Structural MMD favors UniCG (0.011 versus 0.069). The descriptors are degree, clustering, and spectrum. Appendix C now documents the different reference construction in the two evaluation paths.

CorpusMethodValidityUniquenessNoveltyV.U.N.
WN18RRDeFoG96.4085.0989.7473.67
WN18RRUniCG99.6068.8579.2454.37
FB15k-237DeFoG100.00100.00100.00100.00
FB15k-237UniCG94.0098.6599.8692.59

On WN18RR UniCG has higher validity and lower uniqueness and novelty. The decomposition explains how a favorable structural MMD can coexist with lower V.U.N. The Cora GCN comparison is limited by its sub-majority result under the common quantized-input protocol, discussed in Q3. G2PT has a published prediction pathway. Our implemented comparison uses its generation results.

Response to Question #1

Question #1: Does joint training improve few-shot comprehension over Comp.Only?

A: Thank you for suggesting a direct test of whether joint training helps few-shot comprehension. Under full-data Uni-Seen, UniCG has higher submitted comprehension means than Comp.Only on all seven corpora. The submitted few-shot Comp.Only reference starts from scratch, so we added the source-pretrained control below.

The new Comp.Only control pre-trains on the same source domains and uses the same five support sets as UniCG, each with five labeled examples per class. The last column counts how many support sets favor UniCG. Mean differences are in percentage points. The UniCG means for Cora, Arxiv, and BBBP come from the runs for this comparison and differ from those in the submission table. The archived configurations leave the adaptation freeze flag unset.

CorpusSource-pretrained Comp.OnlyUniCGMean differenceSupport sets
Cora31.5340.20+8.665 / 5
CiteSeer42.0939.66-2.431 / 5
PubMed57.0056.73-0.272 / 5
WikiCS35.8440.96+5.125 / 5
Amazon-Computers51.9354.51+2.584 / 5
Amazon-Photo71.6271.49-0.133 / 5
Arxiv11.4714.47+3.005 / 5
BACE56.1961.13+4.945 / 5
BBBP51.7154.94+3.234 / 5
HIV65.3863.48-1.902 / 5
WN18RR66.6363.33-3.301 / 5
FB15k-23756.2860.55+4.275 / 5

UniCG leads on all five draws on Cora (+8.66 points on average), WikiCS (+5.12), Arxiv (+3.00), BACE (+4.94), and FB15k-237 (+4.27). CiteSeer favors the comprehension-only control on four draws. BBBP illustrates support-set sensitivity, with differences from −9.55 to +12.13 points. The observed benefit varies by corpus.

Both objectives can shape the shared backbone during pre-training. During frozen-backbone adaptation, the generation loss has no direct gradient path into the comprehension adapter. The observed effects vary by corpus.


Response to Question #2

Question #2: How are negative triples sampled? Can you report MRR and Hits@k?

A: Thank you for asking how we sample negative examples and how the resulting task relates to entity ranking. The submitted binary task replaces the tail with a sampled entity from the training graph and rejects candidates that equal the head or form an observed training triple. It aims for one negative per positive, yielding about 3,000 of each class per corpus. Each input is an enclosing subgraph with the candidate edge omitted from its serialized edges and the relation supplied as a prefix.

The published ROC-AUC therefore measures discrimination of constructed candidates. A separate sampled-tail evaluation can rank the true tail against corruptions for a fixed head and relation using the trained prediction head. The diagnostic available to us initializes a new head, so its scores cannot support a ranking claim about trained UniCG. For 51 candidates, the expected MRR of a uniform random ordering is approximately 0.0886. Appendix J specifies the binary protocol and what is required for a comparable MRR/Hits@k evaluation.


Response to Question #3

Question #3: How were baselines tuned, and why is Cora GCN accuracy 26.72%?

A: Thank you for flagging the low Cora result and asking about the training settings. The common-input runner uses quantized node features, graph pooling, 60 downstream epochs, Adam at learning rate 0.001, weight decay 0.0005, and batch size 128. Graph foundation model pre-training uses 30 epochs where applicable. These settings evaluate center-node labels on sampled ego-graphs.

GCN's 26.72% on Cora is below the 32.3% majority rate. PubMed shows the same issue (36.00% versus 45.3%). The experiment does not isolate which setting caused these deficits. We retain the original scores and exclude these sub-majority results and the diagnosed GFT configuration from the conclusions.

A separate GCN probe on the same quantized ego-graph corpus. We trained a supervised multi-domain encoder for 30 epochs and fit a target read-out with the encoder frozen for 60 epochs. Across five seeds, this gives 62.13 ± 0.93% on Cora, 77.20 ± 0.65% on PubMed, and 58.50 ± 0.60% on Arxiv. The Cora score is 2.34 points below the submitted UniCG score, while the PubMed and Arxiv probe scores exceed it by 5.13 and 5.90 points. The probe shows that GCN can perform substantially better on these inputs. Appendix J reports the probe separately and Section 5.2 qualifies the original comparison.

Response to Question #4

Question #4: How well do generated node features reflect the graph structure?

A: Thank you for asking us to examine generated node features directly. The RVQ sensitivity study evaluates predictive utility. To assess generated semantics, we need to measure the distribution of generated features and their alignment with graph structure. Code occupancy describes token usage. A structural test must group residual codes by node before comparing edge-wise association against a within-graph permutation. Our current diagnostic indexes a flattened list of residual codes by node ID, so its reported association does not measure node-level alignment. Appendix J explains the limitation. The evidence in this paper supports the utility of the semantic codes for prediction. The fidelity of generated attributes remains an open empirical question.


Response to Question #5

Question #5: Where are the TextGraph and QM9-derived Molecule results?

A: Thank you for noticing that the two regression corpora were missing from the results. Appendix J now includes the five-seed Uni-Seen regression results.

CorpusTargetsComp.OnlyNaiveUniCG
TextGraph40.3903 ± 0.02660.4687 ± 0.00410.3615 ± 0.0167
Molecule (QM9-derived)50.7539 ± 0.00450.7792 ± 0.00940.7743 ± 0.0078

TextGraph has four prediction targets and the QM9-derived Molecule corpus has five. The separate QM9 de novo benchmark evaluates sampled molecules. UniCG trails both references on TextGraph, and its Molecule regression mean falls between Comp.Only and Naive.


Changes in the revision (shown in blue):

  • Section 5 and Appendix J. We add the capacity comparison, the source-pretrained comprehension control, and the knowledge-graph generation breakdown.
  • Appendices C and J. We document the baseline inputs, the GCN probe, and the negative sampling protocol.
  • Appendix J. We report TextGraph and Molecule regression results and clarify the scope of the generated attributes evaluation.

Conclusion

Thank you again for the detailed suggestions. The capacity comparison at a common seed and the source-pretrained comprehension control address the two main ablation questions. We also added the omitted regression results and clarified the scope of the knowledge-graph and semantic evaluations. We hope the revised evidence addresses your concerns about what the framework achieves.

Reviewer_bvjCOpenReview ↗
Rating
2
Confidence
4
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

UniCG is a framework that does graph comprehension and generation with the same model. It combines a graph tokenizer with a decoder-only transformer, and design a tokenization strategy to enable task prediction and comprehension at the same time. When encountering different tasks, UniCG trains domain specific LoRA to accomodate them. The empirical results how noticeable improvement over existing baselines.

Critical strengths

The scope is ambitious to unify generation with comprehension, and it seems to fit the nature of graph learning well, using understanding about graph inductive bias to help learning downstream tasks. The framework seems to enable this capability with unified vocabulary, multi-domain training, etc.

Critical weaknesses
  • The paper claims to be the first such unification work, I did some literature search, LGD unified the two ends already [1] (BTW, the paper seemed to cite this one in the Appendix, but appears as [?], so I am not 100% sure this is the one). Not comparing it is fine, but the " within one shared multi-domain model over the same corpora" is just training a base model on multiple dataset, and I am not convinced by this as an important addition towards unification, as this is essentially just pretraining that is used in many previous works. Likewise, G2PT, which the paper also cites, does generation and prediction at the same time. Given that the architecture still resembles that of G2PT, and a separate finetuning is required to use the system properly, I am not very excited by the novelty and knowledge brought by this paper.

  • At the same time, the performance improvement margin is mild. Mild architectural/ conceptual novelty is fine, but it should bring strong/surprising performance boost to showcase the necessity of such design choices, and I am not seeing it in the paper.

[1] Zhou, Cai, Xiyuan Wang, and Muhan Zhang. "Unifying generation and prediction on graphs with latent graph diffusion." Advances in Neural Information Processing Systems 37 (2024): 61963-61999.

Questions for authors
  • In Table 3, the comp.only is very close to UniCG, and Gen.Only even beats UniCG, does this say the unification is not beneficial? If so, what is the motivation of such unification.
3 comments

Thank you for taking the time to examine whether UniCG adds something beyond existing frameworks and pre-training across datasets. Your question about the single-task models helped us clarify what sharing buys and where the trade-offs remain. We address that question alongside the comparisons at equal adapter budgets.

Preview

  • Weakness #1: Contribution beyond multi-domain pre-training
  • Weakness #2: Capacity and routing
  • Question #1: Value of unification
  • Changes in the revision
  • Conclusion

Response to Weakness #1

Weakness #1: What does UniCG add beyond LGD, G2PT, and multi-domain pre-training, given that it still requires target adaptation? Fix the LGD citation.

A: Thank you for asking what UniCG contributes beyond pre-training on several datasets. Generation needs a decodable structural description, comprehension needs representations useful for prediction, and joint optimization needs a way to give each objective its own update path. UniCG addresses these requirements with a shared graph interface, one backbone with two read-outs, and objective-specific adapters. Both objectives update the backbone during pre-training. During target adaptation the backbone is frozen and the experts and a target head are trained. The experts are indexed by objective.

LGD [1] establishes a common latent diffusion formulation with separate trained models per task in its reported experiments. G2PT [2] establishes autoregressive generation and downstream predictive fine-tuning. Our shared-model evaluation asks where joint training improves either capability and where the two trade off.

Reference
[1] Cai Zhou, Xiyuan Wang, and Muhan Zhang. “Unifying Generation and Prediction on Graphs with Latent Graph Diffusion.” NeurIPS, 2024.
[2] Xiaohui Chen et al. “Graph Generative Pre-trained Transformer.” ICML, 2025.


Response to Weakness #2

Weakness #2: Do the performance gains justify the architectural choices?

A: Thank you for asking for evidence that the design yields a substantial improvement. Appendix J tests the shared and decoupled variants at two total adapter parameter counts.

Each corpus reports ROC-AUC / V.U.N. in percent. The standard rows reproduce five-run submission means, and the new rows are seed-0 runs.

ConfigurationAdaptersBACEBBBPWN18RR
Naive,rank321.18M61.64/12.8967.54/12.3797.37/85.55
Naive,rank642.36M64.97/21.4070.63/20.4796.77/81.61
UniCG,2×rank161.18M63.28/61.7468.33/58.7294.46/52.00
UniCG,2×rank322.36M65.13/58.4967.87/59.9295.66/54.37

Decoupled routing leads by 48.85 and 46.35 V.U.N. points on BACE and BBBP at 1.18M total adapter parameters and by 37.09 and 39.45 points at 2.36M. Doubling the shared adapter adds 8.51 and 8.10 points. Total adapter count alone cannot account for the observed gap. The configurations also differ in active rank per objective. WN18RR favors the shared configuration on V.U.N. at both counts. With seed 0 for all four configurations, the V.U.N. gains on BACE and BBBP are 50.37 and 44.67 points at 1.18M parameters and 37.46 and 39.86 points at 2.36M.


Response to Question #1

Question #1: What does unification offer when Comp.Only is competitive and Gen.Only sometimes performs better?

A: Thank you for raising the comparison with the single-task specialists. It helps clarify the purpose of sharing a backbone. One 10.65M-parameter backbone supports both capabilities, with about 1.18M parameters in each standard expert and separate predictive heads. This is a parameter-sharing benefit. A runtime or matched-compute advantage would require additional measurements.

In the submitted full-data comparison, UniCG comprehension means exceed Comp.Only on the seven Table 3 corpora. Against the scratch 5-shot reference, three means are higher and four are lower. The new source-pretrained control shows domain-dependent differences. The Gen.Only references have a different target adaptation exposure. On BACE and BBBP the capacity-controlled comparisons against Naive give more direct evidence that routing can recover generation while retaining a predictive pathway.


Changes in the revision (shown in blue):

  • Sections 1 and 2. We explain how the shared graph representation and backbone support both objectives and how that design relates to prior work.
  • Section 5 and Appendix J. We compare shared and decoupled adapters at two parameter budgets and clarify the scope of the specialist comparisons.

Conclusion

Thank you again for challenging us to show what the unified design contributes in practice. The capacity comparison shows a substantial generation benefit on BACE and BBBP at both tested parameter budgets. The WN18RR results help delimit that benefit. We hope this addresses both the motivation for sharing and the reason for giving each objective its own adapter.

Dear authors,

I appreciate the change where you take a more consrevative stance on the contribution and made. And I appreciate your diligence to provide many new results. I am leaning toward borderline for this paper. The reason for not giving it score 3:

  • The performance gain by unification is mixed at best, with a handful of data falling behind (from appendix J)

  • For Q1, two things, 1, at scale of 10M param, I am not sure how important it is for parameter sharing. 2, the parameter sharing benefit does not justify training the two objectives together, you can still use the same backbone without unification, achieving the same benefit.

We sincerely thank you for taking another look at our paper and for explaining what would make the case for unification more convincing. We designed UniCG as an initial empirical study of whether a single model can comprehend and generate graphs on the same corpora through a shared representation. Your questions help us distinguish the feasibility of that design from the gains we observe under specific training and adaptation settings.

1. Mixed performance gains in Appendix J. The matched-source five-shot control shows that the comprehension gains vary by corpus. UniCG leads on all five shared support sets for Cora (+8.66 points on average), BACE (+4.94), and FB15k-237 (+4.27). The comprehension-only control leads on four of five support sets for CiteSeer (−2.43 points on average for UniCG). Table 1 addresses the broader feasibility question by evaluating both capabilities on the same seven corpora. For example, UniCG achieves 65.13 ROC-AUC and 58.49 V.U.N. on BACE, and 88.61 ROC-AUC and 92.59 V.U.N. on FB15k-237. The five-shot results characterize one aspect of that design: how well the shared representation supports prediction when target labels are scarce.

2(i). The role of parameter sharing at the 10M scale. We agree that the storage savings alone provide limited motivation at this scale. The 10.65M-parameter backbone gives both tasks access to a common graph representation. A predictive readout uses that representation for comprehension, and autoregressive decoding uses it for generation. Separate experts give each objective its own task-specific parameters.

2(ii). What joint pre-training adds when both tasks already share a backbone. To test your proposed setup, we compared backbones pre-trained with generation only, comprehension only, or both objectives. In each case, we initialized both task adapters and the prediction head afresh. Each backbone was pre-trained for 8,000 steps at batch size 48. We used 500 adaptation steps in each of two settings: with the backbone frozen so each loss updates its own branch, and with full fine-tuning so adaptation also updates the backbone. The single-seed controls below report ROC-AUC / V.U.N. (%). The complete UniCG pipeline in Table 1 carries its pre-trained experts into adaptation.

AdaptationBackbone pre-trainingBACEBBBPHIVWN18RRFB15k-237
Frozen backboneGeneration only67.0 / 52.774.1 / 58.778.5 / 62.197.4 / 44.097.5 / 91.3
Frozen backboneComprehension only64.1 / 1.367.4 / 0.075.0 / 1.291.7 / 28.485.6 / 32.4
Frozen backboneJoint63.3 / 2.871.0 / 3.167.2 / 2.496.7 / 96.090.0 / 88.1
Full fine-tuningGeneration only67.8 / 58.574.0 / 57.875.6 / 61.297.5 / 30.796.7 / 89.0
Full fine-tuningComprehension only62.3 / 2.165.4 / 1.874.1 / 2.991.1 / 88.385.8 / 78.5
Full fine-tuningJoint62.1 / 48.168.4 / 48.868.7 / 56.997.0 / 82.590.4 / 95.2

The frozen comparison shows that a single-objective backbone can serve both tasks well. Generation-only pre-training gives the strongest pair of scores on all three molecular corpora and on FB15k-237. On WN18RR, the jointly pre-trained backbone reaches 96.0 V.U.N. versus 44.0 for generation-only pre-training, with similar ROC-AUC (96.7 versus 97.4). With fresh adapters and a frozen joint backbone, molecular V.U.N. ranges from 2.4 to 3.1. With full fine-tuning, the joint initialization reaches at least 48.1 V.U.N. on all five corpora, and generation-only initialization still leads on the molecular corpora. The value of joint pre-training therefore depends on both the corpus and the adaptation protocol.

For models trained on both objectives, the capacity-matched analysis in the revised appendix shows where UniCG's task routing helps. At the same seed and 1.18M total adapter parameters, separate experts improve generation V.U.N. by 50.37 points on BACE and 44.67 on BBBP over a shared adapter. This shows that how the objectives are routed can materially affect whether one backbone retains both capabilities.

Thank you again for your time and valuable insights, and we hope our responses can address all your concerns. If these additional results address your remaining concerns, we would be grateful if you would kindly consider raising the rating.

Reviewer_EizHOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes UniCG, a single decoder-only transformer (6 layers, width 384, about 13.4M parameters including adapters) that both predicts on graphs and generates them over the same multi-domain corpora. Graphs become token sequences through a canonical BFS serialization of nodes and typed edges, which is invertible by construction, combined with a three-level, 64-entry residual vector quantization of continuous node features into semantic tokens; one shared vocabulary covers all domains. Generation is next-token prediction, optionally conditioned on a density or relation token and decoded under a grammar mask; comprehension is a mask-mean-pooled read-out with a per-domain linear head, covering node classification on 2-hop ego-graphs, graph classification, link prediction on enclosing subgraphs, and structural regression. To limit interference between the two objectives, each gets its own rank-32 LoRA expert on the shared backbone with deterministic routing; training is round-robin multi-domain pre-training (8,000 steps) followed by target-domain adaptation with the backbone frozen. Experiments span 20 domains (citation and co-purchase, molecular, knowledge-graph, synthetic structure and controlled diagnostics), 13 baselines, Uni-Seen and Uni-Unseen settings and 1/5/10-shot adaptation. The paper reports competitive comprehension against graph foundation models, V.U.N. and MMD generation results against DiGress, DeFoG, G2PT and an Erdős–Rényi reference, ablations against Comp.Only, Gen.Only and Naive (single shared adapter) variants, a frozen-backbone diagnostic, and sensitivity to adapter rank and scaling and to RVQ depth and codebook size. The claimed contributions are the unified formulation and evaluation, the invertible vocabulary with task-decoupled experts, and the empirical demonstration that one model can support both capabilities.

Critical strengths

The paper poses a clear question, whether one autoregressive model can both predict on and generate graphs over the same corpora, and builds a working system around a lossless structural serialization plus residual-quantized node features, with invertibility checked during corpus construction rather than assumed.

The evaluation is broad: 20 domains, four comprehension task types, three generation regimes, Uni-Seen / Uni-Unseen / few-shot settings, five seeds for the main tables, and dedicated generation benchmarks reported with both V.U.N. and MMD.

The ablation design is the right one. Matched Comp.Only, Gen.Only and Naive (single shared adapter) variants isolate what joint training and task decoupling each contribute.

Critical weaknesses

The "first framework" claim does not hold. G2PT (Chen et al., ICML 2025), the paper's own closest baseline, fine-tunes the same autoregressive graph model for molecular property prediction on MoleculeNet, so the statement that generation baselines "provide no predictive read-out" is wrong for G2PT. Latent Graph Diffusion (Zhou et al., NeurIPS 2024) solves classification, regression and generation in one model and appears only through a broken "(?)" citation, excluded from comparison. InstructGraph (Wang et al., Findings of ACL 2024) also trains one model for graph reasoning and graph generation. The defensible novelty is multi-domain pre-training over a shared discrete vocabulary with task-decoupled experts, and the paper should claim that.

Several baseline numbers indicate mis-configured baselines, which undermines Table 1. GCN reaches 26.7% on Cora and 36.0% on PubMed, at or below the majority-class rates of those datasets (about 30% and 40%); GFT reports identical values across five seeds on four node datasets (29.33 ±0.00, 22.18 ±0.00, 37.29 ±0.00, 24.32 ±0.00), the signature of a collapsed predictor; GCN and GAT reach R² of only 0.4–0.6 on structural targets such as node and edge counts, which a sum-pooled GNN recovers trivially; and the reproduced DiGress and DeFoG obtain 0.9% and 2.1% V.U.N. on BACE under a validity predicate that only requires connectivity. Either the ego-graph protocol or the matched training budget is crippling the baselines, and the paper needs to show that it is not.

The few-shot claims rest on differences inside the noise. In Table 2 the margins over the runner-up are 0.4, 1.3 and 2.4 points with standard deviations of 1–6 and no significance test, and the matched Comp.Only specialist (Table 3) beats UniCG on four of the seven datasets.

Questions for authors

How are dataset-native node features of different dimensionality (Cora 1433, PubMed 500, Arxiv 128) mapped into the shared codebooks C_ℓ? Are the RVQ codebooks per domain, and under Uni-Unseen do the held-out domain's node features enter the k-means fit in Algorithm 1 (line 1)?

What inputs do GCN/GAT and the graph-foundation-model baselines receive for node classification: the same 30-node, 2-hop ego-graphs with native features, or quantized features? Please report the majority-class rate on the 1,500-graph ego subsets and explain the sub-majority GCN numbers (26.7% Cora, 36.0% PubMed) and the zero-variance GFT results. For DiGress and DeFoG on BACE/BBBP, what are the separate validity, uniqueness and novelty components behind 0.9% and 2.1% V.U.N., and what training budget was used relative to the original papers?

Why is G2PT absent from Table 1, given that it is the closest autoregressive model and its own paper fine-tunes it for property prediction? Can G2PT's comprehension numbers be reported on the same corpora?

4 comments

Thank you for your careful and constructive review. Your comments identified where the paper needed to be more precise about prior work, what the baseline comparison measures, and which target information is available under Uni-Unseen. We appreciate the opportunity to clarify those points and report the additional controls below.

Preview

  • Weakness #1: Contribution and prior work
  • Weakness #2: Reliability of the baselines
  • Weakness #3: Few-shot evidence
  • Question #1: Codebooks and target data access
  • Question #2: Baseline inputs and reproduction budgets
  • Question #3: G2PT comprehension
  • Changes in the revision
  • Conclusion

Response to Weakness #1

Weakness #1: Qualify the novelty claim in light of G2PT's property prediction, LGD, and InstructGraph. Fix the LGD citation and clarify why LGD was excluded from the comparison.

A: Thank you for identifying the prior work that our original wording overlooked. We removed the priority claim and corrected the characterization of G2PT. LGD [1] formulates generation and prediction within latent diffusion, and its reported experiments train separate models for individual tasks. G2PT [2] fine-tunes an autoregressive representation for molecular property prediction. InstructGraph [3] supports graph reasoning and generation through graph-centric instruction tuning and preference alignment.

UniCG jointly trains next-token generation and a pooled predictive read-out over a multi-domain backbone. Its discrete interface preserves graph structure and represents continuous attributes through residual quantization. Separate adapters provide objective-specific update paths while both objectives train the backbone. Evaluation on common corpora exposes the effect of sharing on each capability. Appendix J shows that routing matters for BACE and BBBP generation at both tested adapter counts.

Reference
[1] Cai Zhou, Xiyuan Wang, and Muhan Zhang. “Unifying Generation and Prediction on Graphs with Latent Graph Diffusion.” NeurIPS, 2024.
[2] Xiaohui Chen et al. “Graph Generative Pre-trained Transformer.” ICML, 2025.
[3] Jianing Wang et al. “InstructGraph: Boosting Large Language Models via Graph-centric Instruction Tuning and Preference Alignment.” Findings of ACL, 2024.


Response to Weakness #2

Weakness #2: Explain the low GCN scores, identical GFT results, and weak GCN/GAT structural regression. Could the ego-graph protocol or training budget explain the low DiGress and DeFoG V.U.N. on BACE?

A: Thank you for checking the baseline scores against the class distributions and the task definitions. The submitted Cora and PubMed GCN accuracies fall below the majority rates of the actual ego-graph test sets. The baseline uses quantized features and pools each graph to predict the center node label. An additional comparison with native features and a center node readout would help isolate these effects.

CorpusTrain / test graphsMajority (%)GCN accuracy (%)
Cora1200 / 30032.326.72 ± 0.33
PubMed1200 / 30045.336.00 ± 1.01
Arxiv2400 / 60016.730.00 ± 0.32

The inspected GFT diagnostic predicts a single class for 100% of examples on three node datasets. We retain the submitted row as a record of that configuration and exclude it from the paper's conclusions. For structural tasks, the reported regression score aggregates targets. Mean pooling can obscure graph size, while summing a constant feature recovers node count. The aggregate score alone cannot establish the cause of the node- or edge-count errors. On BACE and BBBP, the reproduced DiGress and DeFoG rows are dominated by low connectivity validity, as detailed in Q2.

A separate GCN probe on the same quantized ego-graph corpus. We trained a supervised multi-domain encoder for 30 epochs and fit a target read-out with the encoder frozen for 60 epochs. Across five seeds, this gives 62.13 ± 0.93% on Cora, 77.20 ± 0.65% on PubMed, and 58.50 ± 0.60% on Arxiv. The Cora score is 2.34 points below the submitted UniCG score, while the PubMed and Arxiv probe scores exceed it by 5.13 and 5.90 points. The probe shows that GCN can perform substantially better on these inputs. Appendix J reports the probe separately and Section 5.2 qualifies the original comparison.

Response to Weakness #3

Weakness #3: Are the few-shot gains reliable given run variability and the lack of a significance test? Comp.Only leads on four of seven datasets in Table 3.

A: Thanks for pointing this out. The submitted 5-shot margins over the external runner-up on Cora, PubMed, and Arxiv are 0.40, 1.26, and 2.43 points. We report these as differences in means. Table 3 also shows the scratch Comp.Only reference ahead of UniCG on 4 of 7 tasks.

The new Comp.Only control pre-trains on the same source domains and uses the same support sets as UniCG, each with five labeled examples per class. The last column counts how many support sets favor UniCG. Mean differences are in percentage points. The UniCG means for Cora, Arxiv, and BBBP come from the runs for this comparison and differ from those in the submission table. The archived configurations leave the adaptation freeze flag unset.

Source-pretrained Comp.OnlyUniCGMean differencSupport sets
Cora31.5340.20+8.665/5
CiteSeer42.0939.66-2.431/5
PubMed57.0056.73-0.272/5
WikiCS35.8440.96+5.125/5
Amazon-Computers51.9354.51+2.584/5
Amazon-Photo71.6271.49-0.133/5
Arxiv11.4714.47+3.005/5
BACE56.1961.13+4.945/5
BBBP51.7154.94+3.234/5
HIV65.3863.48-1.902/5
WN18RR66.6363.33-3.301/5
FB15k-23756.2860.55+4.275/5

UniCG leads on all five draws on Cora (+8.66 points on average), WikiCS (+5.12), Arxiv (+3.00), BACE (+4.94), and FB15k-237 (+4.27). CiteSeer favors the Comp.Only control on four draws. BBBP illustrates support-set sensitivity, with differences from −9.55 to +12.13 points. The observed benefit varies by corpus.


Response to Question #1

Question #1: How does RVQ handle different feature dimensions? Are codebooks fitted per domain, and does the fit use held-out domain features?

A: Thanks for the question. Cora (1433 dimensions), PubMed (500), and Arxiv (128) each use RVQ fitted in their native feature space. Each shared SEM token indexes a level and centroid number in the local codebook. Its embedding is shared by the backbone.

The held-out domain's features enter its own unsupervised k-means fit before the graph-instance split. Uni-Unseen permits target schema and feature access during tokenizer construction. Backbone pre-training excludes the target graph sequences. Target adaptation uses the training split under the stated label budget, and held-out graph instances are used for evaluation. Overlapping ego-graphs can contain the same underlying nodes.


Response to Question #2

Question #2: Specify the baseline inputs and majority-class rates, and explain the low GCN scores and zero GFT variance. Report the V.U.N. and training budgets for DiGress and DeFoG on BACE and BBBP.

A: Thank you for the valuable question. The comprehension implementations receive the same two-hop ego-graph instances capped at 30 nodes, with type one-hot, per-level RVQ code one-hot, and log degree. Their graph-level read-out predicts the center-node label. Test majority rates are 32.3% on Cora, 45.3% on PubMed, and 16.7% on Arxiv. The inspected GFT runs predict a single class throughout.

Validity uses the evaluated graph list as its denominator. For UniCG this list contains parsed graphs, with sequence decodability tracked separately. Uniqueness uses valid graphs. Novelty uses distinct valid graphs relative to the evaluation reference. The DiGress and DeFoG references for novelty are their fitting graphs. UniCG uses up to the first 300 corpus sequences as its reference. V.U.N. is their product within a seed. The table averages components independently across five seeds.

MethodValidityUniquenessNoveltyV.U.N.
BACEDiGress0.9480.0080.000.94
BACEDeFoG2.1180.0080.002.11
BACEUniCG58.49100.00100.0058.49
BBBPDiGress0.2320.0020.000.23
BBBPDeFoG2.81100.0098.002.73
BBBPUniCG59.9999.89100.0059.92

Both reproductions use a width-64, 4-layer edge denoiser, 3000 updates, batch size 32, and Adam at learning rate 0.001. We scored 256 samples per domain and seed, using 50 reverse steps for DiGress and 50 Euler steps for DeFoG [4-5]. The original methods use graph-transformer implementations and benchmark-specific budgets. For example, DeFoG reports full-step schedules of 500 steps for its molecular benchmarks and 1000 for synthetic benchmarks. Neither original budget is being reproduced on these property corpora. These simplified runs characterize the common-corpus protocol.

Hardware. The experimental platform documented in Appendix D uses an Intel Xeon Platinum 8358 CPU at 2.60 GHz, 1 TB DDR4 RAM, and an NVIDIA Tesla A100 SXM4 GPU with 80 GB memory.

Reference
[4] Clément Vignac et al. “DiGress: Discrete Denoising Diffusion for Graph Generation.” ICLR, 2023.
[5] Yiming Qin et al. “DeFoG: Discrete Flow Matching for Graph Generation.” ICML, 2025.

Response to Question #3

Question #3: Why was G2PT evaluated only for generation? Can its predictive performance be reported on the same corpora?

A: Thank you for pointing out that G2PT also supports molecular property prediction. We used G2PT as a generation reference. A same-corpus comparison of its molecular property-prediction pathway on BACE and BBBP remains open. Node classification and relation-query prediction would additionally require task read-outs and a specified adaptation protocol. §5.1 now distinguishes G2PT's published capabilities [2] from the scope of our evaluation.


Changes in the revision (shown in blue):

  • Abstract and Sections 1, 2, and 5.1. We qualify the novelty claim, describe G2PT's predictive capability, and fix the LGD citation.
  • Section 4.1 and Appendices B and C. We describe the domain-specific RVQ codebooks and specify which target features enter tokenizer construction.
  • Section 5.2 and Appendix J. We report the five-seed GCN probe, actual majority rates, GFT diagnostic, generation components, and the few-shot control.

Conclusion

Thank you again for drawing attention to these details. The revision specifies what is shared across domains and what information enters the tokenizer. The GCN probe and the source-pretrained comprehension control clarify which conclusions the empirical results support. We hope these additions make the evaluation and its limitations easier to assess.

AuthorsOpenReview ↗

Friendly Reminder

Dear Reviewer EizH,

This is a friendly reminder that we greatly value your input on our rebuttal. Your comments helped us clarify how UniCG relates to prior work and sharpen the scope of our contribution.

As the discussion period closes on November 18 (AoE), we would be grateful for any feedback you might have on our rebuttal. If any points remain open or would benefit from further explanation, we would be glad to address them during the discussion.

Thank you again for your time and valuable insights, and we hope our responses can address all your concerns.

Best regards,
Authors of Submission #10322

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.