acceptodds
Under review as a conference paper at ICLR 2027

Graft: In-Context Classification and Regression at Node, Edge and Graph Level

Abstract

A single pretrained model can now match graph neural networks trained on the same labels, at the node, edge and graph levels and for classes or values. We show this with Graft, a 27M-parameter in-context model and the first to answer all six graph task types. Given labeled examples from a graph, it predicts every query with no gradient step or dataset-specific head. Graft accepts arbitrary feature columns through an exchangeable tokenizer, turns every target (a node, a pair of nodes, a whole graph) into a row over the nodes it spans, and treats classification as a special case of regression. Pretraining combines sixteen families of random graphs with random labeling functions and 59 real datasets. Across 49 dataset–task pairs, frozen Graft matches or beats the better of GCN and GIN trained on the labels in its prompt on 45 of them at a standard label budget per task, including every link-prediction and every regression benchmark, and it has the best average rank among all same-label methods on five of the six task types. The gap is widest in regression, where networks trained on few labels often fall below the mean predictor. Graft stays stable on prompts sixteen times longer than any it saw in pretraining. It also generalizes beyond its pretraining data: on dataset families withheld entirely, classification is nearly unchanged from the model that saw them, and fine-tuning on the dataset's labels recovers much of the regression loss.

Primary area:
learning on graphs and other geometries & topologies
Keywords:
in-context learning, graph foundation models, graph neural networks, prior-data fitted networks

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_beHJOpenReview ↗
Rating
2
Confidence
1
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces Graft, a pretrained in-context graph model for node-, edge-, and graph-level classification and regression. Given labeled examples from a graph, Graft predicts labels for query nodes, node pairs, or whole graphs without gradient updates or dataset-specific prediction heads. The model uses an exchangeable tokenizer for arbitrary node feature columns, a unified row-based representation across prediction levels, and an in-context head that combines closed-form estimators, label propagation, and a transformer. The authors evaluate Graft on 49 dataset-task pairs and compare it with GCN/GIN models trained on the same label budgets, as well as other graph and tabular foundation models.

Critical strengths
  1. This paper addresses an important practical problem in graph learning: conventional GNNs usually require dataset-specific training, task-specific output heads, and hyperparameter tuning whenever the feature schema, prediction level, or label type changes.

  2. This paper proposes a genuinely unified formulation in which one frozen model takes labeled examples in context and handles node-, edge-, and graph-level prediction, for both classification and regression, without gradient updates or a dataset-specific prediction head.

  3. Its experimental evaluation is broad and mostly compelling: it covers 49 dataset-task pairs across citation, co-purchase, web, social, road, rating, molecular, and image-derived graphs, and reports that Graft matches or outperforms same-label GCN/GIN baselines on 45 of them, especially for link prediction and few-shot regression.

Critical weaknesses
  1. The primary evaluation does not provide a fully clean test of cross-dataset generalization, because Graft may have seen other labeled rows from the same evaluated dataset during pretraining, whereas the GNN baseline is trained only on the labels in the prompt. The family-held-out experiments partially address this issue, but regression performance drops more substantially in that stricter setting.

  2. The “unified” design still relies on several ad-hoc, manually specified task conventions. Node pairs are encoded using pooled and differenced node embeddings. Graph-level targets are represented with fixed pooling statistics, including the mean, standard deviation, maximum, and graph size. Multiclass labels are decomposed into one-vs-rest tasks. Outputs are represented using 64 fixed quantile bins with a learned residual, and wide node features are compressed to 32 PCA dimensions.

  3. The framework remains limited to scalar or categorical supervised prediction. It does not directly support edge features, graph generation, graph editing, clustering, graph matching, list-wise ranking, sequence generation, or decision-making tasks. Can the proposed method be effectively extended to graph tasks beyond classification and regression?

Questions for authors

See Weakness.

1 comment

We thank the reviewer for the comments on the practical value of one frozen model across prediction levels, and on the scope of its transfer.

Weakness 1 (pretraining exposure and regression under family holdout). The cleanest test of cross-dataset generalization is now the paper's primary result: pretrained on synthetic graphs only, without ever seeing a real graph, Graft matches or exceeds a GCN trained on the same labels on 28 of 49 real dataset-task pairs, without a gradient step. (GCN is the GNN type with the best mean validation rank, chosen without test labels; every type is reported in the revised appendix.) Synthetic-only transfer includes 13 of 13 link-prediction graphs, 5 of 5 graph-regression sets and 4 of 6 TU/OGB graph-classification sets. Revised Figure 3 compares all regimes against the same GCN on the same test rows: Graft+real (family held out), which adds real data from other families, reaches 36 of 49, and Graft+real, which also sees the evaluated training splits, 46 of 49. The latter comparison measures adaptation to datasets represented in pretraining.

Weakness 1, continued (the regression drop). Regression benefits substantially from real data, but transfer differs by prediction level. Synthetic-only graph regression reaches mean skill .110 against −.146-.146 for the same-label GCN and retains 5 of 5 against the tuned GCN. Absolute skill is modest. Family-held-out models reach 5 of 5 graph-, 3 of 6 node- and 0 of 3 edge-regression sets against the same-label GCN, compared with 5 of 5, 6 of 6 and 3 of 3 when evaluated training splits are included. The appendix reports family-held-out results at every budget over three pretraining seeds. These results support useful graph-regression transfer while showing that node and edge regression depend more on real-data coverage.

Weakness 2 (fixed conventions within the unified interface). "Unified" refers to one set of pretrained weights and one labeled-context interface across node, pair and graph targets, for classification and regression. Endpoint combinations turn a pair into a row; mean, standard deviation, maximum and graph size turn a graph into a row. One-vs-rest passes express categorical labels through scalar targets, while 64 quantile bins and a learned residual provide a common output representation. The 32-column PCA projection controls feature width before tokenization. These are fixed design choices with learned projections: pair symmetry and permutation-invariant pooling respect the predicted object, and scalar targets let the head share a prediction rule. The same network processes these rows without fitting a dataset-specific head or changing its weights. Unification does not require every input adapter to be learned. For the camera-ready, we will collect these choices and their equations in the restructured method section and redraw Figure 1 to make their roles explicit.

Weakness 3 (extension beyond scalar and categorical prediction). Graft's evaluated interface covers supervised scalar and categorical targets. It does not currently consume edge features, and edge classification is evaluated as link prediction rather than relation-type classification. Pair scores can rank candidate links, as the added ranking evaluation demonstrates, but this is not a list-wise prediction interface. Graph generation, editing, clustering, matching, sequence generation and decision-making are outside the evaluated scope. These extensions are promising directions for future work; each needs new inputs, structured outputs or task objectives beyond the interface evaluated here.

Reviewer_pyQwOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

The paper proposes Graft, a graph foundation model that handles node-, edge-, and graph-level classification and regression tasks with arbitrary features. Graft is pretrained on a mix of synthetic and real-world data to make predictions via in-context learning. It matches or outperforms GNNs on 45 out of 49 dataset-task pairs considered in this study.

Critical strengths

Graft unifies all six task types in a principled way.

Critical weaknesses
  1. The experimental results and setup raise significant soundness concerns.

First, GNNs receive no hyperparameter tuning, which is crucial for their strong performance. Moreover, PCA with 32 components is used for GNNs, which might hinder their performance. Since GNNs are unlikely to reach the memory limit even without PCA, it would be fairer to evaluate them without PCA. Inspecting, for example, Table 9 (full results for node regression) with K=allK=\mathrm{all} reveals poor GNN performance compared to the original GraphLand results, including negative R2R^2 values on certain datasets.

Second, the same Table 9 shows that GraphPFN outperforms Graft on 6/6 node regression tasks with K=allK=\mathrm{all}, and the gap is more than 10 percentage points in 4/6 cases. Moreover, Table 6 shows that GraphPFN outperforms Graft on 10/13 node classification datasets with K=allK=\mathrm{all}. These results raise concerns about Graft's performance compared to node-level graph foundation models.

Third, as the authors already mention, the main experimental results (Section 4.2, Appendix E) are not representative of graph foundation model evaluation, since during pretraining Graft has seen the datasets used in evaluation, even though it has not seen the test targets. Thus, only Section 4.3 can provide fair comparisons, but it is limited to comparisons of different Graft versions. Therefore, to the best of my knowledge, it is difficult or impossible to obtain results of a fair comparison of Graft against other models.

  1. The paper is hard to follow.

The paper often contains excessive comments and clarifications that make reading harder, while omitting proper descriptions of important or non-trivial details. For example, the main text specifies that outer quantile buckets are half-open (L251), but provides little description of how per-column identity tokens, random identifiers, or closed-form estimators incorporated into the model work.

Sometimes, information is present but is hard to find. For example, if the reader wants to understand how the graph is used in the model (in particular, in the encoder), Figure 1 (method overview) provides no such information. This information is hidden in the middle of a paragraph and is not highlighted. I believe that for a graph foundation model, it is important to highlight how the graph is used.

That said, I acknowledge that this point can be highly subjective and that my examples may not be representative.

Questions for authors
  1. Could you compare your model against GNNs without PCA and with proper hyperparameter tuning?
  2. Could you provide a stronger comparison of leave-dataset-family-out Graft models (or Graft pretrained entirely on synthetic data) against other models?

I also strongly encourage the authors to double-check the contents of the paper. In particular, Table 5 contains missing values (e.g., the metric for tolokers-2) and incorrect entries (e.g., city-reviews is marked as a node regression dataset, while it is actually a node classification dataset).

1 comment

We thank the reviewer for identifying the raw-feature control, the node-level comparison with GraphPFN, and the missing explanation of how graph structure enters the model.

Weakness 1, first point (GNN tuning, PCA and weak all-label results). Revised Table 2 includes a tuned-GCN column and significance marks. GCN is the comparator because it ranks best on validation across our cells, with no test labels involved; choosing GIN would raise Graft's count from 46 to 47 of 49, and the revised appendix reports GCN, GIN, SAGE and GAT per dataset. The search includes raw features without PCA; its protocol is given under Q1. The original all-label GNN columns use a fixed configuration with two message-passing layers of width 64. You are right that they are weak next to the GraphLand results, including negative R2R^2 on some sets: one untuned configuration understates what a tuned GNN reaches with every label. Our claim is about the few-label regime, and there the GCN is tuned on exactly the labels Graft reads, with raw features available (Q1 below).

Weakness 1, second point (GraphPFN's node-level advantage). GraphPFN is stronger with all labels, leading on 6 of 6 node-regression sets and 10 of 13 node-classification sets. The paper now states this explicitly. Graft's contribution is the breadth of one frozen predictor across node, edge and graph targets, for both classification and regression; it does not establish the best node-level performance.

Weakness 1, third point (comparison without evaluation-data pretraining). The revised paper now leads with exactly this comparison. Pretrained on synthetic graphs alone, without ever seeing a real graph, Graft matches or exceeds the same-label GCN on 28 of 49 real dataset-task pairs without a gradient step. This includes 13 of 13 link-prediction graphs, 5 of 5 graph-regression sets and 4 of 6 TU/OGB graph-classification sets. Revised Figure 3 compares all regimes per dataset against the same GCN: Graft+real (family held out) reaches 36 of 49 and Graft+real, which also sees the evaluated training splits, 46 of 49. Synthetic-only pretraining removes real-graph exposure during pretraining; recipe selection still used the reported test splits.

Weakness 2 (method presentation, graph use and Figure 1). The revised method section highlights that the graph enters through a log-degree embedding before the encoder and an additive neighbourhood term in every encoder layer, alongside global linear attention. That term combines the neighbour mean with a degree-scaled copy; graph rows also receive log graph size. The head uses propagated features in ridge regression and graph-based label propagation for node prompts. Random column codes identify feature columns within an input and are bound to their values; random node identifiers break structural symmetries. The closed-form terms supply prototype predictions, ridge predictions and propagated labels to the head. For the camera-ready, we will restructure the method around these components and their equations, move implementation asides out of the main explanation, and redraw Figure 1 to show the graph's path through the model.

Q1 (properly tuned GNNs without PCA). The search covers 26 configurations across GCN, GIN, SAGE and GAT, varying depth, width, learning rate, dropout, weight decay and features: raw without PCA, PCA, or both. Selection uses only prompt labels, with a validation split for classification and inner cross-validation for regression. Against the tuned GCN, synthetic-only Graft reaches 24 of 49, retaining 11 of 13 link, 5 of 5 graph-regression and 4 of 6 TU/OGB graph-classification sets. Graft+real, whose pretraining also includes the evaluated training splits, reaches 38 of 49. PCA uses test-node features without labels, a transductive choice. Refitting Graft's projection on non-test nodes changes accuracy by −0.005-0.005 to +0.002 at 20 labels per class and by at most 0.013 with all labels, within seed variation.

Q2 (stronger comparisons with held-out-family or synthetic-only models). The appendix reports family-held-out models at every budget and against node-level in-context baselines on the same context draws. At 20 labels per class, they match or exceed GILT on 8 of 13 graphs, GraphAny on 5 of 13 and NodePFN on 4 of 13. At 100 labels for node regression, the counts are 3 of 6 against TabPFN v2, 1 of 6 against G2T-FM and 0 of 6 against GraphPFN. These comparisons exclude the evaluated family from pretraining and distinguish broad task coverage from node-level strength.

Table 5 (missing metrics and incorrect task entries). Thank you for catching these; entries are fixed in the revised Table 5.

Reviewer_qb7oOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes Graft, a single pretrained in-context model (27M parameters) that solves node-, edge-, and graph-level classification and regression tasks without gradient updates or dataset-specific heads. Graft uses a tokenizer that accepts arbitrary feature columns and represents each target (a node, a node pair, or a whole graph) as a single row. Its in-context head combines closed-form estimators (ridge regression, class prototypes, and label propagation) with a transformer, and treats classification as a special case of regression. The model is pretrained on 16 synthetic graph families together with 59 real datasets. The authors report that it matches or outperforms GCN/GIN trained on the same labels on 45 of 49 dataset–task pairs.

Critical strengths
  1. Handling classification and regression at the node, edge, and graph levels in context with a single set of weights is a meaningful contribution. In particular, the design is simple: each target is constructed as a row over the nodes it spans, and classification is treated as a special case of regression. Yet, as shown in Table 1, this design extends to edge- and graph-level regression, which existing in-context graph models do not support.
Critical weaknesses
  1. In addition to synthetic graphs, the paper uses the evaluation datasets themselves for pretraining, holding out only their test splits (Section 3.5, Table 5). However, in the main results (Table 2, Figure 3), GCN and GIN are trained on only K labeled samples, whereas the entire training split of each dataset was available to Graft during pretraining. I therefore do not think this can be regarded as a fair comparison. A comparison with matched label budgets is provided in the “all” column of Table 2 and in Appendix E. In this setting, however, Graft matches or outperforms the GNNs on only 31 of 49 dataset–task pairs, and on only 18 of 36 when link prediction is excluded. Consequently, the main results do not support the paper’s claim that Graft “matches or beats a GNN trained on the same labels on 45 of 49 dataset–task pairs.”
  2. For link prediction, apart from Cora, CiteSeer, and PubMed, none of the standard benchmarks with node features (e.g., ogbl-collab, ogbl-citation2) are used. In addition, the chosen metric (ROC-AUC against uniformly sampled negatives) tends to inflate performance. Indeed, the results in Appendix E appear saturated: on Minesweeper, a synthetic grid graph, AnyGraph and OpenGraph achieve an AUC of 0.999 (Table 7). Link prediction is inherently a ranking problem, in which true links must be ranked above a much larger number of negative node pairs. Because ROC-AUC averages over uniformly sampled negatives, most of which are easy, it saturates quickly. Ranking metrics such as Hits@K and MRR, which focus on how positives compare with the highest-scoring negatives, are therefore the standard choice. Furthermore, apart from GCN and GIN, the paper does not include supervised state-of-the-art models specialized for each task as baselines. For link prediction in particular, a wide range of established baselines exists, from heuristics such as CN, AA, and RA to GNN-based models such as SEAL, Neo-GNN, BUDDY, and MPLP, yet none of them are included. Without these comparisons, it is difficult to assess how competitive Graft actually is.
Questions for authors
  1. Could you update Figure 3, Table 2, and Table 3 so that the baselines are also trained on the entire training split (or, for the in-context baselines, given the entire training split in the prompt)?
  2. Could you evaluate Graft and the baselines on standard link prediction benchmarks such as ogbl-collab and ogbl-citation2, and report Hits@K?
  3. Could you re-report Figure 3, Table 2, and Table 3 using the fold models from Section 4.3, i.e., models whose pretraining excludes the family of the evaluated dataset?
  4. For link prediction, could you include heuristics such as CN, AA, and RA, as well as link prediction-specific models such as SEAL, BUDDY, and MPLP, as baselines under both the same label budget and full training?
  5. Could you share the results of a model pretrained only on the synthetic prior, without any real datasets? With the current results, it is difficult to determine whether the performance stems from the synthetic prior or from the real data, which includes the evaluation datasets.
2 comments

We thank the reviewer for separating pretraining exposure from label-budget fairness and for identifying the ranking tests and specialist baselines needed to interpret link prediction.

Weakness 1 (evaluation datasets in pretraining and same-label fairness). Revised Figure 3 separates Graft (synthetic-only pretraining), Graft+real (family held out) and Graft+real (real data including the evaluated training splits), using the same comparator and test rows, at each task's headline label budget; a new appendix figure repeats the comparison at every label budget of the grid. The last regime has unequal pretraining exposure; its few-label comparison measures adaptation to datasets represented in pretraining. We compare against GCN because it has the best mean validation rank across our cells, a rule that uses no test labels; GIN would have been the easier opponent (Graft is at or above it on 47 of 49 headline cells, against 46 for GCN), and every GNN type and link heuristic is reported per dataset in the revised appendix. With all training labels in both the prompt and the GCN's training set, Graft+real reaches 34 of 49, so the few-label result is not a claim of superiority at every budget.

Weakness 2 (uniform-negative AUC and missing link baselines). The revised appendix adds HeaRT MRR, ogbl-collab Hits@50 and individual link heuristics. The base head trails heuristics under hard negatives. A separate structural readout fits closed-form coefficients on the prompt labels, combining path-count features with Graft's score without updating the pretrained network. Its ranking gain is largely structural: with real-data pretraining, path counts alone reach HeaRT MRR 0.277 versus 0.281 with Graft's score. We distinguish this combined readout's ranking performance from the base model's transfer in AUC.

Q1 (using the entire training split). The full-split GCN comparison is reported separately from the few-label comparison. With all training labels also in Graft's prompt, the counts are 15 of 49 for synthetic-only, 30 of 49 for family-held-out and 34 of 49 for evaluated-training-split pretraining. Revised Table 2 adds the tuned-GCN column; the appendix gives full-budget results. This update does not provide a new full-split comparison for every in-context baseline.

Q2 (ogbl-collab, ogbl-citation2 and Hits@K). On ogbl-collab, synthetic-only Graft with the prompt-fitted structural readout reaches Hits@50 0.642, equal to the readout with real-data pretraining. We use the official evaluator and shared negatives, include validation edges in the message-passing graph as permitted, and remove the queried edge from path-count features. This result belongs to the combined readout. We have no ogbl-citation2 evaluation.

Q3 (re-reporting with the fold models). Revised Figure 3 and the appendix compare family-held-out models against the same GCN at every budget over three pretraining seeds: 36 of 49 headline cells. Across budgets, they reach 52 of 52 link cells and 15 of 18 TU/OGB graph-classification cells. The Table 3 node-level comparison is also repeated on the same context draws: at 20 labels per class, the fold models match or exceed GILT on 8 of 13 graphs, GraphAny on 5 of 13 and NodePFN on 4 of 13. At 100 labels for node regression, the counts are 3 of 6 against TabPFN v2, 1 of 6 against G2T-FM and 0 of 6 against GraphPFN. The fold comparison does not include AnyGraph and OpenGraph at link level.

Q4 (heuristics and specialists at matched and full budgets). On ogbl-collab, CN/AA/RA reach Hits@50 0.619/0.644/0.640, and Katz reaches 0.603; these heuristics require no target-label training. BUDDY, using the authors' code, reaches 0.138 with 128 positive and 128 negative pairs, matching Graft's prompt budget, and 0.659 with the full training split. Thus the readout is competitive with the heuristics, while full-split BUDDY is stronger. Published full-split values for SEAL, Neo-GNN and MPLP are 0.647, 0.661 and 0.671; these are literature references, not our matched-budget results. We have no matched-budget SEAL/MPLP comparison.

Q5 (synthetic-only pretraining). This is now the primary result of the revised paper (abstract and Figure 3). With no real graph ever seen during pretraining, Graft matches or exceeds the same-label GCN on 28 of 49 real dataset-task pairs without a gradient step: 13 of 13 link-prediction graphs, 5 of 5 graph-regression sets and 4 of 6 TU/OGB graph-classification sets. Node classification reaches 3 of 13 and edge regression 0 of 3. Synthetic-only link prediction has mean AUC 0.919; adding the structural readout gives HeaRT MRR 0.279 against Katz's 0.253. These results establish transfer from synthetic structure while showing where real pretraining data helps.

We thank the authors for the rebuttal. However, my concerns have not been resolved.

The GCN in the main comparison is not designed for the few-label regime, so outperforming it provides only weak evidence of few-label performance. Moreover, against comparators that target the few-label regime, Graft’s advantage is not consistent. A comparison with link prediction foundation models such as UniLP [1] is also needed.

When the GNN is trained on all training labels, Graft+real, which saw the training splits during pretraining, matches or outperforms the tuned GCN on only 21 of 49 dataset–task pairs. Even in the few-shot setting, which favors Graft, synthetic-only Graft matches or outperforms the untuned GCN on only 15 of 36 dataset–task pairs when link prediction is excluded. For node classification, it does so on only 3 of 13 datasets, and on only 1 of 13 against the tuned GCN.

Although I appreciate the additional experiments and the move toward narrower claims, I will maintain my score for these reasons.

[1] Kaiwen Dong, Haitao Mao, Zhichun Guo, and Nitesh V. Chawla. Universal Link Predictor by In-Context Learning on Graphs. Transactions on Machine Learning Research, 2025. https://openreview.net/forum?id=EYpqmoejB8

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.