- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
The paper proposes the Universal Classifier (UC), a pretrained graph model that handles graphs with any number of features and classes. Every task is treated as a comparison between labeled items, called anchors, and unlabeled items, called queries. Each feature and each label entry is mapped to a -dimensional vector, so a node becomes an matrix, where is the number of node features and is the number of classes. A GNN encoder and a transformer process these matrices, a similarity score is computed for each query-anchor pair, and the scores are averaged per class to obtain class logits. Regression is turned into classification over soft bins, link prediction compares a source node with its neighbors and with random nodes, and graph-level tasks use mean-pooled graph representations. One checkpoint, pretrained on an undisclosed mixture of node- and edge-level datasets, is evaluated without training on the target graph (called "zero-shot") and after fine-tuning. The evaluation covers node classification, node regression, link prediction, and graph regression, though the latter only in a single domain (electronic circuits). The paper reports that zero-shot UC often outperforms supervised GNNs trained on the target graph.
Critical strengths
The query-anchor formulation is simple and general. It handles arbitrary feature and class counts, and it covers node, link, and graph tasks with one similarity-based objective. The evaluation uses recent and diverse datasets.
Critical weaknesses
1. The results cannot be verified.
- The authors release neither the code, the pretraining data, nor the parameter count (lines 305-307 and 500-505). As a result, the key condition that "no evaluation dataset appears in the pre-training mixture" (lines 308-309) cannot be checked. The statement that other pretraining mixtures give similar results (lines 501-505) comes without any evidence.
- Withholding the parameter count is also hard to understand, since Appendix F describes a very small model: hidden size 16 in the GNN encoder and 32 in the transformer.
- No result for UC comes with a standard deviation. The paper explains that retraining the model is "prohibitively expensive" (lines 312-313), but Table 1 reports five ablated variants, and each of them is a model retrained from scratch (lines 423-424). For fine-tuning, the paper only states that seed randomness "stays below 0.1% in most cases" (line 314).
- Table 1 calls a 2-layer transformer "detrimental" (lines 427-428), while Appendix F states that the model has 2 transformer layers (line 915), so it is unclear which configuration produced the reported results.
- The description of the transformer misrepresents the cited models. The paper states that only queries are updated by cross-entity attention, "as is standard in tabular foundation models" (lines 209-211), and that this "follows the same convention" as TabPFN and TabICL (lines 811-814). However, in these models the labeled rows attend to each other, and only their attention to the unlabeled rows is removed. In Algorithm 1 (steps 13-18), each anchor attends only to its own features and labels, so it never sees the other anchors. Either the description is wrong, or the model differs from the cited ones in a way the paper does not discuss.
2. The baselines are weaker than published results, and several claims do not match the tables.
- In Table 4, the supervised GNN results (43.93% accuracy on roman-empire, 47.18% on amazon-ratings) are the GAT results from the appendix of the GraphAny paper, which uses the same split and metric as the paper that introduced these datasets. In the same GraphAny table, even an MLP reaches 65.80% on roman-empire, and the dataset paper reports up to 88.75% (GAT-sep) and 53.63% (GraphSAGE). Against these results, zero-shot UC (78.61% and 52.83%) is worse than the best GNNs, so the claim that UC "in most cases" surpasses supervised GNNs (lines 366-367) is not supported.
- For node regression, the paper claims that UC is "matching or exceeding the transfer performance of specialized tabular foundation models" (line 377). The only foundation model in this comparison is GraphPFN (Figure 4, Table 6). However, in all three cases where GraphPFN is reported, zero-shot UC is worse: 60.54 vs 63.55 on city-roads-M, 66.09 vs 73.27 on twitch-views, and 50.85 vs 61.77 on artnet-views. Even the fine-tuned UC stays below zero-shot GraphPFN.
- For graph regression, zero-shot UC has a relative squared error above 1.0 on three of six targets in Table 8, which, by the paper's own definition (line 386), is worse than always predicting the training mean. Nevertheless, the abstract claims "strong zero-shot transfer performance across node-, link-, and graph-level tasks". Also, graph classification is described in Figure 1 and Section 2.5 but never evaluated.
3. The zero-shot results are hard to believe without the evaluation code.
On the RL split of GraphLand, zero-shot UC is far above the best published results. For example, in the latest version of the GraphPFN paper, fine-tuned GraphPFN outperforms all compared methods, including GNNs and gradient boosting, on every GraphLand dataset evaluated there with the RL split. Yet zero-shot UC reaches 65.46 vs 53.49 average precision on artnet-exp, 86.53 vs 80.90 on city-reviews, and 67.18 vs 62.80 on tolokers-2. Gains of this size from a small model that never saw these graphs need an explanation.
Until the authors release the code that applies the model to the evaluation datasets, I find it hard to trust these results.
Questions for authors
- I understand that organizational policy limits what can be released. Could you release only the evaluation code, without the weights and the pretraining code? This would allow checking how the model is applied to the evaluation datasets.
- Why are the supervised GNN results in Table 4 taken from the GraphAny paper, although the dataset paper reports much higher results under the same split?
- Do anchors attend to other anchors in cross-entity attention? If not, how does the model compare with a variant where they do, as in TabPFN and TabICL?
- How many anchors are used at test time on each dataset, and how are they chosen when the training split is larger than the anchor budget?
13 comments
Dear Reviewer, as the discussion period is closing soon, we would highly appreciate your response so we have a chance to address any remaining issues. We look forward to hearing from you. Thank you!
Thank you for the detailed replies and additional experiments.
- W1 is mostly addressed.
- Q2, Q3 and Q4 are addressed.
- W2 is partly addressed. Zero-shot UC still loses to several supervised GNNs, so please adjust the claims about outperforming them.
- W3 and Q1 remain unresolved. The evaluation code has not been provided, and I explain my specific concern below.
1. What changed when you "improved the model"?
In R1, you write "improved the model" and report two additional training seeds and averaging predictions over ten forward passes. The zero-shot regression results on the RL split improved substantially: increased from 60.54 to 64.91 on city-roads-M, from 66.09 to 75.42 on twitch-views, and from 50.85 to 62.08 on artnet-views. What exactly changed in the architecture, training, or evaluation? How much of this improvement comes from averaging predictions, and how much from the model changes? Did the ablations in R5 use the improved model?
2. I am concerned that labels of validation or test nodes may reach the model through neighboring nodes.
After seeing these strong results, thinking about how they could have been achieved, and reading the descriptions and Algorithm 1, I am concerned that some labels may not be masked correctly. This concern motivated my request for evaluation code.
Section 2.3 describes GNN message passing over the union of sampled neighborhoods of anchors and queries, using both feature and label representations. These neighborhoods can include validation or test nodes that are neither anchors nor current queries. Algorithm 1, step 3, specifies true labels for anchors and zeros for queries, but does not specify labels for these additional nodes. Steps 7–9 then lift and propagate labels. If validation and test neighbors keep their true labels, those labels could reach queries through message passing even when the current queries' own labels are masked.
You say that neighboring labels are "known from the training set." Could you therefore confirm that, in the evaluation used for your main results, all validation and test labels are masked before they are encoded and passed through the GNN, including labels of neighboring nodes that are neither queries nor anchors in the current batch? Please explain how the code ensures this. If only training labels are available throughout the sampled graph, the leakage described above cannot occur.
3. Which results can the community reproduce?
While I admit that the code release is not mandatory, I also note that the ICLR Code of Ethics says:
Methods and results should be presented in a way that is transparent and reproducible.
I believe that the model itself could be implemented from the paper once the descriptions above are corrected. However, the paper does not identify the pretraining datasets, so the community cannot repeat pretraining and reproduce the evaluation results from scratch.
An actionable step would be to pretrain UC on a publicly available collection of datasets, a synthetic prior, or a combination of the two, and report the evaluation results. If UC achieves strong performance in this setting too, this will make the paper more reproducible and verifiable.
We thank the reviewer for the constructive comments and would like to address the raised concerns:
Please find a discussion on data mixtures, parameter count and training flops in R5 of the General Response.
We understand why withholding code limits verification, and would like to point out that under ICLR 2027 Author Guidelines, code submission is encouraged rather than mandatory, and the ICLR Code of Ethics obligates authors to honor intellectual-property and confidentiality agreements alongside transparent paper and appendix documentation.
Still, we do not ask you to accept surprising results on faith. In the General Response to your critique, we added checkpoints trained with more random seeds, commented on graph regression performance, expanded evaluation to 7 new graph-level tasks, fixed baseline and manuscript discrepancy, and verified via label permutation that predictions depend causally on anchor labels.
Thank you for catching this. It’s a typo and we ablated a 1-layer transformer encoder there, will be fixed.
Thank you for catching this, the description is wrong. In UC, anchors attend to other anchors - the intra-entity block in Figure 2 does include anchor-to-anchor attention, we will correct Section 2.3, Figure 2, and Algorithm 1.
We also tested removing anchor-to-anchor attention in R5 of the General Response: it improves classification and regression yet hurts link prediction and circuit RSE , so we retain it for balanced multi-task stability.
We hypothesize the reason could be in a different ICL mechanism - tabular models pool rows to a fixed number of latents while UC is able to process each data point independently (and at scale). To test whether gains reflect genuine in-context graph inference rather than feature memorization, we ran inference interventions isolating the GNN propagation and test-time anchor labels in R5 of the General Response and find
By default we use all anchors, and UC can process arbitrarily large labeled datasets in contrast to GraphPFN. If it happens that the training split is larger than the anchor batch size, we randomly chunk the anchors and score the batches sequentially. We found a sufficiently large anchor batch size (1024+) reduces the sampling noise in anchor-to-anchor attention. Besides that, Figure 6 in the manuscript shows that it is possible to randomly subsample 1 to 5% of anchors and retain 80+% of the full performance.
We hope our responses address your concerns and appreciate your reconsideration of the evaluation and increasing the score.