acceptodds
Under review as a conference paper at ICLR 2027

AnyGRAM: Rotation-Invariant Global Operators for Fully-Inductive Node Classification

Abstract

A fully-inductive node classifier is trained on some graphs and then applied, frozen and without gradient steps, to a new graph whose features, label space, and structure it did not see during training. Existing models combine a fixed bank of local propagation operators, a closed-form least-squares readout on the labeled support nodes, and a head, learned on the source graphs, that weighs the operators. Two parts of this design remain open: the operators see only local neighborhoods, and the choice of how to read the bank, which depends on the new graph, is learned in advance on other graphs. In this paper, we ask which parts of a fully-inductive classifier should be learned across graphs and which should be decided on the target graph from its support labels. We introduce , which learns encoders whose inputs are invariant to rotations of the random feature projection and uses them to add global covariance operators to the local bank, applied in factored form without forming a node-by-node matrix. The ridge regularization of each branch and the branch used on each graph are selected by cross-validation on the support set of the target graph. Trained on a single source graph and applied frozen to 26 target graphs, reaches average accuracy, percentage points above the strongest zero-shot baseline and above end-to-end models trained on each target graph. Our empirical results show that the covariance branch family and the decisions made on the target graph contribute to this gain, and accuracy changes little with the choice or number of source graphs.

Primary area:
learning on graphs and other geometries & topologies
Keywords:
Graph foundation models, fully-inductive node classification, graph neural networks, cross-domain transfer, covariance operators, few-shot learning

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_Wry4OpenReview ↗
Rating
2(+1)
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes AnyGRAM, a fully-inductive node classifier that separates transferable components learned across graphs from target graph-specific decisions based on target-graph support labels. To achieve this, it introduces rotation-invariant encoders with global covariance operators for cross-graph transfer, while making target-specific decisions through GCV-based ridge regularization selection and support-set cross-validation for branch selection. Trained solely on the Cora citation network, AnyGRAM achieves an average accuracy of 71.98% across 26 target graphs from various domain, exceeding the strongest existing fully-inductive baseline by 6.16 percentage points and outperforming GNN baselines trained separately on each target graph.

Critical strengths

S1. This paper tackles the important and challenging problem of fully inductive node classification, a key step toward developing general-purpose graph foundation models.

S2. The conceptual distinction between what should be learned across graphs and what should be learned directly from the target graph's labeled support set is intuitive and well-motivated.

Critical weaknesses

W1. Insufficient motivation and justification for the methodological complexity.

The proposed framework introduces several components without sufficient motivation. In particular, the rationale for selecting and combining the rotation-invariant input features is unclear. The learned fusion module is replaced by support-based branch selection at inference, yet uniform fusion during training achieves comparable performance (Table 2). Similarly, node-level overrides improve accuracy by only 0.1 percentage points. These results raise questions about whether the additional complexity is justified.

W2. Limited evidence for the benefits of global covariance propagation.

The paper presents global covariance propagation as a key contribution to overcoming the limitations of local operators. However, Table 2 shows that removing global covariance propagation reduces accuracy by only 0.3 percentage points, compared to a 2.2-point drop when removing the entire covariance branch family. These results provide limited evidence that global covariance propagation itself is a major contributor to the reported gains.

W3. Incomplete comparisons with recent fully-inductive and graph foundation models.

The experimental evaluation lacks comparisons with recent relevant approaches, including a recent fully-inductive baseline [1], and GFMs [2,3]. While their training and adaptation protocols differ, the paper should provide comparisons under compatible settings or clearly justify why such comparisons are not feasible.

[1] Lee et al., View Space: Learning Representation across Arbitrary Graphs, ICML 2026.

[2] Choi et al., Learning Posterior Predictive Distributions for Node Classification from Synthetic Graph Priors, ICLR 2026.

[3] Eremeev et al., GraphPFN: A Prior-Data Fitted Graph Foundation Model, ICML 2026.

Questions for authors

Q1. Could the authors clarify the motivation behind the specific choice of rotation-invariant inputs?

Q2. Why is the learned attention fusion module necessary during training if it is not used at inference time, especially given that uniform fusion achieves comparable performance?

Q3. How does the overall inference time scale with the number of support nodes and graph size? Could the authors provide runtime comparisons with existing baselines?

Q4. Could the authors provide comparisons with recent fully-inductive models and GFMs [1–3], or clearly justify why such comparisons are not feasible?

[1] Lee et al., View Space: Learning Representation across Arbitrary Graphs, ICML 2026.

[2] Choi et al., Learning Posterior Predictive Distributions for Node Classification from Synthetic Graph Priors, ICLR 2026.

[3] Eremeev et al., GraphPFN: A Prior-Data Fitted Graph Foundation Model, ICML 2026.

7 comments
AuthorsOpenReview ↗

Response to Reviewer Wry4 (Part 1)

Thank you for the detailed and specific review. We are glad that you found that the paper "tackles the important and challenging problem of fully inductive node classification, a key step toward developing general-purpose graph foundation models", and that the "conceptual distinction between what should be learned across graphs and what should be learned directly from the target graph's labeled support set is intuitive and well-motivated".

In short, your review asks whether each component of AnyGRAM is needed, and how AnyGRAM compares with recent fully-inductive models and graph foundation models in accuracy and cost. Following your comments, we:

  1. derived each input group from two requirements of the setting, and retrained AnyGRAM without each group;
  2. stated the two roles of the learned fusion and the role of the override margin, with the simpler configurations;
  3. measured the effect of the global product on each graph where support CV selects covariance branches;
  4. evaluated View Space, NodePFN, and GraphPFN from their released checkpoints, on the splits of Table 1;
  5. measured the inference time and memory of all four models on graphs of different sizes, and the scaling with the support size.

Regarding the motivation of the rotation-invariant inputs (W1, Q1). We appreciate the question on why AnyGRAM uses these inputs. They follow from two requirements of the setting. First, no learned weight may have a shape that depends on the feature dimension dd or the number of classes CC, and thus the input width is fixed. Second, the input must stay the same when the projection is rotated: the projection is redrawn at every forward pass in training and averaged over 10 draws at inference, and an encoder that read Rˉ(p)\bar R^{(p)} would receive its input in a new basis at every draw. Every function of K(p)=Rˉ(p)Rˉ(p)⊤K^{(p)}=\bar R^{(p)}\bar R^{(p)\top} meets the second requirement, and among these functions we use the parts whose width does not depend on the number of nodes: the diagonal of each K(p)K^{(p)}, which gives the propagated norms, and the columns of K(0)K^{(0)} at L=16L=16 landmark nodes. Two groups that do not depend on Ω\Omega complete the input: RWSE and log-degree, which give the identity and adjacency branches node descriptors on every graph, and the entropy, top-1 mass, and margin of the propagated support labels, which summarize the support labels with a width independent of CC and stay the same when the classes are renamed. To address your question directly, we retrained AnyGRAM on Cora without each group, with five seeds (Table 11):

Configuration26 graphsΔ\DeltaSix covariance graphsΔ\Delta
AnyGRAM71.98–64.90–
Without norms and landmark similarities71.81−0.1764.19−0.70
Without RWSE and log-degree71.86−0.1264.13−0.77
Without label-propagation statistics71.89−0.0964.28−0.62

Removing each group lowers the observed mean accuracy by 0.62 to 0.77 points on the six graphs where support CV assigns at least 80% of the query nodes to covariance branches, but all paired 95% confidence intervals for the individual group removals include zero. Removing the whole branch family lowers accuracy by 2.17 points on the 26 graphs (p<0.05p<0.05, Table 2). In the revision, Section 3.1 motivates each group using these two requirements, and Appendix H reports the ablations.

AuthorsOpenReview ↗

Response to Reviewer Wry4 (Part 2)

Regarding the learned fusion and the node-level overrides (W1, Q2). Thank you for asking why the learned fusion is used in training. It has two roles. First, it is the training readout through which the task loss reaches the encoders, and for this role a uniform average gives a similar observed mean: trained with a uniform fusion, AnyGRAM reaches 72.04%, compared with 71.98% (Table 2). Second, it is the learned readout with which the paper tests its division, on exactly the same branch predictions (Table 2):

Readout of the same branch predictionsAccuracy (%)
Hard attention routing (top-1)66.9
Learned soft fusion69.5
Uniform branch average70.0
Per-node selection71.7
Global selection by support CV71.9
Global selection with node-level overrides (AnyGRAM)72.0

In these experiments, support-based branch selection outperforms the source-trained fusion on the same predictions, supporting the claim of Section 4.4. The node-level overrides address the difference between the per-node oracle, at 90.1%, and the per-graph oracle, at 72.7%: different branches succeed in different regions of one graph. Their margin rule is intended to reduce harmful node-level decisions: per-node selection without a margin is 0.2 points below global selection, while the overrides are 0.1 points above it. They also need no additional model evaluation, because they reuse the out-of-fold predictions of support CV. In the revision, Section 3.3 states both roles of the learned fusion, that a uniform training readout reaches a similar observed accuracy, and why the overrides use a margin.

Regarding the contribution of global covariance propagation (W2). Thank you for this question, which helped us locate where the global product contributes. Removing the covariance branches removes their inputs, encoders, and product together, while the second ablation keeps the inputs in FF and removes only the product K(p)VbK^{(p)}V_b. We now report the second ablation on each of the six graphs where support CV assigns at least 80% of the query nodes to covariance branches (Section 4.3), with five seeds (Table 12):

GraphAnyGRAMWithout the global productChange
CiteSeer68.48 ± 3.7360.92 ± 2.19−7.56*
full-DBLP73.84 ± 1.4871.60 ± 2.97−2.23
USA52.04 ± 2.3851.21 ± 1.89−0.83
Tolokers80.03 ± 0.1780.15 ± 0.22+0.12
Brazil65.38 ± 4.7166.15 ± 4.21+0.77
Europe49.63 ± 3.4451.25 ± 4.03+1.62
All 26 graphs (Table 2)71.9871.67−0.31

*Uncorrected paired tt-test across five seeds: CiteSeer, p=0.022p=0.022; full-DBLP, p=0.257p=0.257.

On CiteSeer and full-DBLP, support CV assigns 80% of the query nodes to global branches (Figure 3a), and removing the product lowers accuracy by 7.56 and 2.23 points. On Tolokers, the identity branch receives 89% of the query nodes, and on the airport graphs the selected branches include the identity branch; on these four graphs, removing the product changes accuracy by −0.83 to +1.62 points. On the other 20 graphs, the mean decreases by less than 0.01 points. These results locate the largest observed benefits of the global product on CiteSeer and full-DBLP, but do not uniquely attribute the remaining branch-family gain to individual input groups. The 0.31-point mean decrease over 26 graphs combines the decreases on CiteSeer and full-DBLP, the changes on the other four covariance graphs, and the nearly unchanged mean on the remaining 20. Its paired 95% confidence interval is [−0.68,+0.05][-0.68,+0.05] points. Following your comment, Section 4.3 now reports the effect of the global product on CiteSeer and full-DBLP, and Appendix H gives the values for every graph.

AuthorsOpenReview ↗

Response to Reviewer Wry4 (Part 3)

Regarding comparisons with recent fully-inductive models and graph foundation models (W3, Q4). Thank you for naming these three models. Following your suggestion, we evaluated the released checkpoints of View Space [1] (RGVT with an MLP head), NodePFN [2], and GraphPFN [3] on the splits of Table 1, with the training split as the support set, the same test nodes, and five seeds (Table 7):

ModelPretrainingOn the target graphGraphsIts accuracyAnyGRAM
View SpaceArxivtrains an MLP head; validation labels select the head and the depth2572.29 ± 0.3072.35 ± 0.56
NodePFNsynthetic graph priorsupport labels as context2470.07 ± 0.5272.79 ± 0.57
GraphPFNsynthetic graph prior, LimiX initializationsupport labels as context2673.78 ± 0.4071.98 ± 0.54

Each row uses the graphs that both models support: View Space is pretrained on Arxiv, which we exclude from its comparison, and in the evaluated configuration NodePFN supports up to 20 classes, which excludes Arxiv and full-Cora. NodePFN and GraphPFN run frozen with the support labels as context, as AnyGRAM does, while View Space trains a head on each target graph and selects it with validation labels. AnyGRAM, trained on Cora in 2.3 minutes, is close to View Space in observed mean accuracy (+0.06 points), above NodePFN by 2.72 points, and within 1.80 points of GraphPFN, which is pretrained on synthetic graphs starting from the tabular foundation model LimiX. Combining NodePFN's published accuracies on the 21 overlapping graphs with our reproduced accuracies on Wiki, BlogCatalog, and Roman-empire gives a descriptive hybrid average of 71.07% over 24 graphs. On the 21 overlaps, the published values average 1.14 points above our reproduction. This hybrid calculation is not a matched-protocol evaluation. In the revision, Section 4.2 compares AnyGRAM with these models, Appendix G gives their protocols and per-graph accuracies (Table 8), the abstract and Section 1 state the margin of AnyGRAM over the zero-shot baselines trained on the same source graph, and Section 2 now cites NodePFN [2].

A separate composition control preserves native GraphPFN predictions and adds the ten AnyGRAM branches as candidates for support-based selection. It reaches 74.73 ± 0.21% on the same 26 graphs and five seeds, with no target gradient updates (Appendix G). The standalone comparison above continues to use the submitted AnyGRAM.

Reviewer_Hkr6OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper studies node classification in a fully-inductive setting: the model is trained on source graphs and then applied frozen, without gradient updates. The authors propose that the part learned across graphs must read inputs whose meaning is the same on every graph; they therefore build the cross-graph part on inputs they regard as meaning-stable, and delegate the target-dependent readout regularization and branch selection to the support labels of the target graph. The construction combines local propagation branches, random-projection covariance branches, a ridge regression readout, and cross-validation-based selection on the target graph.

Critical strengths
  • The ablation methodology is sound and the structure is clear. The design partition proposed in the paper is well defined: which structures should be learned across graphs, and which decisions should be made from the support labels of the target graph. This offers a useful methodological perspective for discussing the transfer boundary in fully-inductive node classification.
  • The paper is highly reproducible: the setting of 26 graphs, single-source training and frozen parameters delineates the scope within which the conclusions apply.
Critical weaknesses
  • The design principle is clear, but the concrete incremental contribution is limited. I consider the work a compositional engineering contribution with insufficient methodological novelty: the way the parts are combined does not exhibit anything distinctive, including the treatment of rotation invariance. The current results show that the combination works, but not that the combination itself yields a new estimation principle or a transferable methodological regularity.
  • Even after a basis-invariant aggregation, features from different graphs may still differ in meaning. The "input that means the same on every graph" is therefore a working assumption that requires qualification, and it cannot be derived from rotation invariance alone.
Questions for authors
  • Beyond combining existing operators with a target-graph selection step, what new methodological viewpoint does the paper offer?
  • What applicability requirements does the method impose on its input data? How does rotation invariance support transfer between graphs whose features differ in semantics?
3 comments
AuthorsOpenReview ↗

Response to Reviewer Hkr6 (Part 1)

Thank you for the careful and constructive review. We are glad that you found that the design partition proposed in the paper is "well defined", that it "offers a useful methodological perspective for discussing the transfer boundary in fully-inductive node classification", that the ablation methodology "is sound", and that the paper "is highly reproducible".

In short, your review asks what the paper contributes beyond its components, and what its invariance guarantees for graphs whose features differ in meaning. Following your comments, we:

  1. stated the methodological viewpoint as a principle with two testable consequences, together with the evidence for each;
  2. stated exactly what the invariance guarantees, and how inputs with the same construction support transfer across feature semantics;
  3. stated the requirements of the method on its input data, in a new Appendix I;
  4. qualified the wording of Sections 1 and 3, which now speak of "inputs defined by the same construction on every graph".

Regarding the methodological viewpoint beyond the components (W1, Q1). Thank you for this question on what remains once the components are accounted for. The viewpoint is a division of a fully-inductive classifier: each choice that depends on the target graph is estimated from the support labels of that graph, and what is learned across graphs reads only inputs whose construction is the same on every graph. Its first consequence can be tested at every level at which AnyGRAM makes a choice: an estimate made on the target graph should outperform a choice learned on the source graphs or fixed in advance. Four comparisons show it:

ChoiceLearned or fixed in advanceMade on the target graphSource
Ridge regularizationfixed multiplier: 70.8GCV: 72.0Table 2
Branch, same predictionslearned fusion: 69.5; uniform average: 70.0support CV: 71.9Table 2
Readout of the five branches of GraphAny [1]learned attention of GraphAny: 65.49GCV and support CV: 69.81Tables 1, 2
Branch, for each of four source graphslearned fusion: 68.66 to 70.00support CV: 71.95 to 72.16Table 4

The third row applies the division to the bank of another model, GraphAny, and raises its accuracy by 4.32 points.

Its second consequence is that, once every choice that depends on the target graph is made on that graph, accuracy should depend little on the source. It does: the averages for four source graphs lie within 0.21 points of each other (Table 4), nine source graphs change the one-source average by 0.15 points (Table 5), and with its covariance encoders at random initialization, AnyGRAM reaches 72.15%, compared with 71.98% (Table 13). This control does not establish an accuracy benefit from source optimization of the encoders. The estimators are classical by design: ridge regression solves each candidate fit in closed form, and for fixed branch features one eigendecomposition per fit evaluates all candidate regularization strengths and the analytical leave-one-out predictions. In the revision, Section 1 states the division with these two consequences, and Section 4.4 opens with the four comparisons as instances of the first.

Composition with a modern predictor. We also tested the decision rule beyond the original branch bank: native GraphPFN predictions are added as a candidate alongside the ten AnyGRAM branches, and support CV with the original cosine override selects among them. GraphPFN's native predictor is preserved, with no ridge fit on its logits or hidden states and no target gradient updates. On 26 graphs and five seeds, the composition reaches 74.73 ± 0.21%, versus 73.78 ± 0.40% for native GraphPFN: a paired gain of 0.95 points, with 95% confidence interval [+0.42,+1.48][+0.42,+1.48] across seed-wise benchmark means. The graph-paired interval is [−0.26,+2.16][-0.26,+2.16], so the improvement is not uniform across graphs. This supports complementary predictors and target-side selection beyond the original bank. Appendix G gives the protocol and controls.

AuthorsOpenReview ↗

Response to Reviewer Hkr6 (Part 2)

Regarding what the invariance guarantees and what the method requires (W2, Q2). We appreciate this question, which helped us state the invariance property exactly. Rotation invariance removes the arbitrary basis of the random projection; it does not align the meaning of features across graphs. Precisely:

  1. Replacing Ω\Omega by ΩO\Omega O, for any orthogonal OO, leaves K(p)K^{(p)}, every input in FF, and every branch output exactly unchanged.
  2. The distribution of K(p)K^{(p)} depends on the raw features only through their Gram matrix within the graph, with E[K(p)]=hXˉ(p)Xˉ(p)⊤\mathbb{E}[K^{(p)}]=h\bar X^{(p)}\bar X^{(p)\top}. It is therefore the same when the raw features are permuted, sign-flipped, rotated, or padded with zeros to another dimension, and it changes when individual features are rescaled.
  3. A fresh draw of Ω\Omega varies K(p)K^{(p)} around this expectation, and AnyGRAM averages its predictions over 10 draws.

Transfer between graphs whose features differ in meaning rests on two further parts of the design. First, each input is defined by the same construction on every graph, as a relation between a node and its own graph: the propagated norm of node ii measures how far its propagated features lie from the mean of its graph, a landmark similarity compares node ii with the highest-degree nodes of its graph, and the label-propagation statistics measure how concentrated the support labels are around node ii. These definitions hold whether the features are word indicators, as on Cora, or worker statistics, as on Tolokers. Second, the class readout is fitted on the support labels of each target graph, and support CV decides whether these inputs relate to its labels. Covariance branches trained on Cora improve accuracy on Tolokers by 1.9 points (Figure 3b), and support CV can favor other branches; empirically, adding the covariance branches lowers accuracy by more than 1.2 points on none of the 26 graphs (Section 4.3).

Following your question, Appendix I now states these properties and the requirements of AnyGRAM on its input data:

  1. numerical node features of any dimension, with categorical attributes given as one-hot vectors, and the graph structure; no shared feature space, feature dimension, or label set across graphs;
  2. a labeled support set on the target graph in which every class appears; support CV scores the classes with at least two support nodes, and a class with one support node stays in every fitting fold (Appendix D);
  3. for the covariance branches to help, a relation between the labels of the target graph and its feature geometry, its structure, or its propagated support labels; support CV estimates their usefulness but can make selection errors.

Sections 1 and 3 now say "inputs defined by the same construction on every graph and invariant to the projection basis", and Section 3.1 points to Appendix I.

We sincerely appreciate the thoughtful feedback, which strengthened the statement of our methodological viewpoint and of the exact scope of its invariance. We hope that the two consequences of the division and their evidence, the exact invariance statement, and the requirements in Appendix I address your concerns, and that you will consider revising your score. We are happy to discuss any remaining questions.

References:
[1] Fully-inductive Node Classification on Arbitrary Graphs

Dear Reviewer Hkr6,

Today is the last day of the discussion period, and we have not yet received a reply from you on our responses.

Thank you again for your careful and constructive review. In particular, you asked:

“Beyond combining existing operators with a target-graph selection step, what new methodological viewpoint does the paper offer?”

We therefore followed your guidance and made this viewpoint explicit and directly testable, together with the additional analyses you requested:

  • Target-dependent decisions should be made on the target graph: selecting ridge regularization by GCV improves over a fixed choice by 1.1 points, while support CV improves over the learned source-graph fusion by 2.4 points on exactly the same branch predictions.
  • The principle transfers beyond AnyGRAM: applying GCV and support CV to the original five branches of GraphAny raises its accuracy from 65.49% to 69.81%, a 4.32-point improvement, without changing its branch bank.
  • The same effect holds across source graphs: for each of 4 different source graphs, target-side branch selection improves over learned fusion by approximately 1.9–3.3 points, while final AnyGRAM accuracy varies by only 0.21 points across the sources.
  • Composition with a recent graph foundation model: applying the same target-side selection principle to a bank that also contains native GraphPFN predictions reaches 74.73% vs. 73.78% for GraphPFN, with no target gradient updates.
  • Invariance and applicability: following your second question, we now state exactly what rotation invariance guarantees, what it does not imply about feature semantics, and the requirements on the target data. A new Appendix I makes these assumptions and guarantees explicit.

Following your review, we also revised Sections 1 and 3 to state the methodological contribution as a division between what should be learned across graphs and what should instead be estimated from the labeled support of the target graph, together with the two empirical consequences above.

We believe these additions directly address the methodological and applicability concerns you raised and materially strengthen the paper. We hope you find our responses and the revised paper satisfactory, and that you will consider revising your score.

Thank you again for your time and for the thoughtful review.

Best regards,
The Authors

Reviewer_LkpROpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper addresses the issues of differences in feature space, label space, and topological structure between different graphs in fully inductive node classification, as well as the difficulty of existing methods in fully utilizing global information. To address these problems, a cross-graph node classification framework called AnyGRAM is proposed. The method constructs a global covariance operator via a rotation-invariant feature encoding mechanism and integrates it with traditional local graph propagation operators to form a multi-branch representation learning architecture. Furthermore, by employing generalized cross-validation on a support set from the target graph alongside a branch-selection strategy, the framework adaptively determines ridge regression parameters and selects the optimal prediction branch, all without requiring gradient updates on the target graph. Experimental results demonstrate that this approach achieves strong classification performance in cross-graph transfer settings that necessitate zero gradient updates.

Critical strengths
  1. The research problem holds practical significance. This paper focuses on the problem of inductive node classification, specifically addressing the generalization challenges—such as discrepancies in feature dimensions, label spaces, and structural properties—that arise when transferring models between different graphs. The work offers substantial research value and promising application potential.

  2. The proposed methodology demonstrates a degree of novelty. By integrating rotation-invariant feature encoding, global covariance propagation, and adaptive model selection for the target graph, the paper seeks to enhance model transferability through both cross-graph shared learning and local decision-making on the target graph. The overall research approach is well-conceived and clear.

  3. The experimental evaluation is comprehensive, and the performance is highly competitive. Experiments conducted across various types of graph datasets show an improvement of approximately 6.16 percentage points over the strongest zero-shot transfer baseline, indicating that the framework performs effectively under the adopted evaluation protocols.

Critical weaknesses
  1. There is still room for improvement regarding the technical innovation of the core methodology. AnyGRAM primarily integrates established techniques—such as the local propagation branch from GraphAny, existing randomized projection covariance operators, ridge regression, and generalized cross-validation. Furthermore, the property of rotational invariance stems fundamentally from the fact that covariance matrices remain invariant under orthogonal transformations. The paper needs to more clearly highlight its essential innovations compared to existing methods like ALL-IN and GraphAny.

  2. The practical utility of cross-graph pre-training has not been sufficiently validated. Although the paper emphasizes learning representations that are transferable across graphs, experimental results show that model performance remains largely stable regardless of the source graph used for training; increasing the number of source graphs also fails to yield any significant performance gains.

  3. The actual contribution of the global covariance propagation operator is quite limited. While the paper presents the global covariance operator as a key technical contribution, Table 2 indicates that removing this component results in a decline in average accuracy of only 0.3 percentage points (from 72.0% to 71.7%), with no reported statistical significance. In contrast, removing the entire covariance branch leads to a drop of 2.2 percentage points. This suggests that the observed performance improvements likely stem more from the additional structural encoding and branch representations than from the global covariance propagation itself.

Questions for authors

Please see the weaknesses.

3 comments
AuthorsOpenReview ↗

Response to Reviewer LkpR (Part 1)

Thank you for the detailed and constructive review. We are glad that you found that the research problem "holds practical significance", that the overall research approach is "well-conceived and clear", and that the experimental evaluation "is comprehensive" and "the performance is highly competitive".

In short, your review asks what AnyGRAM adds to GraphAny and ALL-IN, and which of its parts produce its improvement. Following your comments, we:

  1. compared the mechanisms of AnyGRAM, GraphAny, and ALL-IN, with the measured effect of each difference;
  2. evaluated AnyGRAM with its covariance encoders at random initialization, which tests what training on the source graph contributes;
  3. measured the effect of the global product on each graph where support CV selects covariance branches.

Regarding the innovations relative to GraphAny and ALL-IN (W1). Thank you for asking us to state the innovations explicitly. Relative to GraphAny [1], AnyGRAM makes two changes. It adds five covariance branches to the five local branches, and it makes the readout choices on the target graph: the ridge regularization of each branch is selected by GCV and the branch by support CV, where GraphAny combines its branches with an attention head learned on the source graphs. Each change is measured on the 26 graphs, after training on Cora:

ConfigurationAccuracy (%)Source
GraphAny65.49Table 1
The same five branches, with GCV and support CV69.81Table 2
With the five covariance branches (AnyGRAM)71.98Table 1

On the same branch predictions, support CV reaches 71.9%, compared with 69.5% for a learned fusion and 70.0% for a uniform average, and GCV improves on a fixed ridge multiplier by 1.1 points (Table 2).

Relative to ALL-IN [2], the difference is what the learned layers read. ALL-IN passes the projected features Rˉ(p)\bar R^{(p)} to its layers, and their coordinates change with every draw of the projection. In AnyGRAM, no learned weight multiplies a coordinate of Rˉ(p)\bar R^{(p)}. The encoders read entries of K(p)=Rˉ(p)Rˉ(p)⊤K^{(p)}=\bar R^{(p)}\bar R^{(p)\top} and descriptors that do not depend on Ω\Omega, and the global term is an attention whose queries and keys are fixed to Rˉ(p)\bar R^{(p)}, so that the attention matrix is K(p)K^{(p)} and only the values are learned. The paper uses the identity (Rˉ(p)O)(Rˉ(p)O)⊤=K(p)(\bar R^{(p)}O)(\bar R^{(p)}O)^\top=K^{(p)} as a construction rule, not as a result: it determines which global operators a learned fully-inductive model can apply, namely those expressed through K(p)K^{(p)} and evaluated in O(nhh′)O(nhh') time, and which node inputs it can read. In the revision, the contributions in Section 1 now state these three differences explicitly.

AuthorsOpenReview ↗

Response to Reviewer LkpR (Part 2)

Regarding the role of cross-graph training (W2). We appreciate this question on what training on source graphs contributes. To address it directly, we evaluated the same architecture with its covariance encoders left at their random initialization, with the ridge readout, GCV, and support CV unchanged, on the 26 graphs with five seeds (Table 13):

ConfigurationAccuracy (%)
Without covariance branches (Table 2)69.81 ± 0.58
Covariance encoders at random initialization72.15 ± 0.51
Covariance encoders trained on Cora (AnyGRAM, Table 2)71.98 ± 0.54

With trained encoders, the covariance branches improve accuracy by 2.17 points (p<0.05p<0.05, Table 2). With encoders at random initialization, the observed mean is similar: the paired random-minus-trained difference is +0.17 points, with a 95% confidence interval of [−0.13,+0.48][-0.13,+0.48]. These runs support the usefulness of the complete branch construction with the target-side readout, but do not establish an accuracy benefit from source optimization of the encoder weights. This is consistent with the source-insensitivity results: the averages for four source graphs lie within 0.21 points of each other (Table 4), and nine source graphs change the one-source average by 0.15 points on the fixed 18-target panel (Table 5). In practice, training on one source graph takes 2.3 minutes and gives the submitted accuracy. In the revision, Section 4.5 reports this control and its limitation, and Appendix H gives its per-graph results.

Regarding the contribution of global covariance propagation (W3). Thank you for comparing the two ablations, which helped us locate the contribution of the global product. Removing the covariance branches removes their inputs, encoders, and product together, while the second ablation keeps the inputs in FF and removes only the product K(p)VbK^{(p)}V_b. We now report the second ablation on each of the six graphs where support CV assigns at least 80% of the query nodes to covariance branches (Section 4.3), with five seeds (Table 12):

GraphAnyGRAMWithout the global productChange
CiteSeer68.48 ± 3.7360.92 ± 2.19−7.56*
full-DBLP73.84 ± 1.4871.60 ± 2.97−2.23
USA52.04 ± 2.3851.21 ± 1.89−0.83
Tolokers80.03 ± 0.1780.15 ± 0.22+0.12
Brazil65.38 ± 4.7166.15 ± 4.21+0.77
Europe49.63 ± 3.4451.25 ± 4.03+1.62
All 26 graphs (Table 2)71.9871.67−0.31

*Uncorrected paired tt-test across five seeds: CiteSeer, p=0.022p=0.022; full-DBLP, p=0.257p=0.257.

On CiteSeer and full-DBLP, support CV assigns 80% of the query nodes to global branches (Figure 3a), and removing the product lowers accuracy by 7.56 and 2.23 points. On Tolokers, the identity branch receives 89% of the query nodes, and on the airport graphs the selected branches include the identity branch; on these four graphs, removing the product changes accuracy by −0.83 to +1.62 points. On the other 20 graphs, the mean decreases by less than 0.01 points. These results locate the largest observed benefits of the global product on CiteSeer and full-DBLP, but do not uniquely attribute the remaining branch-family gain to individual input groups. The 0.31-point mean decrease over 26 graphs combines the decreases on CiteSeer and full-DBLP, the changes on the other four covariance graphs, and the nearly unchanged mean on the remaining 20. Its paired 95% confidence interval is [−0.68,+0.05][-0.68,+0.05] points. Following your comment, Section 4.3 now reports the effect of the global product on CiteSeer and full-DBLP, and Appendix H gives the values for every graph.

Thank you again for the constructive review, which improved how the paper states the innovations of AnyGRAM and the source of each part of its improvement. We hope that the comparison of mechanisms, the control with randomly initialized encoders, and the per-graph analysis of the global product address your concerns, and that you will consider raising your score. We are happy to discuss any remaining questions.

References:
[1] Fully-inductive Node Classification on Arbitrary Graphs
[2] Bridging Input Feature Spaces Towards Graph Foundation Models

Dear Reviewer LkpR,

Today is the last day of the discussion period. Thank you again for your constructive review and for your positive assessment of the problem, methodology, and experimental evaluation.

In particular, you wrote:

“The paper needs to more clearly highlight its essential innovations compared to existing methods like ALL-IN and GraphAny.”

We therefore followed your guidance and added direct experiments and revisions addressing the three main points you raised:

  • GraphAny and the target-side decisions: keeping the same five GraphAny branches but making ridge regularization and branch selection on the target graph raises accuracy from 65.49% to 69.81%, a 4.32-point improvement. Adding the covariance branches then reaches 71.98%.
  • ALL-IN and the methodological distinction: we revised Section 1 to state explicitly how AnyGRAM differs from both ALL-IN and GraphAny, including the division between what is learned across graphs and what is estimated from the support labels of the target graph.
  • Global covariance operator: on CiteSeer and full-DBLP, where support CV assigns 80% of query nodes to global branches, removing the global product lowers accuracy by 7.56 and 2.23 points, respectively. The 0.31-point average over all 26 graphs therefore combines these larger effects with graphs where the global branch is rarely selected.
  • Role of cross-graph training: following your concern about source-graph dependence, we added the random-initialization and multi-source controls. The results sharpen the interpretation of which parts of the method come from the transferable construction and which are determined on the target graph.

We believe these additions directly address the concerns you raised and substantially clarify both the innovations of AnyGRAM and the source of its empirical gains. We hope you find our responses and the new evidence satisfactory, and that you will consider revising your score.

Thank you again for your time and for the constructive review.

Best regards,
The Authors

Comments on the paper

2 comments
AuthorsOpenReview ↗

General Response

We thank the reviewers for their careful and constructive reviews. We are glad that the reviewers found that the paper "tackles the important and challenging problem of fully inductive node classification, a key step toward developing general-purpose graph foundation models" (Wry4), that the "conceptual distinction between what should be learned across graphs and what should be learned directly from the target graph's labeled support set is intuitive and well-motivated" (Wry4), that the design partition "offers a useful methodological perspective" (Hkr6), that the ablation methodology "is sound" (Hkr6), and that the overall research approach is "well-conceived and clear" (LkpR).

The reviews share one question: what each part of AnyGRAM contributes beyond the components it builds on, and how it compares with recent graph foundation models in accuracy and cost. We added the following results.

  • The global product (LkpR, Wry4). On CiteSeer and full-DBLP, where support CV assigns 80% of the query nodes to global branches, removing the global product lowers accuracy by 7.56 and 2.23 points, while the mean over the 20 graphs outside the six covariance graphs changes by less than 0.01 points (Table 12). The average decrease of 0.31 points over all 26 graphs combines these effects.
  • The invariant inputs (Wry4, Hkr6). Each input group follows from two requirements of the setting: a width independent of the feature dimension and the number of classes, and invariance to rotations of the projection. Removing any one group lowers the observed mean accuracy on the six covariance graphs by 0.62 to 0.77 points (Table 11). Appendix I states exactly what the invariance guarantees and what the method requires of its input data.
  • Decisions on the target graph (Hkr6, LkpR, Wry4). Choosing the ridge regularization by GCV improves on a fixed multiplier by 1.1 points; choosing the branch by support CV improves on the learned fusion by 2.4 points on the same predictions, and by 1.9 to 3.3 points for each of four source graphs; and making both choices on the target graph raises the five branches of GraphAny [1] from 65.49% to 69.81% (Tables 1, 2, and 4).
  • Recent graph foundation models (Wry4). From their released checkpoints, on the splits of Table 1, AnyGRAM is close to View Space in observed mean accuracy [2] (+0.06 points), above NodePFN [3] by 2.72 points, and within 1.80 points of GraphPFN [4], which is pretrained on synthetic graphs starting from the tabular foundation model LimiX (Table 7).
  • Composition with GraphPFN (Hkr6, Wry4). Adding GraphPFN predictions to AnyGRAM's ten-branch bank and applying support-based selection reaches 74.73±0.2174.73 \pm 0.21%, versus 73.78±0.4073.78 \pm 0.40% for native GraphPFN with no target gradient updates. A separate extension with support-trained MLP heads reaches 75.64±0.1675.64 \pm 0.16% (Appendix G.1).
  • Cost (Wry4). On Arxiv, with 169,343 nodes, AnyGRAM takes 14.6 s and 1.93 GiB of GPU memory, and GraphPFN 568 s and 57.6 GiB (Table 9).
  • Simpler training and readout (Wry4). Training with a uniform fusion gives a similar observed accuracy to the learned fusion (72.0%), and the node-level overrides reuse the out-of-fold predictions of support CV, with no additional model evaluation (Table 2).

Added material, marked in blue in the revised PDF:

  • Abstract and Section 1: the division with its two testable consequences, and "inputs defined by the same construction on every graph" (Hkr6); the three differences from GraphAny and ALL-IN (LkpR); the margin of AnyGRAM stated over the zero-shot baselines trained on the same source graph (Wry4).
  • Section 2: NodePFN (Wry4).
  • Section 3: the two requirements behind the input groups (Wry4); the roles of the learned fusion and of the override margin (Wry4); a pointer to Appendix I (Hkr6).
  • Section 4: the comparison with recent graph foundation models and their cost (Wry4); the effect of the global product per graph (LkpR, Wry4); the control with randomly initialized encoders (LkpR, Hkr6).
  • Appendix G: accuracy, protocols, inference time, and memory of View Space, NodePFN, and GraphPFN (Tables 7 to 10; Wry4).
  • Appendix H: input-group ablations, the global product per graph, and randomly initialized encoders (Tables 11 to 13; LkpR, Wry4).
  • Appendix I: the invariance properties of the inputs and the requirements on the input data (Hkr6).

We thank the reviewers again for their feedback, which improved the paper. We hope that these additions address their concerns, and we kindly ask them to consider the additions in their evaluation. We are happy to discuss any remaining questions.

References:
[1] Fully-inductive Node Classification on Arbitrary Graphs
[2] View Space: Learning Representation across Arbitrary Graphs
[3] Learning Posterior Predictive Distributions for Node Classification from Synthetic Graph Priors
[4] GraphPFN: A Prior-Data Fitted Graph Foundation Model

Dear Reviewers,

Thank you for your time and effort in reviewing this paper. The authors have posted responses to your reviews. If you have not yet done so, please read their responses and leave a follow-up comment visible to the authors, indicating which concerns have been addressed and which remain unresolved.

Please also update your review and rating if your assessment has changed. Your feedback will help the authors address any remaining questions during the discussion period.

Thank you!

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.