acceptodds
Under review as a conference paper at ICLR 2027

Task Similarity Predicts In-Context Transfer on Graphs

Abstract

Whether a graph foundation model will learn a new task in context cannot currently be predicted before training. The question is open because alignment theory for in-context learning is derived for linear models and does not reach graphs, and because standard graph few-shot evaluations conflate in-context inference with structural shortcuts such as label propagation. We construct a cross-graph rule-induction testbed that closes these shortcut channels, with no edge between demonstrations and queries and with the label polarity randomized per episode. On this testbed a model's in-context gain is governed by a single measurable scalar, the polarity-symmetrized label agreement between the training task and the test task. A Bayesian posterior-concentration argument predicts the full few-shot gain surface with no free parameters, and predicts that adding graph families or rules brings no gain unless it changes the agreement statistics. The law holds on synthetic rules, on five real graph datasets, and on a frozen, externally pretrained graph foundation model (GraphPFN). In-context transfer on graphs can therefore be forecast, and pretraining task design guided, from labeled examples alone, without model access or any training.

Primary area:
learning on graphs and other geometries & topologies
Keywords:
in-context learning, graph foundation models, transfer learning, task similarity, graph learning

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_CwqEOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The authors investigate whether task similarity can predict in-context transfer on graphs. The authors introduce a cross-graph evaluation protocol with disconnected support and query graphs and randomized label polarity to limit shortcut solutions. They propose polarity-symmetrized label agreement as a similarity measure, derive a finite-context transfer formula for a Bayesian reference learner, and evaluate its predictions across four model classes, five real graph datasets, and a frozen GraphPFN checkpoint.

Critical strengths
  1. The evaluation protocol includes thoughtful controls, including support-label permutation and wrong-rule contexts, providing convincing evidence that the observed gains depend on the support labels.

  2. The paper goes beyond reporting correlations by testing finite-context predictions, out-of-sample forecasts, and complementary task pools, offering a useful framework for studying transfer and selecting pretraining tasks.

Critical weaknesses

The central claim of a general, parameter-free transfer law is broader than the evidence establishes. The theoretical formula relies on an additional symmetric-noise assumption for out-of-distribution contexts and independent agreement observations, while practical predictions sometimes require a fitted efficiency coefficient that does not transfer across settings. Moreover, the GraphPFN experiment varies context–target similarity rather than pretraining–target similarity. These results support a useful predictive relationship under controlled conditions, but do not yet establish a universal law for forecasting transfer from pretraining tasks.

Questions for authors

See weaknesses.

Reviewer_wsrmOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper studies when a frozen graph model can transfer to a new task through in-context learning. The authors introduce a cross-graph rule-induction testbed that prevents label propagation and randomizes label polarity across episodes. They define task similarity as the polarity-aligned label agreement between a training rule and an evaluation rule, and show that it strongly correlates with the in-context accuracy gain across graph families, task types, and model architectures. The paper further derives a Bayesian transfer law, which characterizes the gain using the context size KK and task similarity ss. Experiments support the predicted few-shot behavior and show that additional training-task diversity helps only when it improves task agreement or provides complementary errors. The findings also transfer to real graphs and a frozen GraphPFN model.

Critical strengths

A carefully controlled evaluation of graph in-context learning. The cross-graph testbed removes support-query edges, withholds query labels, and randomizes label polarity. These controls substantially reduce the possibility that the measured gain is caused by label propagation, memorized label semantics, or other shortcuts.

A simple law supported by broad evidence. The Bayesian law connects in-context gain to task similarity and context size in an interpretable form. Its predictions are validated across multiple rule families, graph distributions, model architectures, real-world graphs, and a frozen GraphPFN checkpoint. This combination of theory and controlled experiments is the paper’s strongest contribution.

Critical weaknesses

The theoretical result relies on a restrictive task model. The derivation assumes binary classification, randomized label polarity, and a Bayesian learner over a controlled rule family. It is unclear whether the same law extends to multiclass classification, regression, or realistic graph tasks where adaptation involves more than identifying label polarity.

Task similarity may be difficult to estimate in practical foundation-model settings. Computing label agreement requires labeled samples from both the target task and the relevant pretraining tasks. In many frozen foundation models, the pretraining task distribution is unknown or inaccessible.

Questions for authors

How could the proposed law be extended beyond binary polarity-aligned tasks, particularly to multiclass classification or regression?

How should task similarity be computed when the task distribution used to pretrain a frozen foundation model is unavailable to downstream users?

Reviewer_xJqHOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper investigates whether in-context transfer performance on graphs can be predicted using task similarity. The main novelty is that it introduce a cross-graph evaluation framework to eliminate structural shortcuts and propose a polarity-symmetrized label agreement to predict in-context gain. A Bayesian analysis provides theoretical explanations, and experiments across multiple architectures and graph datasets validate the findings.

Critical strengths
  1. The paper addresses an important question in graph in-context learning: how to predict the performance gains brought by support labels on a target task based on the relationship between source and target tasks, without access to the model.
  2. The proposed cross-graph rule-induction testbed separates support and query graphs and randomizes label polarity, effectively eliminating the proposed structural shortcuts. Meanwhile, polarity-symmetrized label agreement measures the similarity between source and target tasks, providing a effective metric to help predict in-context transfer gains in a training-free way.
  3. The Bayesian framework derives a parameter-free formula for accurately predicting in-context transfer gains, providing a theoretical explanation of how the number of support labels KK and task similarity ss jointly affect in-context transfer gains.
Critical weaknesses
  1. Most experiments rely on binary classification rules, leaving its generalization to more complex graph tasks (e.g., multi-class node classification and link prediction) unclear. Further experiments on these tasks would strengthen the paper's applicability.
  2. Computing task similarity requires sufficient labels from both source and target graphs. However, labels are often difficult to obtain in real-world graph datasets, making task similarity estimation challenging. Could the authors further discuss the generalization of the proposed method in few-shot settings?
Questions for authors

The major concerns are already discussed in the weaknesses above.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.