- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The authors investigate whether task similarity can predict in-context transfer on graphs. The authors introduce a cross-graph evaluation protocol with disconnected support and query graphs and randomized label polarity to limit shortcut solutions. They propose polarity-symmetrized label agreement as a similarity measure, derive a finite-context transfer formula for a Bayesian reference learner, and evaluate its predictions across four model classes, five real graph datasets, and a frozen GraphPFN checkpoint.
Critical strengths
-
The evaluation protocol includes thoughtful controls, including support-label permutation and wrong-rule contexts, providing convincing evidence that the observed gains depend on the support labels.
-
The paper goes beyond reporting correlations by testing finite-context predictions, out-of-sample forecasts, and complementary task pools, offering a useful framework for studying transfer and selecting pretraining tasks.
Critical weaknesses
The central claim of a general, parameter-free transfer law is broader than the evidence establishes. The theoretical formula relies on an additional symmetric-noise assumption for out-of-distribution contexts and independent agreement observations, while practical predictions sometimes require a fitted efficiency coefficient that does not transfer across settings. Moreover, the GraphPFN experiment varies context–target similarity rather than pretraining–target similarity. These results support a useful predictive relationship under controlled conditions, but do not yet establish a universal law for forecasting transfer from pretraining tasks.
Questions for authors
See weaknesses.