acceptodds
Under review as a conference paper at ICLR 2027

LPLP: Learning Patterns for Link Prediction from Context

Abstract

Link prediction is a fundamental task in graph machine learning with applications across diverse domains. It relies on structural patterns that vary substantially across graphs, making predictors learned on one graph difficult to transfer to another. Recent universal link predictors adapt to new graphs through labeled context, but still learn how to use this context from real graphs. We introduce **LPLP**, which **L**earns **P**atterns for **L**ink **P**rediction from context, following the prior-data fitted network paradigm. LPLP is trained entirely on synthetic graphs generated from a prior in which the structural patterns that predict links vary across tasks. Given an unseen graph with labeled context links, LPLP infers the structural pattern of the graph and scores query links without parameter updates. Across seven benchmarks, LPLP achieves the best average rank among methods evaluated on shared splits under both random and HeaRT hard negatives, improving average Hits@50 over the strongest baseline by 5.45 and 4.22 points, respectively. These results indicate that the ability to infer structural patterns from context can be learned from synthetic graph priors and applied to unseen graphs.

Primary area:
learning on graphs and other geometries & topologies
Keywords:
graph machine learning, link prediction, in-context learning

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_JcPJOpenReview ↗
Rating
3(+1)
Confidence
4
Informativeness
2
Presentation
3
Soundness
2(+1)
Read review
Paper summary

The authors propose LPLP which pre-trains on synthetic DC-SBM graphs and integrate the prior data-fitted learning regime to infer context and then predict on links across varying tasks without the need for fine-tuning onto target graphs. The authors demonstrate the different link-formation rules, separation between the node-formation and link-formation task, encoding+context+learning objectives. This is subsequently followed by analysis with random and hard negatives, and deeper studies on ablating model components and analyzing the effects of structural rules.

Critical strengths

LPLP is a powerful addition to the canon of PFN-style models for graph representation learning. I am largely surprised that a link prediction PFN had not existed until this point. So, I believe the formulation as-is has merit as a contribution for demonstrating how to make a PFN work for the link-prediction. The addition of the node-classification results is promising as well for separate study into wider aspects of graph representation learning.

Critical weaknesses

I have no doubts on whether LPLP is an effective model for the link prediction task, the effective pre-training from the link-formation rule is promising, my core concern is around the fairness of the evaluation setting relative to the tested baselines. In particular, LPLP is tested with DRNL+, 12 layers, the attention+message-passing context layers of Graph/NodePFN, all of which leads to 30.8 million hyperparameters in total. This, combined with the results within Table 1 and Table 2 almost make it appear that baseline GNN4LP methods are ineffective, even when trained on their single datasets. The current Table 11 provides no indication of layer number for the benchmarked baselines, so it makes it impossible to isolate if the DC-SBM pretraining boosts performance on a majority of the evaluated datasets or if the stronger performance is due to LPLP has a larger encoder. The ablation on structural information is a helpful start, but I am still not certain from how the manuscript is currently presented.

Separately, the paper does not use the standard of the OGB benchmark for which the evaluated GNN4LP methods (i.e. BUDDY, NCNC, NeoGNN) were constructed for. The current results are promising, but this reviews gathers that the benefit from PFN learning style on link prediction is a pre-train then zero-shot evaluation. Without the OGBL datasets, it remains difficult to determine if LPLP serves as a powerful baseline that the link prediction community could utilize in scenarios demanding scalability.

Questions for authors
  1. This is the first part of my critical weakness. What standard are the encoders for each benchmark baseline tuned to relative to LPLP, is it as comparable as possible? I understand that DRNL+ is specific to UniLP but what standard are the number of layers for benchmarked baselines tuned to? If feasible, this could be benchmarked and correlated to downstream results to alleviate this concern.

  2. This is the second part of my critical weakness. Why is LPLP not evaluated on OGBL [2] datasets? Given that LPLP is a PFN-style model, it seems critical to use a larger dataset in order to determine whether there is a benefit from the prior-fitted paradigm on problems that require scalability. In particular, can LPLP scale to the likes of: ogbl-collab, ogbl-ppa, or even ogbl-citation2?

Minor Concerns, that do not affect evaluation:

  • The Power grid network test for Section 5.6, Line 427-428 cites Watts & Strogatz [1], this seems to be a relic from previous testing but the citation should be updated for clarity.

[1] Duncan J. Watts and Steven H. Strogatz. Collective dynamics of ‘small-world’ networks. Nature, 393(6684):440–442, Jun 1998. ISSN 1476-4687. doi: 10.1038/30918. URL https://doi. [2] Hu W, Fey M, Zitnik M, Dong Y, Ren H, Liu B, Catasta M, Leskovec J. Open graph benchmark: Datasets for machine learning on graphs. Advances in neural information processing systems. 2020;33:22118-33. org/10.1038/30918

4 comments
AuthorsOpenReview ↗

Rebuttal Part 1

We sincerely thank Reviewer JcPJ for the thoughtful feedback and engagement with the contribution of our work. Section, table, and appendix numbers in our responses refer to the revised manuscript, where the changes are marked in blue with the reviewer's name.

Response to W1 & Q1: Fairness of the comparison with baselines

We thank the reviewer for this concern, which we address with three new experiments.

How the baselines are configured. Each GNN4LP baseline is retrained on every split with its official implementation and the training recipe of its repository. Without node attributes, a baseline can be trained with or without a learnable node embedding, and which of the two is better depends on the method and on the graph. We therefore train each of the eight configurations of Table 12 with both node inputs and select the best for each graph by validation Hits@50. SEAL is trained without node embeddings, as in its official implementation. The number of GNN layers follows the official implementation. GCN and NCNC have one GNN layer, Neo-GNN three, and SEAL four, while ELPH, BUDDY, and MPLP have two GNN layers on the graphs where they learn node embeddings and none on the others. LPLP, in contrast, uses a single pretrained checkpoint for every graph and is neither trained nor tuned on the target graphs. The revised Appendix F.2 describes this setup and lists the layers of every baseline (Table 13).

Adding layers to the baselines. We added one and two GNN layers to GCN, NCNC, SEAL, Neo-GNN, and MPLP, keeping their selected configurations. MPLP has no GNN layer on the graphs where it learns no node embedding, so layers are added to it only on the other graphs.

Avg. Hits@50Official layers (Table 1)+1 layer+2 layers
GCN (1)66.1066.1666.01
NCNC (1)66.8666.5365.93
SEAL (4)68.8468.2968.20
Neo-GNN (3)67.8268.3767.21
MPLP (0 or 2)70.2470.6070.53
LPLP74.57

Additional layers bring no consistent gain, changing the average Hits@50 of every baseline by less than one point, from -0.93 to +0.55. Every baseline thus stays at least 3.97 points below LPLP.

Fewer layers in LPLP. We also pretrained LPLP with 4 and 6 layers in place of 12, keeping the prior, the training schedule, the random seed, and every other setting.

LayersParametersLP (avg. Hits@50)NC (avg. accuracy)
411.5M74.4370.31
616.3M74.4971.28
1230.5M74.5771.94

With 4 layers and 11.5M parameters, LPLP loses 0.14 points of average Hits@50 and remains 4.19 points above MPLP, the strongest baseline in Table 1. While changing the number of layers of LPLP has little effect on its link prediction performance, it affects its node classification performance, whose mean accuracy over the 23 datasets of Table 7 rises from 70.31 with 4 layers to 71.94 with 12 layers (Table 26).

Pretraining UniLP on our prior. The two experiments above indicate that the advantage of LPLP over the baselines does not come from the number of layers. To test whether it comes from the prior, we pretrained UniLP on our prior and compared it with its released checkpoint, which is pretrained on real-world graphs. We follow UniLP's official training and checkpoint-selection setting, replace its pretraining graphs with graphs sampled from our prior, and use the same number of pretraining graphs as LPLP.

Hits@50CelegansUSAirPBNSCoraCSFacebookAvg.
UniLP (released)66.2286.9448.0487.8556.9266.2064.3568.07
UniLP + our prior74.3691.2949.2282.1064.5272.5970.6372.10
LPLP75.2492.7153.2188.3865.1572.8774.4074.57

Pretraining on our prior raises the average Hits@50 of UniLP from 68.07 to 72.10. The gain holds in all ten splits on Celegans, USAir, Cora, CS, and Facebook, and in five of the ten splits on PB. With the architecture and training setting of UniLP fixed, exposure to our synthetic prior therefore transfers better to the evaluation graphs than the real-world graphs on which UniLP is pretrained, on six of the seven graphs and by 4.03 points on average.

In summary, adding layers to the baselines brings no consistent gain, and reducing LPLP from 12 to 4 layers changes its average Hits@50 by 0.14 points, so the advantage of LPLP does not come from the number of layers. Pretraining UniLP on our prior, in contrast, raises its average Hits@50 by 4.03 points over its released checkpoint, which is pretrained on real-world graphs. These experiments are reported in the revised manuscript, Appendix G.9, “Effect of the Number of Layers” (Tables 24 to 26), and Appendix G.10, “Pretraining UniLP on Our Synthetic Prior” (Table 27).

AuthorsOpenReview ↗

Rebuttal Part 2

Response to W2 & Q2: Evaluation on OGB

We thank the reviewer for this suggestion. We evaluated LPLP and the baselines on ogbl-ppa and ogbl-citation2 with their official splits and metrics and without node features. LPLP runs on both graphs, including ogbl-citation2 with 2.9 million nodes. LPLP and UniLP use two-hop enclosing subgraphs and 200 context links per class, drawn anew for each seed. Each baseline uses its official implementation with the hyperparameters released for that graph, and only the node input is selected on validation as in the main experiments (Appendix F.2). MPLP+ is the variant of MPLP that its authors use on these graphs, and SEAL trains on 5% of the training links of ogbl-ppa and 2% of those of ogbl-citation2, as in the command of its authors.

Methodogbl-ppa (Hits@100)ogbl-citation2 (MRR)
RA49.3376.03
SEAL49.68 ± 3.5186.75 ± 0.82
Neo-GNN48.44 ± 0.6579.74 ± 0.14$^\dagger$
BUDDY47.05 ± 0.8583.53 ± 0.03
NCNC58.20 ± 1.04$^\dagger$85.97 ± 0.04$^\dagger$
MPLP+64.84 ± 1.02†^\dagger90.86 ± 0.15†^\dagger
UniLP32.66 ± 0.0477.20 ± 0.23
LPLP50.38 ± 2.6586.92 ± 0.60

Values are the mean and standard deviation over three seeds. †^\daggerThe variant with a learned node embedding is selected on validation.

Without any training on these graphs, LPLP ranks third on ogbl-ppa and second on ogbl-citation2 among the eight methods. NCNC and MPLP+ on ogbl-ppa and MPLP+ on ogbl-citation2 learn a node embedding from the 21 million and 30 million training links of these graphs and show strong performance. However, among the methods that learn no node embedding, LPLP is the highest on both graphs. Without the node embedding, these methods fall below LPLP, with 49.06 for NCNC and 35.62 for MPLP+ on ogbl-ppa and 51.63 for MPLP+ on ogbl-citation2. LPLP is also well above UniLP, the other universal link predictor, by 17.72 points on ogbl-ppa and 9.72 points on ogbl-citation2. These results are reported in the revised manuscript, Appendix G.11, “Evaluation on OGB” (Table 28).

Response to Minor Concern: Citation of the Power grid network

We thank the reviewer for pointing this out. The Power grid network is one of the networks studied in Watts and Strogatz [1], which the manuscript cited, but this is not the most direct reference for the graph. In Section 5.6 of the revised manuscript, we replaced it with the Network Data Repository [2], a more direct and recent reference for this graph.

[1] Watts DJ, Strogatz SH. Collective dynamics of 'small-world' networks. Nature. 1998;393(6684):440-442.

[2] Rossi RA, Ahmed NK. The network data repository with interactive graph analytics and visualization. Proceedings of the AAAI Conference on Artificial Intelligence. 2015.

Best regards,

Authors

Reviewer_JcPJOpenReview ↗

Rebuttal Response

Thank you to the authors for their rebuttal response, clarifications, and the added experiments. All of the added experiments have addressed my concerns. I find the results pre-training UniLP on the LPLP prior to be very interesting in particular and provide potential for further efforts into understanding link-formation rules within the PFN-learning paradigm. I have raised my score to account for this.

Dear Reviewer JcPJ,

Thank you for your thoughtful review. We are glad that the added experiments addressed your concerns. We will include the points you raised in the final version, and we will address the remaining limitations of LPLP in future work.

Best regards,

Authors

Reviewer_5bQ6OpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
2
Soundness
3
Read review
Paper summary

LPLP is a prior-data fitted network for link prediction trained only on synthetic graphs. Given an unseen graph and a labeled context of links and non-links, it scores query pairs without parameter updates. It achieves the best average rank on 7 benchmarks under both random and HeaRT negatives.

Critical strengths
  • Transfer results from training on synthetic data alone beat baselines which are trained on the graphs. This shows that there are perhaps some base structures that are generalizable to real-world graphs, and choosing a good synthetic prior can replace pretraining on real graphs.
  • There is strong evidence to show that LPLP learns patterns from the defined context as opposed to the graph alone. Biased contexts induce preferences inherent in their context. Similarly, scrambling the context causes performance to collapse.
Critical weaknesses
  • The gains from the link formation seem quite marginal. In Table 14, w/o link formation, there are marginal performance decreases. Perhaps suggesting that the gains come from the PFN and synthetic graphs (DC-SBM) structure?
Questions for authors
  • I would be very interested to see how this performs on large-scale graph link-prediction. Are the subgraphs sampled from this large-scale graph sufficient in good link-prediction performance for LPLP?
5 comments
AuthorsOpenReview ↗

Rebuttal Part 1

We thank Reviewer 5bQ6 for taking the time to read our paper and leaving insightful comments. Section, table, and appendix numbers in our responses refer to the revised manuscript, where the changes are marked in blue with the reviewer's name.

Response to W1: Role of link formation rules

We sincerely thank the reviewer for this observation.

For the seven benchmarks used to evaluate LPLP, 75% of the test links on average join nodes at distance two in the training graph. This concentration of test links at distance two may partly explain the modest gains from the link-formation rule in Table 16 (Table 14 of the submitted version).

However, the pattern that predicts links differs from graph to graph and is not known before a new graph is given. We therefore want a universal link predictor to infer this pattern from the labeled context of each graph, instead of relying on closeness alone. To this end, the link-formation rule exposes LPLP to a different link-formation pattern in every pretraining graph. Without it, the model adapts less to its context, and its adaptation can fail in some cases. In Section 5.5 (Table 3), under Context A, whose positives are at distances 2 and 4 and whose negatives are at distance 3, the AUC for ranking distance 4 above distance 3 is 93.5 on Cora and 73.8 on CS for LPLP, but 85.3 and 45.1 for the model pretrained without the rule, which on CS does not follow the context.

To test the rule on a real graph where links do not follow closeness, we evaluated both models on PCQM-Contact [1], a molecular link prediction task from the Long Range Graph Benchmark. Each graph is a molecule, and a link is a contact between two atoms that are closer than 3.5 A˚\mathring{\mathrm{A}} in 3D space but at least five hops apart in the molecular graph. Every link therefore joins a distant pair, in contrast to the seven benchmarks. We use no atom or bond features, so only the labeled context shows that links join distant atoms. The context consists of 200 contacts and 200 non-contact pairs drawn from training molecules, and we score every atom pair within each of the 26,472 test molecules. Following [2], we report the extended filtered MRR and Hits@K, which rank each contact against all atoms of its molecule. Because the result depends on which context is drawn, we repeat the evaluation with ten independent context draws and report the mean and standard deviation over these draws.

MRRHits@1Hits@3Hits@10
LPLP0.477 ± 0.0080.267 ± 0.0120.602 ± 0.0030.924 ± 0.001
w/o rule0.421 ± 0.0250.202 ± 0.0300.536 ± 0.0290.911 ± 0.007

LPLP is higher in all ten context draws and at every contact distance (5, 6, and at least 7 hops). The model without the rule also depends more on which context is drawn, with an MRR from 0.379 to 0.455 against 0.465 to 0.487 for LPLP.

To sum up, the results on the seven benchmarks can be interpreted as the reviewer suggests, with most of the gain coming from the PFN and the DC-SBM base structure of the prior. The model pretrained without the rule reaches an average Hits@50 of 73.26 under random negatives and 58.31 under HeaRT negatives, still above the 70.24 and 55.08 of MPLP, the strongest baseline. Because most test links of these benchmarks lie at distance two, the base structure can already fit them, and the rule adds little. The rule brings a clear gain on graphs whose links follow another pattern, as on PCQM-Contact, where it raises the MRR from 0.421 to 0.477, and under contexts that favor longer distances (Table 3). The distance shares and the PCQM-Contact analysis are reported in the revised manuscript, Section 5.5 (Table 4) and Appendix G.4, “Effect of the Link-Formation Rule” (Table 18).

[1] Dwivedi VP, Rampášek L, Galkin M, Parviz A, Wolf G, Luu AT, Beaini D. Long range graph benchmark. Advances in neural information processing systems. 2022;35.

[2] Tönshoff J, Ritzert M, Rosenbluth E, Grohe M. Where did the gap go? Reassessing the long-range graph benchmark. arXiv preprint arXiv:2309.00367. 2023.

AuthorsOpenReview ↗

Rebuttal Part 2

Response to Q1: Evaluation on OGB

We thank the reviewer for this question. We evaluated LPLP and the baselines on ogbl-ppa and ogbl-citation2 with their official splits and metrics and without node features. LPLP and UniLP use two-hop enclosing subgraphs and 200 context links per class, drawn anew for each seed. Each baseline uses its official implementation with the hyperparameters released for that graph, and only the node input is selected on validation as in the main experiments (Appendix F.2).

Methodogbl-ppa (Hits@100)ogbl-citation2 (MRR)
RA49.3376.03
SEAL49.68 ± 3.5186.75 ± 0.82
Neo-GNN48.44 ± 0.6579.74 ± 0.14$^\dagger$
BUDDY47.05 ± 0.8583.53 ± 0.03
NCNC58.20 ± 1.04$^\dagger$85.97 ± 0.04$^\dagger$
MPLP+64.84 ± 1.02†^\dagger90.86 ± 0.15†^\dagger
UniLP32.66 ± 0.0477.20 ± 0.23
LPLP50.38 ± 2.6586.92 ± 0.60

Values are the mean and standard deviation over three seeds. †^\daggerThe variant with a learned node embedding is selected on validation.

Without any training on these graphs, LPLP ranks third on ogbl-ppa and second on ogbl-citation2 among the eight methods. NCNC and MPLP+ on ogbl-ppa and MPLP+ on ogbl-citation2 learn a node embedding from the 21 million and 30 million training links of these graphs and show strong performance. However, among the methods that learn no node embedding, LPLP is the highest on both graphs. Without the node embedding, these methods fall below LPLP, with 49.06 for NCNC and 35.62 for MPLP+ on ogbl-ppa and 51.63 for MPLP+ on ogbl-citation2. LPLP is also well above UniLP, the other universal link predictor, by 17.72 points on ogbl-ppa and 9.72 points on ogbl-citation2. In answer to the question, two-hop subgraphs with 200 context links per class are thus enough for LPLP to reach this performance on graphs of this scale, without any training on them. These results are reported in the revised manuscript, Appendix G.11, “Evaluation on OGB” (Table 28).

Best regards,

Authors

Dear Reviewer 5bQ6,

Thank you again for your review. As the author–reviewer discussion closes on November 18 (AoE), we would like to check whether our response addresses your concerns. In response to your review, we added an experiment on PCQM-Contact on the role of the link-formation rule and results on OGBL with the subgraph sampling we use for large graphs.

If any question remains, we would be glad to answer it before the discussion closes.

Best regards,

Authors

Reviewer_Y4DMOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper proposes LPLP, a prior-data fitted network for link prediction trained solely on synthetic graphs. Its main novelty is a graph prior that varies distance-specific link-formation rules across tasks. The architecture combines enclosing-subgraph pair tokens, in-context attention, and graph-level message passing. The reported evaluation is strong on seven real graphs, and controlled context interventions provide evidence that predictions adapt to labeled structural patterns.

Critical strengths
  1. Prior is designed around task variation: the construction deliberately varies which path lengths predict links
  2. Convincing performance under two negative protocols.
  3. Extensive experiments.
Critical weaknesses
  1. Missing ablation on auxiliary node-classification training.
  2. Limited generalization settings: The proposed framework is evaluated exclusively on homogeneous, undirected graphs without node attributes.
  3. Limited improvement attributable to the proposed synthetic prior: Although the graph-specific link-formation rule is a key contribution, removing it has only a small impact on performance. Average Hits@50 drops from 74.57 to 73.26 with random negatives and from 59.12 to 58.31 with hard negatives.
Questions for authors
  1. Could the authors clarify why the proposed link-formation rule produces only modest improvements in average Hits@50 despite being a central contribution?
  2. Could the authors provide an ablation comparing the current jointly pretrained model with a model pretrained exclusively on link prediction?
  3. How well does LPLP generalize to graphs whose link-formation mechanisms are not represented by this prior?
  4. How will your method apply to directed graphs?
  5. Continuing Q4., as many heterophilic graphs are directed, can you apply your method on a heterophilic directed dataset, like TEXAS, WISCONSIN etc.?
  6. I would be interested to see how does the method perform on a OGB dataset. Can you please try one?
5 comments
AuthorsOpenReview ↗

Rebuttal Part 1

We sincerely thank the reviewer Y4DM for the insightful and constructive feedback. Section, table, and appendix numbers in our responses refer to the revised manuscript, where the changes are marked in blue with the reviewer's name.

Response to W1 & Q2: Ablation on auxiliary node-classification training

We pretrained a model exclusively on link prediction (LP-only). It uses the same prior, architecture, training schedule, and random seed as LPLP, and differs only in that the node-classification loss is not backpropagated.

Hits@50 (%)CelegansUSAirPBNSCoraCSFacebookAvg.
Random negatives
LPLP75.2492.7153.2188.3865.1572.8774.4074.57
LP-only75.3892.3853.5989.0565.0972.5175.2674.75
HeaRT negatives
LPLP55.9278.5929.1781.5350.1355.6662.8759.12
LP-only55.6479.2030.4282.1050.0255.8962.5959.41

As shown in the table, the link prediction performance of LPLP does not depend on the auxiliary node-classification training. Under random negatives, LP-only reaches an average Hits@50 of 74.75 against 74.57 for LPLP, with differences of -0.36 to +0.86 points on the individual graphs. Under HeaRT negatives, the differences are -0.28 to +1.25 points. This ablation is reported in the revised manuscript, Appendix G.7, “Pretraining without Node Classification” (Table 21).

Response to W3 & Q1: Role of link formation rules

We thank the reviewer for the opportunity to clarify the role of the link-formation rule.

On the seven benchmarks, 75% of the test links on average join nodes at distance two in the training graph. On such graphs, a preference for closer pairs already fits most test links, so the link-formation rule may have little effect on the average Hits@50.

We include the rule because a model that uses its context as evidence should adapt to the pattern that the context shows, including patterns other than closeness. The rule exposes LPLP to a different link-formation pattern in every pretraining graph. Without it, the model adapts less to its context, and its adaptation can fail in some cases. In Section 5.5 (Table 3), under Context A, whose positives are at distances 2 and 4 and whose negatives are at distance 3, the AUC for ranking distance 4 above distance 3 is 93.5 on Cora and 73.8 on CS for LPLP, but 85.3 and 45.1 for the model pretrained without the rule, which on CS does not follow the context.

To test the rule on a real graph where links do not follow closeness, we evaluated both models on PCQM-Contact [1]. Every link in PCQM-Contact is a contact between atoms that are at least five hops apart in the molecular graph. We use no atom or bond features, so only the labeled context (200 contacts and 200 non-contact pairs from training molecules) shows that links join distant atoms. We evaluate all 26,472 test molecules with ten context draws and report the extended filtered metrics of [2].

MRRHits@1Hits@3Hits@10
LPLP0.477 ± 0.0080.267 ± 0.0120.602 ± 0.0030.924 ± 0.001
w/o rule0.421 ± 0.0250.202 ± 0.0300.536 ± 0.0290.911 ± 0.007

LPLP is higher in all ten context draws and at every contact distance (5, 6, and at least 7 hops). The model without the rule also depends more on which context is drawn, with an MRR from 0.379 to 0.455 against 0.465 to 0.487 for LPLP. The distance shares and the PCQM-Contact analysis are reported in the revised manuscript, Section 5.5 (Table 4) and Appendix G.4, “Effect of the Link-Formation Rule” (Table 18).

[1] Dwivedi VP, Rampášek L, Galkin M, Parviz A, Wolf G, Luu AT, Beaini D. Long range graph benchmark. Advances in neural information processing systems. 2022;35.

[2] Tönshoff J, Ritzert M, Rosenbluth E, Grohe M. Where did the gap go? Reassessing the long-range graph benchmark. arXiv preprint arXiv:2309.00367. 2023.

AuthorsOpenReview ↗

Rebuttal Part 2

Response to Q3: Ability of generalization

We thank the reviewer for raising this important question.

We do not expect LPLP to generalize to every graph, and characterizing exactly where it does is difficult. Mao et al. [3] group the factors behind links into feature proximity, local structural proximity, and global structural proximity, and show that different factors dominate different links, even within one graph. The mechanisms behind a real graph are therefore mixed and usually unknown, so they cannot be listed and checked against a prior one by one.

Our prior therefore does not model the latent mechanisms behind links. It instead varies the structural patterns that such mechanisms leave in the observed graph, covering the local and global structural proximity of [3]. The base structure ranges from assortative to disassortative communities, and each graph-specific rule weights path counts at distances from 2 to 8 differently, from local proximity (common neighbors) to global proximity (longer paths). LPLP is pretrained to infer from its labeled context which of these patterns a graph follows, so what matters is whether the pattern of a graph lies within the prior, not whether its mechanism does.

PCQM-Contact [1] is an example. Its links are formed by a mechanism that our prior does not model, spatial contact in the 3D geometry of a molecule. This mechanism leaves an observable pattern in the molecular graph, in which every link joins atoms at least five hops apart. Without atom features, bond features, or 3D coordinates, LPLP adapts to this pattern through its labeled context and reaches an MRR of 0.477 on the 26,472 test molecules, against 0.421 for the model pretrained without the rule that varies which distances predict links (the setting is given in our response to W3 & Q1).

This example shows adaptation to an observable pattern without knowledge of its mechanism, not generalization to every graph. Generalization is limited under two conditions.

Information availability. If links depend on information that the topology does not carry, such as node attributes (the feature proximity of [3]), their pattern may not be observable in the graph, and no topology-only predictor can reliably recover them.

Prior mismatch. If the observable pattern differs substantially from those the prior generates, the predictions of LPLP are not guaranteed, since a PFN is trained to approximate the posterior predictive distribution of its synthetic prior [4].

How far PFNs generalize beyond their pretraining prior remains an open question in PFN research [4]. The revised manuscript states these two limitations in the conclusion (Section 7).

[3] Mao H, Li J, Shomer H, Li B, Fan W, Ma Y, Zhao T, Shah N, Tang J. Revisiting link prediction: A data perspective. International Conference on Learning Representations. 2024.

[4] Müller S, Reuter A, Hollmann N, Rügamer D, Hutter F. Position: The future of Bayesian prediction is prior-fitted. International Conference on Machine Learning. 2025.

AuthorsOpenReview ↗

Rebuttal Part 3

Response to W2 & Q4, Q5: Evaluation scope, directed and heterophilic graphs

We thank the reviewer for highlighting the limitations of our evaluation scope and suggesting additional experiments.

LPLP is designed for link prediction on undirected graphs without node attributes. For a fair comparison, all evaluated methods receive the same undirected graph structure without node attributes. LPLP can be applied to graphs with directed edges by converting them to undirected graphs. Its prior generates undirected links and its pair encoding is symmetric in the two endpoints, so on a directed graph it predicts whether two nodes are linked but not the direction of the link. Predicting the direction would require extending the prior and the pair encoding to directed link formation. We state this limitation in the revised conclusion (Section 7) and leave such extensions to future work.

Following the reviewer’s suggestion, we evaluated LPLP on Texas and Wisconsin and on three more heterophilic graphs, Cornell, Actor, and Roman-empire. As described above, each graph is converted to an undirected graph, and we follow the protocol of Table 1 with the baselines tuned as in the main experiments. Texas, Wisconsin, and Cornell have only 27 to 45 validation negatives, so we report Hits@20 on these three graphs and Hits@50 on the other two.

MethodTexas (Hits@20)Wisconsin (Hits@20)Cornell (Hits@20)Actor (Hits@50)Roman-empire (Hits@50)
Katz40.73 ± 6.1046.89 ± 6.7742.00 ± 6.8219.86 ± 0.8247.15 ± 0.74
SEAL76.36 ± 7.1860.56 ± 15.0769.09 ± 5.2130.17 ± 3.0047.65 ± 0.42
Neo-GNN69.82 ± 10.1656.78 ± 6.8278.73 ± 9.3831.99 ± 1.6235.00 ± 15.04
BUDDY38.18 ± 14.6867.00 ± 7.7159.64 ± 16.3625.33 ± 1.3639.15 ± 0.60
NCNC62.55 ± 17.7855.11 ± 18.2066.55 ± 11.1527.53 ± 1.7544.60 ± 1.12
MPLP41.82 ± 19.0341.11 ± 9.3850.00 ± 11.5631.23 ± 1.7046.67 ± 0.64
UniLP78.73 ± 5.3473.78 ± 4.5172.18 ± 8.6123.75 ± 5.4235.45 ± 2.36
LPLP84.18 ± 8.1375.56 ± 4.7976.55 ± 8.3536.02 ± 2.0646.73 ± 0.77

LPLP achieves the highest Hits@K on Texas, Wisconsin, and Actor. On Cornell, it is second, 2.18 points below Neo-GNN. On Roman-empire, the four highest methods lie within one point, with LPLP 0.92 points below SEAL. On Texas, Wisconsin, and Cornell, the common-neighbor heuristics CN, AA, and RA reach an AUROC of only 54.70 to 58.81, so shared neighbors barely separate links from non-links on these graphs, yet LPLP ranks first or second there without any training on them. Thus LPLP can be applied to heterophilic graphs with directed edges, such as Texas and Wisconsin, by treating them as undirected. The full results are reported in the revised manuscript, Appendix G.8, “Evaluation on Heterophilic Graphs” (Tables 22 and 23).

Response to Q6: Evaluation on OGB

We thank the reviewer for the suggestion. We evaluated LPLP and the baselines on ogbl-ppa and ogbl-citation2 with their official splits and metrics and without node features. LPLP and UniLP use two-hop enclosing subgraphs and 200 context links per class, drawn anew for each seed. Each baseline uses its official implementation with the hyperparameters released for that graph, and only the node input is selected on validation as in the main experiments (Appendix F.2).

Methodogbl-ppa (Hits@100)ogbl-citation2 (MRR)
RA49.3376.03
SEAL49.68 ± 3.5186.75 ± 0.82
Neo-GNN48.44 ± 0.6579.74 ± 0.14$^\dagger$
BUDDY47.05 ± 0.8583.53 ± 0.03
NCNC58.20 ± 1.04$^\dagger$85.97 ± 0.04$^\dagger$
MPLP+64.84 ± 1.02†^\dagger90.86 ± 0.15†^\dagger
UniLP32.66 ± 0.0477.20 ± 0.23
LPLP50.38 ± 2.6586.92 ± 0.60

Values are the mean and standard deviation over three seeds. †^\daggerThe variant with a learned node embedding is selected on validation.

Without any training on these graphs, LPLP ranks third on ogbl-ppa and second on ogbl-citation2 among the eight methods. NCNC and MPLP+ on ogbl-ppa and MPLP+ on ogbl-citation2 learn a node embedding from the 21 million and 30 million training links of these graphs and show strong performance. However, among the methods that learn no node embedding, LPLP is the highest on both graphs. Without the node embedding, these methods fall below LPLP, with 49.06 for NCNC and 35.62 for MPLP+ on ogbl-ppa and 51.63 for MPLP+ on ogbl-citation2. LPLP is also well above UniLP, the other universal link predictor, by 17.72 points on ogbl-ppa and 9.72 points on ogbl-citation2. These results are reported in the revised manuscript, Appendix G.11, “Evaluation on OGB” (Table 28).

Best regards,

Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.