acceptodds
Under review as a conference paper at ICLR 2027

KGPFN: Unlocking the Potential of Knowledge Graph Foundation Model via In-Context Learning

Abstract

Knowledge graph (KG) foundation models aim to generalize to graphs with unseen entities and relations by learning transferable relational structure. Most existing methods, however, focus on relation-level universality, leaving in-context learning, the other pillar of foundation models, largely unexplored for KG reasoning. Context in KGs is structured and heterogeneous: accurate prediction requires conditioning both on the local neighborhood of the query entities and on global context that summarizes how the query relation behaves across many instances. We propose KGPFN, a KG foundation model built on a Prior-Data Fitted Network (PFN) that combines transferable relational representations with inference-time in-context learning over structured context. KGPFN learns relation representations by message passing on relation graphs and extracts multi-scale local context from the intermediate head representations of a multi-layer NBFNet. It then builds relation-specific global context from positive and negative examples of the query relation, together with their local structural representations, and aggregates this context with feature-level and sample-level attention. Through multi-graph pretraining, KGPFN learns to combine structural representations with labeled contextual evidence without inference-time parameter updates. On 57 knowledge graphs, KGPFN achieves the best average MRR both without and with fine-tuning, and context sensitivity analyses highlight the value of negative context examples.

Primary area:
learning on graphs and other geometries & topologies
Keywords:
Knowledge graph, In context learning

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_2xwVOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

KGPFN combines local context and global context for knowledge graph link prediction. It uses NBFNet to encode graph structure and a PFN initialized from TabICL to predict candidate triples using positive and negative examples of the query relation.

Critical strengths

1.The distinction between local context and global context is clear and provides a reasonable motivation for the method. Local context captures the query-specific neighborhood, while global context provides relation-specific evidence from positive and negative examples.

2.The paper provides theoretical analysis of conditional message passing and an idealized bilinear surrogate. Proposition 1 shows how labeled support examples change the effective structural weights without updating the model parameters.

3.The paper analyzes inference time and memory costs, including context caching and chunking. Experiments on FB15k-237 report per-query latency and peak GPU memory, showing similar latency to KGICL with substantially lower memory usage.

Critical weaknesses

1.The ablation study does not sufficiently establish the contribution of relation-specific global context. Replacing the embeddings supplied to the PFN with random vectors, all-ones vectors, or embeddings saved at a selected epoch and kept fixed afterward would help asses how much the graph representations produced by the Contextual Graph Encoder & Feature Fusion module contribute to link prediction performance.

2.The paper initializes the PFN from TabICL but does not sufficiently justify this choice. Comparisons with other PFN models, such as TabPFN or LimiX, would clarify whether the results depend on TabICL or extend to other PFN models.

3.The comparison omits SEMMA and SCR, and KGICL is included only in the fine-tuned setting. Since KGICL also performs in-context learning for link prediction, its absence from W/o FT limits the comparison with a closely related method. Including these baselines under comparable evaluation settings would provide a more complete evaluation against existing KGFMs.

4.The improvements of KGPFN are modest: under W/o FT, KGPFN improves MRR over FLOCK from 0.392 to 0.396, while both achieve a Hits@10 of 0.560. Statistical significance tests are needed to assess these gains to avoid the variability of training.

Questions for authors

Please see the Weaknesses.

Reviewer_SvCQOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper proposes KGPFN, a link prediction model that combines transferable knowledge graph representations with in-context learning from labeled examples. The model obtains query-conditioned relation representations through message passing on a relation graph, extracts multi-scale local context from intermediate head-node states of NBFNet, and constructs a global support set consisting of positive and negative triples for the query relation. A Prior-Data Fitted Network (PFN) then integrates structural information within each instance through feature-level attention and allows candidate triples to retrieve evidence from the support set through sample-level attention, enabling adaptation to the target graph without inference-time parameter updates.

The main contribution is the explicit distinction between local structural context and relation-specific labeled examples, together with a PFN-based framework for jointly modeling these two sources of information. The paper is not the first to investigate in-context learning for knowledge graphs. Its primary distinctions from KGICL are the use of positive and negative support examples, multi-scale local representations, and a PFN-based architecture.

The authors evaluate the model on 57 benchmark graphs, both without and with fine-tuning, reporting overall MRR values of 0.396 and 0.434, respectively, which exceed those of the compared methods. The evaluation also includes analyses of context size and depth, ablations of tail-aware interaction features, and inference efficiency comparisons. These results support the competitiveness of the approach. However, they do not fully isolate the contribution of the central in-context learning mechanism or establish the robustness of the relatively small improvements over the strongest baseline.

Critical strengths
  1. A meaningful integration of transferable structure and labeled context

KGPFN goes beyond learning structural representations that transfer across graphs by conditioning predictions on positive and negative examples of the target relation. This is a meaningful adaptation mechanism for knowledge graphs with changing entity and relation vocabularies.

The contribution is therefore more than a replacement of the encoder or scoring function: it investigates how knowledge graph foundation models can exploit evidence available at inference time. Although most of the constituent modules have precedents, the problem formulation and their integration retain research value.

  1. Broad evaluation provides initial empirical support for the approach

The evaluation covers 57 benchmark graphs across three generalization settings and reports results both without and with fine-tuning, alongside inference costs. The reported performance is not limited to a single favorable dataset, and the aggregate metrics exceed those of the compared methods.

This breadth gives the approach an empirical basis beyond its conceptual motivation. For a method intended to support cross-graph adaptation, evaluation across diverse benchmarks is an important component of the case for acceptance.

Critical weaknesses
  1. Insufficient experimental isolation of the central contribution

The central claim is that the PFN uses labeled context to adapt to the target relation at inference time. However, the existing ablations primarily examine interaction features, local-context depth, and context size. They do not adequately address the following questions:

  • Could a simpler aggregation method achieve comparable performance using the same structural representations and support set?
  • How much does performance deteriorate when the context labels are shuffled?
  • How much of the reported improvement is attributable to initialization from pretrained TabICL weights?

Moreover, the context sensitivity experiments retrain the model for each configuration. They therefore do not, by themselves, establish inference-time adaptation by a fixed model. The idealized bilinear analysis does not resolve this empirical gap.

While the results support the effectiveness of the overall system, they do not yet establish whether the gains arise specifically from the proposed in-context learning mechanism rather than from additional features, hard negatives, or pretrained initialization. This limits the evidence supporting the paper’s central scientific claim.

  1. Limited evidence for the robustness of the modest gains and the comparability of the evaluations

The overall MRR improvements over FLOCK are:

  • Without fine-tuning: 0.396 versus 0.392.
  • With fine-tuning: 0.434 versus 0.427.

These differences are worth considering, but the five repeated runs vary only the context-sampling seeds while keeping the model checkpoint fixed. They do not capture variability due to training randomness. The manuscript also does not sufficiently clarify the provenance of all baseline results or whether model selection procedures and hyperparameter tuning budgets are comparable.

Given the modest margins over the strongest baseline, these omissions limit confidence in the robustness of the reported advantage.

Questions for authors

Question 1: Do the context labels materially influence predictions?

With the graph encoder, structural features, and training settings held fixed, how much improvement does a correctly labeled support set provide over a support set with shuffled labels? How does the proposed PFN compare with a simpler support-set aggregation method under otherwise matched conditions?

Question 2: How dependent are the gains on TabICL initialization, and how comparable are the baseline evaluations?

Which PFN parameters are updated during KG pretraining? How does the same architecture perform with random initialization rather than pretrained TabICL initialization? Were the baseline results obtained through reruns or taken from previously reported results, and are the pretraining, fine-tuning, and model selection budgets comparable?

Question 3: Does the overall advantage persist when training variability is included?

Can the authors provide repeated experiments that account for training randomness, together with paired per-graph performance differences and uncertainty estimates relative to the strongest baseline? How were the checkpoints and context-related hyperparameters selected, and was this selection based exclusively on validation data?

Reviewer_ZWMUOpenReview ↗
Rating
2
Confidence
4
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

This manuscript proposes KGPFN, a knowledge graph (KG) foundation model that applies a Prior-Data Fitted Network (PFN) to the KG domain. KGPFN samples positive and negative triplets for the relation in a given query. Similar to ULTRA, KGPFN constructs a relation graph and generates query-specific representations for each entity by applying an NBFNet over the relation graph followed by another NBFNet over the original KG. These representations are then used to encode each sampled triplet, which are subsequently fed into a PFN to make the final prediction for the query. Experimental results demonstrate that KGPFN outperforms existing KG foundation models.

Critical strengths

The manuscript is well-written and easy to follow, providing a clear explanation of the proposed model that facilitates reproducibility.

Critical weaknesses
  • The proposed architecture appears to be a straightforward combination of ULTRA's query-specific encoding mechanisms with a PFN. Furthermore, the manuscript does not sufficiently justify why a PFN-style foundation model is necessary or particularly well-suited for the KG domain.
  • The experiments rely on a pretrained PFN, which introduces a discrepancy in the training data compared to the baselines. This raises questions about the fairness of the evaluation.
Questions for authors
  • How does KGPFN perform when the PFN module is trained from scratch rather than initialized with pretrained weights?
  • Given that KGPFN is positioned as a foundation model, could you provide an analysis of its scaling behavior with respect to dataset size or the number of model parameters?
  • Beyond their empirical success on tabular data, what is the core motivation or theoretical rationale for adapting PFNs to knowledge graphs?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.