acceptodds
Under review as a conference paper at ICLR 2027

R-JEPA: A Relational World Model That Predicts Before It Observes

Abstract

Relational reasoning from a few examples is usually studied by naming the shared relation or scoring candidate answers, while joint embedding predictive architectures (JEPA) hold that predicting a target without negatives is the better objective. These views obscure what a relation does to a state, when that operation can stay fixed, and when predicting the state beats choosing among candidates. We argue that relational reasoning is better understood as an inference cycle in latent space, where a relation induced from example pairs predicts an unobserved state that is reused as the next input and checked against later observations. Based on this view, we propose R-JEPA, a relation reader and a transition operator that returns a state, trained by latent prediction or classification or used without learned parameters. Across knowledge graphs and systems described to language models, the predicted state follows edits of the knowledge, composes over grounded steps and ranks facts missing from a 2019Wikidata snapshot as well as the true relation does. How consistently roles form directions, measured across seven language models, decides whether the transition must be learned, and latent prediction needs no negatives, while classification with one negative per query trails it by up to 0.13. These results suggest that relational reasoning should be evaluated by prediction before observation, and that latent prediction pays where classification cannot see enough negatives.

Primary area:
unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords:
relational world models, joint-embedding predictive architectures, few-shot relational reasoning, latent prediction, negative sampling, knowledge editing, link prediction

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_YCYLOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

This paper proposes R-JEPA, a framework that approaches relational reasoning from a world-model perspective by treating relations as latent state transitions. Given a few support examples, R-JEPA infers an unnamed relation and applies it to a new query to predict the corresponding latent state, which can subsequently be retrieved and reused for multi-step reasoning. To investigate the effectiveness of this formulation, the authors conduct experiments across knowledge graphs and language model representations under various settings, including fact deletion, counterfactual knowledge editing, renamed and invented systems, multi-hop reasoning, and temporal knowledge splits. The paper further analyzes when relational transitions can be performed without learning, based on the consistency of relational directions in the representation space. Finally, it compares JEPA-style latent prediction with classification-based objectives, examining their relative effectiveness under different negative-sampling conditions and candidate space sizes.

Critical strengths

S1. Comprehensive evaluation across diverse settings.

The paper conducts extensive experiments across diverse settings, including fact deletion, counterfactual knowledge editing, multi-hop reasoning, and temporal knowledge splits. These evaluations provide useful insights into the behavior of relational reasoning models under different conditions.

Critical weaknesses

W1. Insufficient motivation and empirical justification for the world-model perspective

The paper motivates R-JEPA by arguing that existing relational reasoning approaches primarily focus on relation identification or candidate scoring, rather than predicting reusable latent states. However, it does not clearly establish what concrete limitations of existing approaches necessitate this reformulation. For instance, few-shot relational learning methods such as MetaR (Chen et al., 2019) infer relations from support examples, while inductive knowledge graph models such as InGram (Lee et al., 2023) and knowledge graph foundation models (Galkin et al., 2024) already investigate generalization to unseen entities, relations, or graphs. The paper does not sufficiently explain which capabilities of these existing approaches are fundamentally inadequate from the proposed world-model perspective. More importantly, the empirical evaluation lacks direct comparisons with these relevant existing approaches. Although the experiments demonstrate knowledge dependence through counterfactual edits, state reuse through multi-hop reasoning, and prediction of unobserved facts, it remains unclear whether these properties are specific to R-JEPA or can also be achieved by existing inductive relational models under the same evaluation protocols. Without such comparisons, the experiments do not sufficiently justify the necessity or distinctiveness of the proposed world-model formulation.

W2. Limited methodological novelty and unclear empirical benefits

The proposed R-JEPA framework exhibits substantial methodological overlap with existing relation prediction and reasoning models. Its core pipeline of inferring relational representations from examples, applying relation-based transformations in latent space, and retrieving entities through embedding similarity is already widely adopted in prior work like logical query reasoning on knowledge graphs. The main methodological distinction appears to be the incorporation of a JEPA-style latent prediction objective, which is itself well established. Furthermore, the empirical results do not demonstrate consistent advantages of the proposed latent prediction approach. Classification matches or outperforms latent prediction in several settings, while parameter-free predicate matching also outperforms learned R-JEPA variants in some experiments. Although consistently achieving the best performance is not strictly necessary, the limited methodological novelty makes it particularly important to demonstrate clear empirical benefits or distinctive capabilities over existing approaches, which remain insufficiently established.

Questions for authors

Q1. Could the authors evaluate recent KG foundation models (e.g., ULTRA) under the same knowledge editing and counterfactual settings to investigate whether they exhibit the limitations highlighted in this paper? How does R-JEPA compare with these models in terms of performance under these settings?

Reviewer_KnQCOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

The paper studies relational reasoning through the lens of joint embedding predictive architectures (JEPA). It argues that relational reasoning should be understood as an inference cycle in latent space, and based on this idea, introduces a relational JEPA (world) model (R-JEPA). R-JEPA consists of a relation reader and a transition operator which returns a state. The authors test this architecture either completely without learned parameters or trained with latent prediction or classification losses. They further evaluate R-JEPA on datasets with counterfactual edits, systems that have been renamed or invented for the test, and on facts missing from a 2019 snapshot of Wikidata (amongst others). They use entity embeddings of knowledge graphs and hidden states of language models, such as Qwen2.5-7B and Pythia-6.9B. The authors derive three main findings from these experiments: 1) the predicted state behaves as we would expect a world model state to behave, 2) the geometry of the state decides if a transition must be learned, and 3) training with latent prediction indeed needs no negatives, while training with classification needs many.

Critical strengths
  • Novelty and theoretical grounding: JEPA-like architectures have - to the best of my knowledge - not been investigated in the context of relational reasoning, which makes this a novel and potentially impactful study. The motivation for the approach is also reasonably clear. Propositions 1 and 2 in section 4 provide additional theoretical insights into the R-JEPA framework and help motivate the organization of the experiments in section 5.
  • Experimental design: the experiments presented in the main text seem sensible and connect more or less directly to a) the two propositions in section 4 and b) the three main findings the paper lists in the abstract and in the introduction. A better notation for table 1 or perhaps splitting that table into multiple smaller tables could have made the results more accessible though.
Critical weaknesses
  • Writing and overall structure: the abstract and introduction frequently and repeatedly make use of terminology that is only introduced much later (mostly in section 3/ on page 4), e.g. “relational reasoning” or “state”. The writing, especially in these early sections, often makes it hard to follow the overall narrative and to connect the various ideas presented by the authors. Figure 1 somewhat helps with this and nicely visualizes some of the key concepts, but in my opinion appears too late (page 3) to mitigate any initial confusion.
  • Dated LLMs and ICL comparison: Qwen2.5 and Pythia are somewhat dated at this point and raise the question to what extent the results presented in the paper still hold for more recent models, such as the Qwen3.5 family. Some results in the appendix (table 11) also seem to suggest that in some settings, in-context learning can be competitive with or even outperform R-JEPA. It would be interesting to see to what this trend looks like with more state-of-the-art open-source models.
Questions for authors
  • The models considered here (Qwen2.5 and Pythia) are somewhat dated. Why not use more recent models like Qwen3.5 or at least Qwen3?
  • Somewhat related: in section 5.1 it says “we use Qwen layer 21 and Pythia layer 8, fixed by a preregistered rule” - why these layers? What’s the preregistered rule?
  • Higher-order relations: it seems that most (all?) relations considered in the paper are binary, or are compositions of multiple binary relations. Have the authors conducted any experiments with higher-order relational reasoning? If not, can the R-JEPA framework be extended to those?
Reviewer_wWAeOpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

This paper studies how to infer a relation from example pairs and predict an unobserved correspondence. Given that existing approaches do not sufficiently test whether a predicted state follows edits of the knowledge, supports the next step, and agrees with later observations, they propose R-JEPA, which consists of a relation reader, a transition operator, and entity retrieval. The idea behind it is to induce an unnamed relation from examples and apply it to a new query’s knowledge to predict a target state.

Critical strengths
  • The paper evaluates relational reasoning through prediction before observation, using deleted facts, counterfactual edits, invented systems, and later observations, which makes sense.
  • It identifies conditions under which a fixed transition suffices and latent prediction pays, supported by two propositions and controlled experiments.
  • It reports every preregistered criterion, including failures, and narrows its claims accordingly.
Critical weaknesses
  1. The term ''world model'' is misleading. The experiments demonstrate relational prediction, grounded multi-hop inference, and static evaluation against later facts, rather than an integrated prediction-observation-update cycle. Please consider a more precise term.
  2. The paper frames relational reasoning as applying operators and reusing intermediate states, but does not distinguish this contribution from NLM[1]’s layered predicate inference or R5[2]’s recurrent rule composition. Neither is discussed or evaluated. A comparison on a shared graph reasoning task would help establish what latent state prediction adds to existing compositional reasoning methods. Or at least, the authors should position their contribution against these neural-symbolic approaches to avoid overstating its novelty.
  3. The experimental comparisons with the closest prior methods are limited. In particular, MetaR is acknowledged as closely related but is not evaluated. The graph experiments rely mainly on simple baselines and internal variants, leaving unclear what R-JEPA adds over existing few-shot and compositional relational reasoning methods.
  4. The central contribution is unclear. The paper shifts between a broad claim about relational reasoning as a world-model inference cycle and narrower findings about representation geometry and training objectives. It does not explain how the latter establish the former. Knowledge following, state reuse, and later-fact prediction are tested with different operators in separate settings, rather than demonstrated as one working cycle.
  5. Code implementation concerns:
    • The main graph results seem not follow the stated prediction pipeline. Predicate matching ranks decoder logits directly on WikiTopics and the time split. It does not retrieve entities from the predicted expected state.
    • Tie handling seems to contradict the paper. The graph and time-split evaluators count tied golds as rank 1, even when an incorrect candidate shares the highest score. The paper states that such ties count as errors.
    • For queries with more than 10 gold answers, the code computes precision@10 rather than the stated R-precision.

[1] Dong, et al. "Neural logic machines." ICLR 2019

[2] Lu, et al. "R5: Rule discovery with reinforced and recurrent relational reasoning." ICLR 2022

Questions for authors
  1. Can you discuss and compare R-JEPA with NLM and R5? R-JEPA and NLM both apply learned operators to knowledge representations and reuse intermediate outputs for further inference. R-JEPA and R5 both compose relational operations over multiple steps. Since you claim that R-JEPA’s predicted states support multi-step reasoning, could you evaluate its compositional generalization against NLM and R5 on a shared graph reasoning task, such as CLUTRR or GraphLog, with matched inputs and supervision?
  2. Can you explain the mismatch between the code and the manuscript?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.