acceptodds
Under review as a conference paper at ICLR 2027

Graph Representations in J-Space

Abstract

Language models appear to have a global workspace: a small part of their internal state that holds active concepts and supports multi-step reasoning. The Jacobian lens reads this workspace as individual words, but reasoning also requires knowing how things are related. Answering “Who is the teacher of the banker of John?” requires following a graph whose nodes are people and whose edges are relations. We investigate whether the workspace holds these edges or only the nodes. We compare questions about the same story that differ in one relation word, and edit the workspace to test which changes affect the answer. Across multiple models, the workspace holds both nodes and edges, but treats them differently. Editing a person affects the answer where the name appears and at every model depth; editing a relation matters only at the relation word and in the middle layers. The two are never available at the same place and time, so the workspace never holds the connected graph. By the time the answer forms, the relation that determined it has left the workspace, although it remains elsewhere in the model. Workspace inspection at the moment of answering can therefore miss the relation responsible for the answer. Our findings motivate interpretability methods that trace relational information across token positions and layers, rather than relying on a final workspace snapshot.

Primary area:
foundation or frontier models, including LLMs
Keywords:
LLM, graph

Reviews

Among:
Shape:
1234this paper, Top 1%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.67 over 3 reviews, above 99% of the others.
Reviewer_K7eFOpenReview ↗
Rating
4
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper investigates how language models represent relational information during multi-hop reasoning in J space. Through controlled graph reasoning tasks and causal interventions, the authors find that entities remain accessible across layers but relations are more localized in middle layers. The findings suggest that the workspace mostly carries entities without preserving the relations.

Critical strengths
  1. Well motivated research question and careful experimental design. It studies and interesting question of how language models represent structured information during reasoning. The design of minimal edge-pair and the specific causal interventions makes the results convincing.
  2. Comprehensive analysis and extensive experiments. Lens-coordinate interventions, subspace transfer, and probing provide a coherent picture. The experiments also span 8 models multiple layers. I also appreciate the analysis connecting workspace capacity to reasoning failures, which gives the representational findings a meaningful behavioral interpretation.
Critical weaknesses

The paper argues that the workspace never contains a bound graph because entity and relation interventions do not have overlapping causal effects. But this remains unknown if we move beyond towards more distributed or nonlinear representations of relational binding.

Questions for authors

In Section 4.7, the authors find that relational information shifts from J space to directions outside J space in Qwen models but not Gemma. Do the authors have any hypotheses about what explains this difference?

Reviewer_C2deOpenReview ↗
Rating
4
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

The authors investigate whether the global workspace, J-space, in language model holds relational structure in addition to the entities during multi-hop reasoning. Using minimal edge pairs and a suite of causal interventions aligned with J-lens directions, the authors find a double dissociation. Entities are broadcast across layers and positions while the relations are local to their token positions. They find that the nodes and edges never appear jointly in the workspace and that the relations leading to the answer leave the J-space before they are formed.

Critical strengths

Clearly stated hypotheses, methodology, and position classes. Figures and selectivity metrics focus on within-category comparisons to avoid confounds near the output. The work provides guidance for interpretability practice, showing how snapshotting only the final workspace risks missing the causal relational information and tracing across positions and layers is necessary

Critical weaknesses

None

Questions for authors

How were the unrelated relations chosen for the matched controls and how sensitive are results to this choice?

Reviewer_ZaZmOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

This work develops a controlled experimental framework to investigate how LLMs represent and use entities and relationships during multi-hop reasoning. Using J-lens-based interventions on synthetic relational queries, the authors find that entity representations have broader propagation and more persistent causal influence, while relationship representations influence answers mainly at relationship-word positions within a middle-layer window. They interpret these findings as evidence that the measured workspace exposes entities and relationships separately, rather than maintaining a fully bound graph.

Critical strengths
  1. The minimal-pair design isolates the queried relationship by changing only one relation word while preserving the surrounding facts and entities. Applying interventions separately across token positions and layers, with matched controls, provides a well-controlled approach to distinguishing the causal roles of entity and relationship representations.

  2. The study provides evidence that entity and relationship representations have different patterns of causal influence within the measured workspace. Combining interventions, subspace transfers, and probes also provides complementary evidence that the workspace readout can omit relationship information relevant to the answer.

Critical weaknesses
  1. All evaluated models have at most 4B parameters, leaving generalization to larger models unestablished. Moreover, the proposed causal handoff outside J-space appears in the tested Qwen models but not the tested Gemma models. Larger-model replication across both families would strengthen the broad claims about workspace organization.

  2. The evaluation primarily uses closely related synthetic templates, single-token concepts, and 60 items per condition by default. These choices support controlled comparisons but provide limited evidence for general relational reasoning. In particular, failure to detect an intervention effect under these conditions does not establish that the workspace never contains bound graph structure. Broader task variation and stronger evidence for absence claims are needed.

Questions for authors

The restriction to models with at most 4B parameters is a major limitation. Publicly fitted J-lenses are available for larger models, including Qwen3-14B and Gemma-3-12B, yet the manuscript does not explain why these models were excluded. The tested models are too small to be considered as a LLM used for reasoning. Larger models may use different representations or reasoning mechanisms, even when answering the same relatively simple queries correctly. Consequently, the observed organization of entity and relationship information in small models may not generalize to larger models.

Although the authors acknowledge the model-size limitation, acknowledging it does not establish that the broad conclusions remain valid beyond the tested range. This concern is particularly relevant because some proposed mechanisms already differ between the tested Qwen and Gemma models. Without such evidence, the abstract and conclusion should more explicitly restrict their claims to the evaluated models and tasks.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.