acceptodds
Under review as a conference paper at ICLR 2027

Vector-Symbolic Object Representations for Causal World Models

Abstract

World models forecast latent states by learning an internal representation of an environment's dynamics. Object-centric variants factor that state into per-object slots, but a slot is an opaque learned vector that requires a trained probe to read, and only supports some edits. We replace the learned slot with a vector-symbolic expression that pairs an object's identity with an encoding of its position, and predict with a graph network over these newly defined slots. Reading a slot, editing a scene and forming the relative position of two objects all become fixed algebraic operations with no trained parameters, and the interaction between any two objects can be toggled inside the predictor. We compare this model with current state-of-the-art, including a coordinate GNN (ref) and C-JEPA on NRI spring-coupled particles and CLEVRER. Beyond single step accuracy, we evaluate the models on closed-loop rollout, recovery of the true coupling graph, and the consequence of removing an object in a setting where ground truth is available. Our model provides superior performance, scoring on rollout compared to ballistic constant velocity prediction versus (coord-GNN) and (C-JEPA), and recovers the coupling graph at ROC-AUC against (coord-GNN) and (C-JEPA).

Primary area:
neurosymbolic & hybrid AI systems (physics-informed, logic & formal reasoning, etc.)
Keywords:
object centric learning, trajectory inference, vector symbolic algebra, spatial semantic pointer, graph neural network

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 4 reviews, above 27% of the others.
Reviewer_S6VzOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes SSP-JEPA, which replaces the learned latent slots in an object-centric world model with fixed vector-symbolic representations that bind an object's identity vector to a Spatial Semantic Pointer (SSP) encoding of its position. The authors evaluate the method on NRI springs and CLEVRER, with ground-truth object states as input, across three evaluation criteria (closed-loop rollout error, interaction-graph recovery, and counterfactual intervention), and report that it outperforms a coordinate-based GNN (coord-GNN) and a version of C-JEPA adapted to consume ground-truth object states.

Critical strengths
  • The idea that a fixed SSP representation makes translation and relative position exact, closed-form algebraic operations, so that the predictor only needs to learn the interaction dynamics, is clear, and the method is described in sufficient detail to be reproduced.
  • The experimental results are consistent. On NRI springs, the proposed method outperforms both coord-GNN and C-JEPA in closed-loop rollout, interaction-graph recovery, and counterfactual intervention. Graph recovery substantially exceeds the proximity baseline, which ranks edges purely by spatial distance, indicating that the model captures physical coupling rather than geometric co-location. On CLEVRER as well, the proposed method achieves the lowest error at contact frames, where constant-velocity extrapolation fails most severely.
Critical weaknesses
  • The authors position this study as an object-centric world model and propose SSP-JEPA as a replacement for the slot representation of C-JEPA, using C-JEPA as the primary baseline. However, all experiments in this paper use ground-truth object states as input and do not address the central challenge in object-centric world model research: extracting object identities and positions from images. Under this setting, the readout without trained probes and the scene editing via algebraic operations, which this paper highlights as advantages over learned slots, do not constitute an advantage over coord-GNN, which uses raw coordinates directly as its state. Regarding C-JEPA as well, while the original method derives slots from video using SAVi or VideoSAUR, the version used here has been adapted to consume ground-truth object states with a coordinate displacement head; therefore, the comparison is against a Transformer predictor operating on coordinates, rather than an evaluation of C-JEPA as published. Although the authors acknowledge this limitation in Section 7, the temporally consistent discrete object identities and precise continuous coordinates required by the proposed representation are precisely the hardest part of extracting objects from images. Therefore, this study should be positioned as a study of representation and loss design for relational dynamics models that take ground-truth states as input, rather than as an object-centric world model; the effectiveness as an object-centric world model claimed in the current title, abstract, and conclusion is not supported by the experiments.
  • There are numerous inconsistencies between the manuscript's descriptions and its claims, making it difficult to assess the reliability of the results. For example, the abstract reports 0.58× for C-JEPA and 0.68× for coord-GNN, whereas Table 1 and Section 5.1 report 0.65 and 0.64, respectively, so the reported values are inconsistent. Furthermore, while the introduction and Contribution 1 state "editing a scene by deletion, translation or insertion", only deletion is actually evaluated. Additionally, the main text contains several typographical errors, which further undermine the manuscript's credibility.
Questions for authors
  • The difference from coord-GNN is the basis for the paper's central claim, but Table 2 shows that replacing the input-side components (edge features and history) with Cartesian coordinates causes only a small drop. In contrast, the "Metric displacement loss" row, which is meant to test the output side, retains the D=127-dimensional phase head while changing only the loss, which produces a failure specific to that configuration: the remaining D−2 phase dimensions receive zero gradient and drift. Consequently, it cannot be determined whether the gain stems from the SSP encoding of the inputs or from the phase-space output and loss. Could the authors add a configuration with SSP inputs, a two-dimensional displacement head, and a metric displacement loss, to identify the source of the gain?
  • In the "Cartesian history" row, graph recovery is unchanged and counterfactual alignment actually improves. Why is it necessary to encode the history as SSPs, and how do the authors interpret this result?
  • In Table 3, only C-JEPA uses a maximum of 3 masked objects and a hidden width of 128, whereas the two graph networks use 2 and 256. Since this makes it impossible to tell whether C-JEPA's weaker performance is due to its architecture or to these settings, could the authors explain the reason for this difference and report results with matched settings?
1 comment

Thank you for your thoughtful review. We have reframed the paper along the lines you suggested.

W1. The paper is now specific in its focus on state representations in dynamics models operating on ground-truth states. The new title is "Spatial Semantic Pointer Representations for State-Based World Models", and the abstract, introduction, and conclusion make clear that states are given. We no longer describe SSP-JEPA as object-centric, reserving that term in the conclusion for full visual pipelines where objects are extracted from video and SSP-JEPA serves as the dynamics predictor. Readout and editing are no longer framed as advantages over coord-GNN, since both operate on given states, and the baseline is now referred to as the C-JEPA predictor on states. We kept "world model" in the sense of a learned transition model on states.

W2. The abstract quoted the baselines at h = 34 (0.58 and 0.68) and SSP-JEPA at h = 20. The CLEVRER numbers labelled h = 20 were also from h = 54. Both are corrected. Translation and insertion are removed, since only deletion was evaluated. We checked all numbers against the run files and fixed the typos.

Q1. We added the requested control: SSP inputs with a 2-D head and metric loss, re-encoded as a valid SSP at every step. It matches the phase head on both datasets, and a coordinate model using only the SSP edge encoding also matches SSP-JEPA. Thus, the accuracy gain comes from the input-side edge encoding rather than the phase output or loss.

Q2. On springs, we find no evidence that encoding history as SSPs improves performance, as differences remain within seed noise. On CLEVRER, the SSP-edge ablation (which uses raw displacement history) also matches SSP-JEPA. We also now note in the text that the Cartesian-history ablation used a learned identity token.

Q3. Mask size 3 was nominally best among {2,3,4}\{2, 3, 4\} for C-JEPA (without a statistically significant difference), and width 128 was its default. When evaluated at the GNN settings (mask 2, width 256), C-JEPA does not improve on any springs metric. On CLEVRER, the configuration mismatch went the other way (C-JEPA used mask 2 while the GNNs used 3); increasing C-JEPA to mask 3 yields a modest improvement (ρ(20)\rho(20) from 0.81 to 0.76; contact error from 0.75 to 0.72), but it still trails SSP-JEPA. Both sweeps are included in Appendix B.

Q3. A sweep was performed for optimal number of masks and it was found that C-JEPA performed best with 3, with no significant difference, and width 128 was the default. At the graph networks' settings (mask 2, width 256) C-JEPA is not better on any springs axis.

Reviewer_Yg37OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces SSP-JEPA, which binds object identity and position using fixed vectors and subsequently employs graph neural networks for future state prediction. This approach decouples identity and positional information, enabling the use of a unified set of algebraic operations to compute object motion and positioning. The authors validated the method's effectiveness through experiments on the NRI springs and CLEVRER datasets—evaluating closed-loop rollouts, graph recovery, and interventions—and conducted ablation studies on the relevant components.

Critical strengths

1.The SSP design is well-structured, with clear operational procedures and definitions, and the innovative aspects are explicitly highlighted. The problem formulation and explanations are accessible, offering a clear comparison regarding identity information decoupling, displacement calculation, and the direct latent vector approach. 2.Three sets of experiments covering diverse scenarios were designed to evaluate the model from various perspectives. Additionally, ablation studies were conducted to compare the effectiveness of different components.

Critical weaknesses

1.Limitations of the evaluation metrics: The closed-loop test employs the ratio against the constant-velocity baseline error as the evaluation metric; however, this metric inherently overlooks the actual absolute error values. While this ratio serves as a reasonable point of comparison—and the shared denominator ensures the correct relative ranking—the ratio decreases as the baseline error grows. Consequently, it fails to reveal the actual variation in absolute error, particularly in later stages, making it difficult to determine whether the error stabilizes. Regarding graph recovery, the use of single-step message ablation is reasonable for avoiding error accumulation during rollouts; however, the absence of multi-step prediction prevents demonstrating long-term effectiveness. For counterfactual evaluation, cosine similarity is used; this metric is insensitive to global scaling, meaning a perfect score can be achieved even when the prediction differs from the ground truth by a scalar multiple, thereby failing to assess the accuracy of magnitude in the approximate direction. It is recommended to include absolute error metrics, conduct multi-step evaluation for graph recovery, and adopt a different evaluation metric for counterfactual analysis.

  1. Issues regarding the ablation study: SSP-JEPA and coord-GNN differ in parameter count, output dimensionality, and training objectives; consequently, the influence of these factors cannot be entirely ruled out. Furthermore, SSP-JEPA did not achieve optimal performance in the Cartesian history setting, meaning it cannot be guaranteed that every component of SSP-JEPA yields the best possible benefit. In the metric displacement loss experiment, while the D-dimensional output was retained, the comparison was based on 2D results; since some high-dimensional directions remained unsupervised and could affect subsequent predictions. An additional control that directly predicts 2D displacement and constructs the corresponding phases would help clarify this distinction.
Questions for authors
  1. Could you provide specific absolute error curves and indicate the scale?
  2. Could you conduct multi-step evaluation for the interaction-graph recovery?
  3. Could you incorporate magnitude-sensitive metrics into the counterfactual analysis? 4.Could you include a baseline that directly predicts 2D displacement and constructs the corresponding phases?
1 comment

Thank you for the review. The three metric additions are in the revision.

W1a and Q1 (absolute error). Appendix C2 reports absolute error with scale. On springs at h = 20 SSP-JEPA is off by 0.185, which is 13% of the mean distance between particles (1.38); constant velocity is at 0.363 and NRI at 0.038. No model's error plateaus. On CLEVRER, the absolute error is ρ\rho times the constant-velocity error, which the caption gives along with the scale.

W1b and Q2 (multi-step recovery). We roll each model forward on its own states for k steps and then score it with the single-step message ablation (Appendix C1). Recovery falls for every masked model: SSP-JEPA goes from 0.864 at k = 1 to 0.734 at k = 20, while the proximity floor rises from 0.563 to 0.671. On true states at the same frames SSP-JEPA stays at 0.86 to 0.88, so the drop comes from rollout error rather than from the readout.

W1c and Q3 (magnitude). We now report a size-sensitive score, ρint=∑j∥ejmodel−ejtrue∥∑j∥ejtrue∥\rho_{\mathrm{int}} = \frac{\sum_j \lVert e^{\mathrm{model}}_j - e^{\mathrm{true}}_j\rVert}{\sum_j \lVert e^{\mathrm{true}}_j\rVert}, next to the cosine, both over all 200 deletions. Predicting no effect scores 1. SSP-JEPA and its ablations reach 0.96 to 1.00, better than C-JEPA (1.16), with the direction partly right (cosine 0.43 to 0.44). No masked model sizes the effect better than predicting no effect, and only NRI does well (0.17). This is stated in the paper.

W2. We added a parameter-matched version of coord-GNN. The Cartesian-history row is within seed noise, and the paper now states that we find no evidence the SSP history in its current form helps. In the metric-loss row the other D − 2 phases receive no gradient; projecting them out at rollout removes the collapse (ρ(20)\rho(20) from 12.2 to 0.55).

Q4. Added: SSP inputs, a 2-D displacement head, and the phases constructed from that displacement at every step. It is not distinguishable from the phase head on springs (ρ(20)\rho(20) 0.55 vs 0.51) or on CLEVRER.

Reviewer_xMkuOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes SSP-JEPA, an object-centric world model using vector-symbolic representations to encode object identities and spatial positions. A graph network predicts object interactions and motion. The evaluation considers closed-loop prediction, interaction-graph recovery, and counterfactual interventions on NRI springs and CLEVRER.

Critical strengths

The use of structured vector-symbolic representations is well motivated. Encoding identity and position algebraically offers a potentially useful alternative to unconstrained latent slots, particularly for interpreting spatial relationships and manipulating object states.

I appreciate the evaluation beyond prediction error. Interaction-graph recovery and counterfactual interventions provide more direct evidence about what the learned dynamics capture. The reported graph-recovery and rollout improvements are promising.

Critical weaknesses

The comparison does not fully isolate the benefit of vector-symbolic representations. SSP-JEPA uses approximately 915K parameters on NRI, compared with 403K for the coordinate GNN. A capacity-matched comparison, ideally including Fourier-feature or other structured coordinate embeddings, is needed before attributing the improvement to the proposed algebraic representation.

The phase prediction head produces unconstrained high-dimensional phase updates. Although these preserve unit-modulus Fourier coefficients, they are not necessarily valid spatial semantic pointers corresponding to a single physical displacement. This distinction matters for the claims of exact geometric structure over long rollouts. Also, the evaluation assumes ground-truth object states rather than learning them from images.

Questions for authors

Can the authors provide a parameter-matched coordinate baseline and an ablation where the network predicts a two-dimensional displacement that is then encoded as a valid SSP? These experiments would help distinguish the benefit of the representation from feature dimensionality and training regularization.

1 comment

Thank you for your review and constructive questions. Both requested experiments are now included in the revision.

W1 and Q1 (capacity and structured embeddings). We added three controls to isolate capacity from representation:

  • SSP-edge ablation: A coordinate model with Fourier features of relative position on its edges (ϕ(pj−pi)\phi(p_j - p_i)) matches SSP-JEPA on every axis.
  • Parameter-matched coord-GNN: Lifting raw inputs to 127 dimensions via an extra ReLU layer (916K parameters on springs, 753K on CLEVRER) reaches 0.907 springs recovery (vs. 0.919) and 0.64 on CLEVRER contact frames (vs. 0.46). It retains SSP-JEPA’s phase head, which our 2-D-head control confirms does not affect accuracy.
  • Frequency-set and length-scale sweep (Appendix C3): Replacing the Gaussian frequency set with a hexagonal grid and sweeping the edge length scale shows that ℓ=0.2\ell = 0.2 minimizes contact error for both, and the two tie at their respective optimal widths.

With random Θ\Theta, an SSP is a random Fourier feature map up to an orthogonal transform (Section 2.2). Thus, the accuracy gain stems from a Fourier encoding of relative position, and on contact events a fixed periodic encoding outperforms a learned lift of equal capacity.

W2 (phase head). The phase head is unconstrained, and Section 3 now notes that a predicted step corresponds to the SSP of a single displacement only when its phases lie in the column space of Θ\Theta. Empirically, predicted phases stay within 0.9% of that subspace, and projecting them back at every step alters ρ\rho by at most 0.003 over 34 and 54 steps. Moreover, the 2-D head from Q1 (re-encoded as an exact SSP each step) is indistinguishable from the phase head on both datasets. Regarding ground-truth states, we have updated the title and framing accordingly (see the summary comment).

Reviewer_15GzOpenReview ↗
Rating
2
Confidence
3
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

The paper proposes SSP-JEPA, an object-centric dynamics model whose slots are fixed HRR expressions, binding an identity vector to a Spatial Semantic Pointer (a random Fourier-phase encoding) of the object's position. A message-passing GNN uses unbound relative-position edges Eij=ϕ(xj−xi)E_{ij} = \phi(x_j - x_i) and displacement histories to regress Fourier phases, which advance each slot by unitary circular convolution. Training uses C-JEPA's masked-history objective with a phase-space loss and input-noise regularization. The authors also propose a probe-free evaluation protocol with three diagnostics: closed-loop rollout error relative to constant velocity, graph recovery via single-edge message ablation, and counterfactual alignment after object deletion against a re-simulated scene. On NRI springs, SSP-JEPA outperforms a coordinate GNN and a state-based C-JEPA on all three axes, and on state-based CLEVRER it leads on rollout and contact-frame error. Ablations attribute graph recovery to the SSP edge encoding and rollout stability to the phase loss and noise term, and an appendix reports better extrapolation to larger object counts.

Critical strengths
  1. A probe-free, simulator-grounded evaluation protocol. Scoring rollout against constant velocity, graph recovery by single-edge message ablation, and counterfactual alignment against re-simulation avoids conflating representation quality with decoder capacity and exposes the inertial shortcut that one-step losses hide. The protocol is reusable beyond this model and is the paper's most durable contribution.
  2. Consistent gains with a mechanistic explanation on springs. SSP-JEPA leads on all three axes on NRI springs and on rollout and contact frames on CLEVRER. The ablation also explains part of the result: a decoded-displacement loss confines gradients to the 2-D column space of Θ\Theta, leaving the remaining phases free to drift, which accounts for the rollout failure of that variant. This is a concrete, transferable insight for using SSP encodings in learned dynamics.
Critical weaknesses
  1. The headline claim that fixed VSA slots outperform learned slots is not directly tested by the current experiments. Section 7 acknowledges that perception is assumed, but the abstract and conclusion do not reflect this. All models consume ground-truth 2-D states, and C-JEPA is stripped of its learned slot encoder, so the comparison is between encodings of known coordinates, not between learned and fixed slot spaces. Within that setting, the gains may reflect the known benefit of Fourier positional features rather than anything vector-symbolic. The binding to OBJi\mathrm{OBJ}_i may also not contribute to the predictions as currently used: it is unbound immediately to form Eij=ϕ(xj−xi)E_{ij} = \phi(x_j - x_i), identity seems to enter only through the mask token, and no ablation removes it. The claimed readout and editing advantages are likewise equally parameter-free for coord-GNN on ground-truth states, so they are never demonstrated where they would matter.
  2. The baselines and the NRI reference may not support the "state-of-the-art" claim. Table 1 labels NRI as "trained with trajectory-level edge supervision," and Section 4 calls graph recovery its "training objective," but the original NRI is an unsupervised VAE trained on an ELBO with latent edges. Its reported AUC (0.995) is close to what unsupervised NRI achieves on 5-particle springs. If the unsupervised variant was used, NRI is a legitimate self-supervised competitor that dominates every axis by a wide margin, contradicting the SOTA claim. The decisive controls are also absent: a coord-GNN with Fourier features of relative position (same Θ\Theta, no binding or unitary update) and a parameter-matched coord-GNN.
Questions for authors
  1. Was NRI trained with edge labels, or as the original unsupervised VAE? How many seeds were used (Table 1 and Table 3 disagree), and why is its AUC reported with zero standard error? If NRI is unsupervised, how do the SOTA claims hold given Table 1?
  2. Can you report (a) coord-GNN with ϕ(xj−xi)\phi(x_j - x_i) edges and a 2-D displacement target re-encoded each step, and (b) a coord-GNN parameter-matched to SSP-JEPA? If these close the gap, what remains specifically vector-symbolic about the contribution?
  3. How are SijS_{ij} and the deletion intervention implemented for C-JEPA: masking attention from jj, dropping the token, or zeroing its value vector? Does this choice affect its AUC or alignment?
  4. The metric-loss failure suggests predicted phases leave the SSP manifold. Does projecting predicted phases onto col(Θ)\mathrm{col}(\Theta) at each step (decode Δx\Delta x, then re-encode) match or beat SSP-JEPA, and if so, what does regressing DD phases buy over predicting Δx\Delta x?
  5. Where are the translation and insertion edits promised in Section 1, and how do they score against re-simulation? Appendix A also allocates CLEVRER videos to recovery, but no CLEVRER recovery result is reported; what does it show?
  6. The Cartesian-history ablation matches or nominally exceeds the full model on intervention alignment, and the metric-loss ablation nominally improves AUC. How do the authors reconcile this with the claim that the SSP representation drives causal fidelity?
1 comment

Thank you for the thorough review. The major updates are outlined in the summary comment above; and we address each point made in this comment.

W1: Because all models take ground-truth states, our comparison strictly evaluates different encodings of known coordinates, and the title, abstract, and conclusion now state this clearly. Regarding Fourier features: an SSP with random phases is a random Fourier feature map—specifically, a fixed orthogonal transform of cosine and sine features with frequencies Θℓ\frac{\Theta}{\ell} (now written explicitly in Section 2.2). Thus, an SSP edge and a Fourier-feature edge represent the same encoding in different bases. The SSP-edge ablation, which retains only ϕ(pj−pi)\phi(p_j - p_i) on the edge, matches SSP-JEPA on every metric, isolating the gain to that relative-position encoding. Furthermore, OBJi\text{OBJ}_i is unbound from the edge and history and reaches a learned layer only as the mask token; replacing it with a learned per-object token leaves all scores within seed noise (recovery 0.922 vs. 0.919). We have dropped the readout and editing claims over coord-GNN, retaining only the finding that keeping the full state in the vector-symbolic algebra costs no accuracy compared to a coordinate model with the same edge encoding.

W2 and Q1. Our NRI baseline is the original published VAE (ELBO, two edge types, no edge supervision). The misdescriptions in Section 4 and the Table 1 caption have been corrected. The zero SE was a rounding artifact; evaluated on the exact same 100 episodes and scorer as the other models, NRI achieves 0.9993±0.00030.9993 \pm 0.0003. Since NRI leads every springs axis—unsurprisingly, given that its static-graph assumption matches the springs data-generation process—we removed the SOTA claim and noted this alignment in the setup. On CLEVRER, NRI matches constant-velocity error on contact frames (0.97 vs. 0.46 for SSP-JEPA).

Q2. (a) corresponds to the new SSP-edge ablation, and (b) corresponds to the parameter-matched coord-GNN (which adds a learned lift layer to match parameter count).

Q3. In C-JEPA, all tokens belonging to object jj are masked from attention in every layer via a key padding mask for both SijS_{ij} and deletion; Section 3.4 now specifies this. Of the two alternatives you mentioned, dropping jj's tokens yields an identical score (0.810), while zeroing their value vectors improves recovery to 0.833 (p=.03p = .03) at the cost of worse deletion performance. Both remain well below SSP-JEPA (0.919).

Q4. Projecting SSP-JEPA's predicted phases onto the column space of Θ\Theta at every step shifts ρ(20)\rho(20) by −0.0001-0.0001 on springs and by −0.003-0.003 over 54 steps on CLEVRER. On the metric-loss model, this projection prevents rollout collapse (ρ(20)\rho(20) drops from 12.2 to 0.55). Meanwhile, a 2-D head re-encoded as an exact SSP at every step performs on par with the phase head on both datasets (springs 0.55 vs. 0.51). Regressing all DD phases therefore offers no measurable accuracy advantage.

Q5. We explored translation and insertion prior to submission, but neither produced effects distinguishable from chance against re-simulation, so we removed both from the introduction and contributions. On CLEVRER, collision labels turn out to be predictable from proximity alone (0.93 ROC-AUC, outperforming every model) and therefore do not meaningfully test graph recovery. This is now explained in Section 4, and the unused allocation in Appendix A has been removed.

Q6. Both the Cartesian-history row's higher deletion score and the metric-loss row's higher AUC fall within seed noise, and the revised text reports them accordingly. We removed the claims that SSP history and phase loss drive causal fidelity, focusing our claim instead on the edge encoding.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary of changes in the revised PDF

We thank all reviewers for their time and detailed feedback. Across the reviews, several key points converged, and we have updated the paper accordingly. Below is a summary of the main changes:

Scope and title. All models operate on ground-truth object states, which is now explicitly stated in the title, abstract, introduction, and conclusion. The new title is "Spatial Semantic Pointer Representations for State-Based World Models". We dropped "object-centric" because that term typically refers to extracting objects from raw visual scenes (and maintaining consistent identities and coordinates), whereas we use "world model" in the sense of a learned transition model over states. We also removed the claim that SSP-JEPA's readout and editing offer an advantage over a coordinate-based model given the same states.

coord-GNN. The purpose of the coord-GNN baseline was to show a full ablation of VSA representation and operations from SSP-JEPA by replacing SSPs with coordinate counterparts (i.e. a 2d raw displacement history, 2d relative distances, a learned identity token, a 2d displacement head, and a coordinate loss). This is now explicitly stated in the text. The original submission inadvertently differed in how the noise term (and its loss) was implemented; we fixed this discrepancy, which updated the springs results (rollout error from 0.64 to 0.54; graph recovery from 0.866 to 0.887). CLEVRER numbers remain unchanged.

Added controls (Tables 1 and 2, and a new CLEVRER table):

  • SSP-edge ablation: coord-GNN using the SSP of relative position, ϕ(pj−pi)\phi(p_j - p_i), on its edges, without binding, a phase head, or unitary updates.
  • Parameter-matched coord-GNN: coord-GNN with a learned 127-D ReLU lift on its edge and history inputs (916K parameters on springs, 753K on CLEVRER).
  • 2d head on SSP inputs: SSP inputs paired with a 2d head, re-encoded as an exact SSP at every rollout step.
  • Projected phase rollout: The metric-loss model with its predicted phases projected onto the column space of Θ\Theta during rollout.

The SSP-edge ablation matches SSP-JEPA across every axis on both datasets (springs recovery 0.914 vs. 0.919; CLEVRER contact error 0.46 vs. 0.46). This indicates that the accuracy gain comes from encoding relative position as an SSP on the GNN edges, while keeping the rest of the state in the vector-symbolic algebra incurs no loss in accuracy. The SSP edge raises springs graph recovery from 0.887 to 0.914 and lowers the CLEVRER contact error by half (0.93 to 0.46), whereas the parameter-matched coord-GNN reaches 0.64 at contacts.

Corrections and clarifications:

  • NRI: Our baseline is the original published unsupervised NRI model (trained via ELBO without edge labels); Section 4 and the Table 1 caption were misworded and have been fixed. Because NRI's inductive bias matches how the springs data is generated (one static interaction graph per window with a shared pairwise interaction function), it leads on every springs metric, and we have removed our SOTA claim. On CLEVRER, where collisions are brief events between changing pairs rather than a static graph, NRI stays at constant-velocity error on contact frames (0.97 vs. 0.46 for SSP-JEPA).
  • Horizon labels: The abstract previously quoted baselines at h=34h = 34 alongside SSP-JEPA at h=20h = 20, and the CLEVRER numbers labeled h=20h = 20 were actually from h=54h = 54. Both are corrected (CLEVRER at h=20h = 20: SSP-JEPA 0.57, coord-GNN 0.96, C-JEPA 0.81).
  • Unimplemented edits: Translation and insertion were not evaluated in the paper and have been removed.

New Appendix C: Includes multi-step graph recovery, absolute rollout error with physical scale, edge length-scale and frequency-set ablations, and contact error at wider tolerances.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.