acceptodds
Under review as a conference paper at ICLR 2027

OCCAM: Object-Centric Causal World Model via Vision-Language Graph Grounding

Abstract

Latent world models trained via joint-embedding predictive architecture (JEPA) excel at planning from pixels, but their transition functions are monolithic, lacking explicit representations of objects and interaction dynamics necessary for controllable and explainable embodied planning and reasoning. To address this issue, we introduce Object Centric CAusal World Model (OCCAM), a framework that integrates language-grounded causal structures directly into the transition function. OCCAM first uses open-vocabulary segmentation and tracking to obtain object-level latent states without hand-labeled masks. Then, OCCAM queries a vision-language model over motion-salient frames to build a sparse causal graph, validated by an independent conditional mutual information estimator. By combining language-grounded object abstraction with verified causal graph discovery, OCCAM enforces sparse, explainable state updates that isolate task-irrelevant entities, mitigating the extensive data requirements in classical causal discovery. We systematically evaluate OCCAM, analyzing dense, object-centric, and causal transition regimes across tasks of increasing visual and relational complexity. Experimental results demonstrate that OCCAM achieves a highly interpretable, object-centric latent space while preserving goal-conditioned planning accuracy. Crucially, on the complex multi-object LIBERO benchmark, OCCAM achieves superior downstream task control compared to dense and object-centric baselines, indicating that the structured causal reasoning is essential for reliable control as scene complexity scales.

Primary area:
applications to robotics, autonomy, planning
Keywords:
World Models, Causality, Vision-Language Models

Reviews

Among:
Shape:
1234this paper, Bottom 25%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.75 over 4 reviews, above 25% of the others.
Reviewer_T4zXOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies how to add explicit object and interaction structure to a JEPA-style latent world model. Qwen3-VL and SAM3 provide object names and tracked bounding boxes, while a VLM proposes a graph from motion-salient frames. Object-specific readouts and graph-restricted predictors then regularize the shared representation. A trajectory-based predictive-error test is used to assess graph agreement. The evaluation covers Push-T, OGBench-Cube, and ten LIBERO-Goal tasks, with CEM planning in the first two environments and behavior cloning on a frozen encoder in LIBERO.

Critical strengths
  1. The pipeline offers a practical alternative to discovering object slots entirely from scratch. Section 4.1 specifies confidence-based interpolation and object-wise episode filtering, and Appendix F.3 provides the VLM and an example graph-discovery prompt. These details make the proposed use of semantic priors reasonably concrete.
  2. The dense, object-centric, and causal variants share a backbone and training recipe. Figure 4 reports eight-seed planning results on Push-T and Cube: OCCAM obtains 89.5 ± 2.7% and 76.0 ± 3.9%, compared with LeWM at 90.0 ± 3.1% and 75.9 ± 4.4%. Thus, the added structure appears compatible with strong planning performance in these settings, even though it does not improve it.
Critical weaknesses
  1. The graph validation does not establish the central causal claim. The causal validation is not fully convincing. Appendix F.1 estimates “CMI” using a held-out MSE gap, which measures incremental predictive value rather than conditional mutual information without additional assumptions. Appendix C also acknowledges missing orientation and action–manipulator collinearity, weakening the identifiability conditions. On LIBERO, graph agreement is only F1 = 0.52, with precision 0.37 (Figure 7). The text and caption appear to use different reference graphs when defining recall and precision. Please clarify the metric definitions and validate direct dependencies through controlled simulator interventions.
  2. The stated architectural guarantees are not demonstrated by the formulation. Equation (8) restricts inputs to selected object projections, but each projection WnztW_n z_t comes from the same global representation. Box supervision therefore does not guarantee that non-parent information is excluded, or that grounding errors remain confined to one object as claimed in Appendix E.2. The implementation is also unclear: Section 4.2 replaces the dense predictor, whereas Algorithm 2 retains it and adds an auxiliary loss; Eq. (3) excludes self-parents despite Appendix C assuming self-loops. Clarifying the rollout path and comparing against sparsity-matched random or simple graphs would strengthen the contribution.
  3. The strongest result evaluates representations rather than causal planning. The main improvement, 57.0% versus 51.2% on LIBERO (Figure 4), uses a frozen encoder with a BC head rather than world-model planning. It therefore demonstrates representation gains, not directly better causal rollouts. Deterministic evaluation does not eliminate variability from pretraining or BC-head training, so multiple training seeds are needed. Table 3 also shows no improvement on bowl tasks (0.548 versus 0.560), where grounding fails. A corrected-grounding control and multi-object planning evaluation would help establish whether the gains come from causal structure rather than robustness to one annotation failure.
Questions for authors
  1. Can you specify the exact causal-mode training and rollout path, including self-state inputs, object-state dimensions, the graph-to-embedding loss, and parameter counts for all three variants?
  2. Which graph is the reference in Figure 7? Please provide the complete edge sets, threshold sensitivity, and evidence from controlled simulator interventions that separates direct from mediated effects.
  3. Does the LIBERO gain persist across pretraining and BC-head seeds, after correcting the bowl grounding, and relative to simple or random graphs matched for sparsity and supervision?
  4. Can you demonstrate a benefit of the causal predictor itself on a multi-object planning task, and test predicted object isolation under distractor or object interventions?
4 comments
AuthorsOpenReview ↗

T4zx_response_1

Reviewer T4zX

We thank the reviewer. We respond to the main concerns as follows. (1) Metric definitions: the VLM edges are the reference for recall and precision, and we have corrected the Fig. 7 caption in the revised manuscript; we give the full edge sets and threshold-sensitivity tables for all three environments (Q2). (2) Direct versus mediated effects: simulator interventions on Push-T show that the action reaches the block only through the pusher (Q2). (3) Implementation: we state the exact training and rollout path, how self-state is handled, and the parameter counts of all three variants (Q1). (4) Isolation: the guarantee is at the level of what each transition head reads, not of the shared embedding (Q4 and our reply to Reviewer YVrE, Q3).


Response 1 (W1, Q2): graph validation, reference graph, orientation, collinearity and direct dependencies

  • Reference graph and metrics. The VLM edges are the reference in both Fig. 7 and its text. Recall is the share of VLM edges recovered by CMI; precision is the share of CMI edges that are in the VLM graph. The caption and text now state this, and "false positive" for action→T-block is replaced by "CMI-only edge".

  • MSE gap versus CMI. Appendix F.1 previously described the leave-one-out MSE difference directly as CMI. We clarify that this difference first measures incremental predictive value. Under a Gaussian predictive-likelihood approximation, we convert the full and leave-one-out prediction errors into a log-likelihood-based dependency score, following the motivation of Wang et al. (2022). The resulting score approximates conditional mutual information under the stated predictive-model assumptions; it is not an assumption-free causal identification statistic.

CMIj→i=12log⁡(1+Δj→iMSEfull)\mathrm{CMI}{j\to i}=\tfrac{1}{2}\log\Big(1+\frac{\Delta{j\to i}}{\mathrm{MSE}_{\text{full}}}\Big)

  • Orientation. Every tested edge goes from the state at t to the state at t+H, so time fixes its direction. The reverse edge scores lower on Push-T: pusher→T-block 0.022 ± 0.006 against T-block→pusher 0.010 ± 0.005 (5 estimator seeds).

  • Collinearity and direct dependencies (Push-T simulator interventions, 600 start states, 5 steps). The action's effect on the T-block displacement, by pusher start distance, is 14.48 px (50–80), 11.79 px (80–110), 3.51 px (110–160) and 0.37 px (160–260). With the pusher trajectory held fixed the effect is below 0.002 px. Dropping the action from the conditioning set raises pusher→T-block from 0.023 to 0.198.

  • LIBERO precision of 0.37. The CMI union has 19 edges against 8 VLM edges, and 7 are shared (recall 0.88, F1 0.52). The one VLM-only edge is action→robot. The 12 CMI-only edges are action→bottle, bowl, plate, rack, stove; bottle→robot; cabinet→bottle, plate; plate→bowl, cabinet; rack→bottle, cabinet.

  • Cube. robot→cube scores 0.043 ± 0.036 over 5 estimator seeds and is recovered in 1 of 5 seeds at ε = 0.05, 3 of 5 at ε = 0.04 and 4 of 5 at ε = 0.03.

  • Threshold sensitivity. Recall of the VLM edges is 0.73–0.93 on Cube for ε from 0.02 to 0.10, 0.80 on Push-T at ε = 0.015–0.016, and 0.88 on LIBERO at ε = 0.015–0.03.

Cube (5 estimator seeds)

EpsilonPrecisionRecall
0.020.830.93
0.030.880.93
0.040.930.87
0.05 (ours)1.000.73
0.101.000.73
0.201.000.67
0.301.000.40

Push-T (5 estimator seeds)

EpsilonPrecisionRecall
0.0100.601.00
0.0120.600.90
0.015 (ours)0.570.80
0.0160.670.80
0.0180.730.60
0.0200.530.40
0.0300.600.30
0.040 and above0.000.00

LIBERO (5 seeds per task, union over tasks)

EpsilonCMI edgesPrecisionRecall
0.010220.361.00
0.015200.350.88
0.020 (ours)190.370.88
0.030160.440.88
0.050130.460.75
0.080110.550.75
0.12080.500.50
0.20030.330.12

Related Revised Sections: Sec. 3.3 (conditional-dependence formulation); Sec. 5.3 (CMI validation and Fig. 7); Appendix C (orientation, collinearity and identifiability); Appendix E.4 (Push-T interventions); Appendix F.1 (CMI estimator and threshold sensitivity).


AuthorsOpenReview ↗

T4zx_response_2

Response 2 (W2, Q1): architectural guarantee, implementation and sparsity-matched graphs

  • The guarantee, as now stated: each transition head reads only the projected states of its parents (Eq. 8). The projections are linear maps of the shared embedding, s^(n) = W_n z, and grounding a projection to box state does not by itself exclude information from other objects; App. E.2 now says what each head reads and no longer says that a grounding error stays confined to one object.

  • Implementation, one pipeline in Sec 4.2 and Algorithm 2:

    • The autoregressive predictor P_θ is trained in every mode as the shared scaffold.
    • In causal mode the graph branch g is trained toward P_θ with L_gr = ||P_θ − g||² (λ = 0.05, stop-gradient on P_θ).
    • Planning uses g on Cube and P_θ on Push-T; dense and object-centric modes use P_θ.
  • Eq. 3 and self-loops. Each head takes only its parents' states and has no own-state input; Eq. 3 excludes self-edges, and the self-loop in Appendix C is an assumption on the environment dynamics.

  • Alternative graph controls. We additionally evaluate a fully connected graph and a random graph on Push-T, obtaining 89.6 ± 2.1% and 89.8 ± 2.4% planning success, respectively, compared with 89.5 ± 2.7% for the VLM graph over eight evaluation seeds. Thus, planning performance is largely insensitive to the chosen graph topology in this setting. The VLM graph is not presented as a source of improved Push-T success, but as an explicit, interpretable structure whose dependencies can be assessed against observational and interventional evidence.

Related Revised Sections: Sec. 4.2 and Algorithm 2 (training and rollout pipeline; Eq. 9 with L_gr); Appendix C (self-loop assumption); Appendix E.2 (scope of architectural guarantee); Sec. 5.1 (planning path and parameter counts); Appendix E.6 (alternative graph controls, Table 6).


AuthorsOpenReview ↗

T4zx_response_3

Response 3 (W3, Q3, Q4): representation versus planning, seeds, bowl tasks and the benefit of the graph

  • What the LIBERO-Goal result measures. Language specifies the goal on LIBERO-Goal, so the benchmark is evaluated with a behavior-cloning head on frozen encoders and measures the representation. The revised Sec 5.2 states this. Reported quantities: object-state probing, sign test across objects p = 0.004 (random frame-level split), and BC success with the original head seed (3072): LeWM 0.512, object-centric 0.366, causal 0.570. Linear-probe r by arm (dense, object-centric, causal): Cube 0.835, 0.851, 0.854; LIBERO 0.738, 0.758, 0.768. On Cube the gain sits on the manipulated object (cube-x r 0.749, 0.819, 0.833).

  • Pretraining and BC-head seeds. We extend the LIBERO-Goal evaluation to three paired encoder-pretraining configurations, each evaluated using three independently trained BC-head seeds. The dense/causal success rates are 0.369/0.411, 0.449/0.353, and 0.333/0.497, respectively. Across configurations, the descriptive means are 0.384 for dense and 0.420 for causal, with causal encoders outperforming dense encoders in two of three comparisons. The results therefore retain a positive average effect while showing substantial variability across pretrained encoders. Dense vs. Causal Encoder Comparison (3 head seeds)

ComparisonDense MeanCausal MeanΔ (Causal − Dense)
Paper encoders0.3690.411+0.042
Pretraining pair 10.4490.353−0.096
Pretraining pair 20.3330.497+0.164
Overall descriptive mean0.3840.420+0.037
  • Bowl tasks. On the bowl tasks the dense arm reaches 0.560, the causal arm 0.548 and the object-centric arm 0.248. On the five tasks that never involve the mis-grounded object, causal reaches 0.592 against 0.464 for LeWM (250 rollouts per arm).

Benefit of Causal structure:

  • An auditable dependency structure: each object's next state is predicted only from its listed parents, checked against CMI (recall 0.73–0.93 on Cube, 0.80 on Push-T, 0.88 on LIBERO) and Push-T interventions.

  • Over dense predictors such as LeWM and DINO-WM, whose dependencies are implicit: an explicit structure at +1.07% parameters with matched planning success.

  • Over C-JEPA, which induces a causal bias by masking without an explicit graph and whose authors note that its influence neighbourhoods are not validated against explicit temporal causal graphs (Nam et al., 2026, Remark 2 and Conclusion): a graph that is explicit and validated.

Related Revised Sections: Abstract and Sec. 1 (LIBERO evaluation scope); Sec. 5.2 (LIBERO results, Table 2, bowl and non-bowl split); Sec. 5.3 (object-state probing); Sec. 2 (causal structure and C-JEPA comparison).


References

  • Kipf, T., van der Pol, E., and Welling, M. (2020). Contrastive Learning of Structured World Models. https://arxiv.org/abs/1911.12247.
  • Nam, H., Le Lidec, Q., Maes, L., LeCun, Y., and Balestriero, R. (2026). Causal-JEPA: Learning World Models through Object-Level Latent Masking. https://arxiv.org/abs/2602.11389.
  • Schölkopf, B., Locatello, F., Bauer, S., Ke, N. R., Kalchbrenner, N., Goyal, A., and Bengio, Y. (2021). Towards Causal Representation Learning. https://arxiv.org/abs/2102.11107.
  • Wang, Z., Xiao, X., Xu, Z., Zhu, Y., and Stone, P. (2022). Causal Dynamics Learning for Task-Independent State Abstraction. https://arxiv.org/abs/2206.13452.
  • Richens, J., and Everitt, T. (2024). Robust Agents Learn Causal World Models. ICLR.
  • Liu et al. (2023). Causal Triplet: An Open Challenge for Intervention-centric Causal Representation Learning. CLeaR.
  • World Models: A Comprehensive Survey of Architectures, Methodologies, Reasoning Paradigms, and Applications. (2026). arXiv:2606.00133.
  • Yuan et al. (2026). Representation World Model: Learning States, Transition and Executable Plans in Representation. arXiv:2609.29171.
AuthorsOpenReview ↗

Thank you and we are happy to take further questions if any

Dear Reviewer T4zX,

Thank you once again for your insightful review. We have provided a detailed response along with related discussions and revisions in the paper.

With the rebuttal deadline approaching, please feel free to let us know if you have any further questions or concerns. We will be happy to address them in the revision before the deadline.

We sincerely appreciate your time, effort, and valuable comments.

Thanks and regards,

Submission 20795 Authors

Reviewer_3uWNOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

This paper adds object-centric and causal structures to a JEPA latent world model. A VLM first names the objects in the first frame and let SAM3 track them, giving per-object boxes that a readout from the shared latent is trained to regress. The same VLM also proposes a corresponding sparse causal graph over objects and factored action tokens. Three modes are evaluated: dense, object-centric, and causal (OCCAM). On Push-T and Cube, OCCAM produces nearly no changes (90.0 vs 90.0 vs 89.5 and 75.9 vs 76.1 vs 76.0).

Critical strengths

None.

Critical weaknesses

The central claim is thin and not tested. Where the causal structure is actually used for planning (Push-T, Cube), it gives no gain. Where a gain is claimed (LIBERO-Goal), the encoder is frozen and a BC head is trained, so the causal structure plays no role in control. It is also unclear whether the causal network replaces the dense predictor at rollout time, since Eq. 9 keeps the dense loss.

Questions for authors
  1. In causal mode, which network produces the next latent during rollouts?

  2. Can you report LIBERO-Goal over at least 3 training seeds, and an evaluation on LIBERO that actually uses the predictor?

4 comments
AuthorsOpenReview ↗

3uWn_response_1

Reviewer 3uWN

We thank the reviewer. Your concerns come down to what is used at rollout time and what the LIBERO evaluation shows. (1) Rollout path: we state which network produces the next latent in each environment, and why the dense loss is kept (the AR predictor trains the shared encoder and is the distillation target of the graph branch) (Q1). (2) LIBERO: the goal is given by language, so there is no goal image to plan towards; LIBERO-Goal measures what causal pretraining puts into the encoder through a frozen-encoder BC head, not the predictor.


Response 1 (W1, Q1): which network produces the next latent at rollout, and what the structure adds

Rollout path per environment.

  • Cube: the graph branch g of the causally trained model produces the next latent at planning time.

  • Push-T: the autoregressive predictor P_θ of the causally trained model does.

  • Dense and object-centric modes use P_θ in every environment.

  • Why Eq. 9 keeps the dense loss. P_θ is trained in every mode as the shared training scaffold, and in causal mode the graph branch g is trained toward it with L_gr = ||P_θ − g||² (λ = 0.05, stop-gradient on P_θ). Sec 4.2 and Algorithm 2 now describe this one pipeline.

  • Planning results (Push-T, Cube). Push-T success: dense 90.0 ± 3.1, object-centric 90.0 ± 4.3, causal 89.5 ± 2.7. Cube: dense 75.9, causal 76.0. Evaluation uses 8 seeds, and the causal mode adds +1.07% parameters (18,228,492 vs 18,034,628).

Benefit of Causal structure:

  • An auditable dependency structure: each object's next state is predicted only from its listed parents, checked against CMI (recall 0.73–0.93 on Cube, 0.80 on Push-T, 0.88 on LIBERO) and Push-T interventions.

  • Over dense predictors such as LeWM and DINO-WM, whose dependencies are implicit: an explicit structure at +1.07% parameters with matched planning success.

  • Over C-JEPA, which induces a causal bias by masking without an explicit graph and whose authors note that its influence neighbourhoods are not validated against explicit temporal causal graphs (Nam et al., 2026, Remark 2 and Conclusion): a graph that is explicit and validated.

Related Revised Sections: Sec. 4.2 and Algorithm 2 (training and rollout pipeline; Eq. 9 with L_gr); Sec. 5.1 (planning paths, evaluation seeds, parameter counts); Sec. 1 and Sec. 2 (contribution and comparison with causal/object-centric world models).


AuthorsOpenReview ↗

3uWN_response_1

Response 2 (W1, Q2): the role of the causal structure on LIBERO-Goal, and seeds

  • Role on LIBERO-Goal. The encoders are frozen and a behavior-cloning head is trained on top, so the causal structure is not used in control there. The goal is a language instruction and no goal observation exists at test time (Yuan et al., 2026, App. A.2). The revised abstract, introduction and Sec 5.2 state the result as an effect of causal pretraining on the representation.

Measured effect on the representation.

  • Object-state probing from the frozen latent (linear probe to each object's box center, random frame-level split): sign test across objects p = 0.004. Linear-probe r by arm (dense, object-centric, causal): Cube 0.835, 0.851, 0.854; LIBERO 0.738, 0.758, 0.768. On Cube the gain sits on the manipulated object (cube-x r 0.749, 0.819, 0.833).

  • BC success with the original head seed (3072): LeWM 0.512, object-centric 0.366, causal 0.570.

Q2, seeds.

  • Single-seed versus multi-seed evaluation. The originally reported LIBERO-Goal results (0.512 dense versus 0.570 causal) correspond to BC-head seed 3072. The expanded evaluation averages over three independently trained BC-head seeds for each encoder configuration. Therefore, the original single-seed result and the multi-seed averages measure the same evaluation protocol under different seed aggregation, and the lower multi-seed averages reflect variability in BC-head training.

Dense vs. Causal Encoder Comparison (3 head seeds)

ComparisonDense MeanCausal MeanΔ (Causal − Dense)
Paper encoders0.3690.411+0.042
Pretraining pair 10.4490.353−0.096
Pretraining pair 20.3330.497+0.164
Overall descriptive mean0.3840.420+0.037

Causal encoders outperform dense encoders in 2 out of 3 comparisons. The paper's original encoders, evaluated across three head seeds, show a causal advantage of +0.042. We further evaluate two additional pretraining configurations, each comparing trained dense and causal encoders across three head seeds. One favors causal encoders (+0.164), while the other favors dense encoders (−0.096). Overall, the descriptive average favors causal encoders by +0.037.

Related Revised Sections: Abstract and Sec. 1 (LIBERO evaluation scope); Sec. 5.2 (LIBERO protocol and results, Table 2); Sec. 5.3 (object-state probing).


References

  • Kipf, T., van der Pol, E., and Welling, M. (2020). Contrastive Learning of Structured World Models. https://arxiv.org/abs/1911.12247.
  • Nam, H., Le Lidec, Q., Maes, L., LeCun, Y., and Balestriero, R. (2026). Causal-JEPA: Learning World Models through Object-Level Latent Masking. https://arxiv.org/abs/2602.11389.
  • Schölkopf, B., Locatello, F., Bauer, S., Ke, N. R., Kalchbrenner, N., Goyal, A., and Bengio, Y. (2021). Towards Causal Representation Learning. https://arxiv.org/abs/2102.11107.
  • Wang, Z., Xiao, X., Xu, Z., Zhu, Y., and Stone, P. (2022). Causal Dynamics Learning for Task-Independent State Abstraction. https://arxiv.org/abs/2206.13452.
  • Richens, J., and Everitt, T. (2024). Robust Agents Learn Causal World Models. ICLR.
  • Liu et al. (2023). Causal Triplet: An Open Challenge for Intervention-centric Causal Representation Learning. CLeaR.
  • World Models: A Comprehensive Survey of Architectures, Methodologies, Reasoning Paradigms, and Applications. (2026). arXiv:2606.00133.
  • Yuan et al. (2026). Representation World Model: Learning States, Transition and Executable Plans in Representation. arXiv:2609.29171.
AuthorsOpenReview ↗

Thank you and we are happy to take further questions if any

Dear Reviewer 3uWN,

Thank you once again for your insightful review. We have provided a detailed response along with related discussions and revisions in the paper.

With the rebuttal deadline approaching, please feel free to let us know if you have any further questions or concerns. We will be happy to address them in the revision before the deadline.

We sincerely appreciate your time, effort, and valuable comments.

Thanks and regards,

Submission 20795 Authors

We thank the authors for the clarification, but it confirms my concerns: the graph is only used for Cube planning, where the success is essentially unchanged (76.0 versus 75.9). Other experimental gains also cannot be separated from seed variance. Thus my rating is unchanged.

Reviewer_YVrEOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper introduces OCCAM, which incorporates VLM-derived object grounding and a dynamics graph into a JEPA-style latent world model. A VLM and SAM3 generate object-box supervision; object representations are obtained through projections of a shared visual embedding; and graph-constrained object dynamics restrict each prediction to its designated parents. A separate trajectory-based prediction-error test assesses graph agreement. Experiments cover 12 tasks across Push-T, OGBench-Cube, and LIBERO-Goal, using CEM planning for the first two environments and frozen-encoder behavior cloning for LIBERO. OCCAM retains approximately baseline planning performance in the simpler environments and improves downstream control success and linear object-position decodability on LIBERO.

Critical strengths
  1. The integration of semantic object grounding with structured latent dynamics is practically valuable. Using a VLM and SAM3 to obtain object supervision provides a feasible route to introducing named objects and inspectable relations into a world model without pixel reconstruction or manual mask annotation.

  2. The results suggest that structured pretraining can improve downstream representations while retaining basic planning performance. Eight-seed results on Push-T and Cube remain close to the dense baseline, while frozen-encoder LIBERO success increases from 51.2% to 57.0%. Reporting the bowl-grounding failure and separating affected tasks is useful for understanding the limitations of automatically generated supervision.

Critical weaknesses
  1. The evidence does not sufficiently establish the claimed causal interpretation. Appendix F.1 labels the MSE difference between full-input and leave-one-out predictors as CMI. In general, a difference in conditional-mean prediction error is not conditional mutual information and therefore does not directly inherit the cited conditional-independence guarantee. Appendix C appropriately acknowledges insufficient state information and collinearity, but these limitations weaken the validator as evidence of causality; the reported LIBERO graph-agreement F1 is also only 0.52. Furthermore, all object representations are projections of the same global embedding. Bounding-box decodability does not establish that information about different objects has been separated, so sparse connections between these representations do not automatically exclude information from non-parent physical objects. These issues affect the central contribution concerning causal structure and mechanistic interpretability.

  2. The experiments do not isolate a causal-transition benefit in complex scenes. Push-T and Cube, where the world model is directly used for planning, show no clear performance improvement. The positive aggregate result on the complex LIBERO suite evaluates frozen-encoder behavior cloning, which cannot directly establish that the causal predictor improves complex-scene planning. Environment complexity also changes together with the task, dataset, and evaluation protocol. Moreover, alternative sparse graphs with matched supervision, dynamics losses, and comparable capacity are needed to distinguish the VLM graph’s content from generic regularization. Finally, deterministic evaluation does not remove variability from world-model pretraining or BC-head training, so the single-point LIBERO results do not establish the stability of the 5.8-percentage-point gain.

Questions for authors
  1. Implementation clarification: How should the predictor replacement in Section 4.2 be reconciled with Algorithm 2, which retains dense prediction and adds an auxiliary loss? Which branch is actually used for CEM rollouts? How is self-state provided, given that Equations (3) and (8) exclude the target object from its parent set while Appendix C assumes self-loops?

  2. Causal validation: Under the conditions of these experiments, why is the MSE gap a reliable substitute for CMI or a conditional-independence test? How were the environment-specific thresholds selected, and how sensitive is graph agreement to these thresholds and estimation randomness?

  3. Attribution: Are there alternative-graph controls, such as equal-edge-count random graphs, with matched supervision, losses, and comparable capacity? Is there object-intervention or information-leakage evidence supporting the exclusion of non-parent object information from the projected representations? Such evidence would materially affect my assessment of the causal contribution.

  4. Reliability and scope: Does the LIBERO improvement persist across independent pretraining and BC-training seeds? Is there evidence from direct predictor-based planning in a complex multi-object environment that supports the main claim concerning complexity and planning benefits?

4 comments
AuthorsOpenReview ↗

Yvre_response_1

Reviewer YVrE

We thank the reviewer. We respond to the main concerns as follows. (1) Implementation: we state which branch trains and which one plans in each environment, and how self-state is handled, and have rewritten Section 4.2 and Algorithm 2 accordingly for the revised manuscript (Q1). (2) Causal validation: we define the CMI test and its relation to the MSE gap, and give threshold sensitivity on all three environments and the estimator-seed spread for Cube and Push-T (Q2). (3) Isolation: projections of one shared embedding do not by themselves exclude information from non-parent objects, and the guarantee we claim is about what each transition head reads; we made this precise in the revised manuscript (Q3). (4) Scope: LIBERO-Goal measures what causal pretraining puts into the encoder, not planning (Q4).


Response 1 (Q1): which network plans, and what the graph heads read

Planning path per environment.

  • Cube: planning uses the graph branch g of the causally trained model.

  • Push-T: planning uses the autoregressive predictor P_θ of the causally trained model.

  • Dense and object-centric modes use P_θ in every environment.

  • P_θ is trained in every mode as the shared training scaffold. In causal mode the graph branch g is trained toward it with L_gr = ||P_θ − g||² (λ = 0.05, stop-gradient on P_θ).

  • What the graph heads read. Each head takes only the projected states of its parents and has no own-state input. Eq. 3 excludes self-edges; the self-loop in Appendix C is an assumption on the environment dynamics.

  • Sec 4.2 and Algorithm 2 now describe this one pipeline, and Sec 5.1 lists the planning path for each environment.

Related Revised Sections: Sec 4.2 (one description of training and rollout path; Eq. 3 and Eq. 9 with L_gr) · Algorithm 2 (P_θ retained, graph branch distilled) · App. C (self-loop assumption) · Sec 5.1 (planning path per environment).


AuthorsOpenReview ↗

Yvre_response_2_3

Response 2 (W1 a and b, Q2): MSE gap versus CMI, collinearity and the F1 of 0.52

  • (a) MSE gap versus CMI. Appendix F.1 previously described the leave-one-out MSE difference directly as CMI. We clarify that this difference first measures incremental predictive value. Under a Gaussian predictive-likelihood approximation, we convert the full and leave-one-out prediction errors into a log-likelihood-based dependency score, following the motivation of Wang et al. (2022). The resulting score approximates conditional mutual information under the stated predictive-model assumptions; it is not an assumption-free causal identification statistic.

CMIj→i=12log⁡(1+Δj→iMSEfull)\mathrm{CMI}{j\to i}=\tfrac{1}{2}\log\Big(1+\frac{\Delta{j\to i}}{\mathrm{MSE}_{\text{full}}}\Big)

  • (b) Collinearity. On Push-T, the pusher→T-block score rises from 0.023 to 0.198 once the action is dropped from the conditioning set. Simulator interventions (600 start states, 5 steps) separate the action's direct effect from its effect through the pusher: the action's effect on the T-block displacement, by pusher start distance, is 14.48 px (50–80), 11.79 px (80–110), 3.51 px (110–160) and 0.37 px (160–260); with the pusher trajectory held fixed it is below 0.002 px.

  • Orientation. Every tested edge goes from the state at t to the state at t+H, so time fixes its direction. The reverse edge scores lower on Push-T: pusher→T-block 0.022 ± 0.006 against T-block→pusher 0.010 ± 0.005 (5 estimator seeds).

  • The F1 of 0.52 on LIBERO combines precision 0.37 and recall 0.88 against the 8 VLM edges. The CMI union has 19 edges, 7 of them shared with the VLM graph; the one VLM-only edge is action→robot. Precision is measured against the VLM graph, not against ground truth. The 12 CMI-only edges are action→bottle, bowl, plate, rack, stove; bottle→robot; cabinet→bottle, plate; plate→bowl, cabinet; rack→bottle, cabinet.

  • Threshold sensitivity. Recall of the VLM edges is 0.73–0.93 on Cube for ε from 0.02 to 0.10, 0.80 on Push-T at ε = 0.015–0.016, and 0.88 on LIBERO at ε = 0.015–0.03.

Cube (5 estimator seeds)

EpsilonPrecisionRecall
0.020.830.93
0.030.880.93
0.040.930.87
0.05 (ours)1.000.73
0.101.000.73
0.201.000.67
0.301.000.40

Push-T (5 estimator seeds)

EpsilonPrecisionRecall
0.0100.601.00
0.0120.600.90
0.015 (ours)0.570.80
0.0160.670.80
0.0180.730.60
0.0200.530.40
0.0300.600.30
0.040 and above0.000.00

LIBERO (5 seeds per task, union over tasks)

EpsilonCMI edgesPrecisionRecall
0.010220.361.00
0.015200.350.88
0.020 (ours)190.370.88
0.030160.440.88
0.050130.460.75
0.080110.550.75
0.12080.500.50
0.20030.330.12
  • Corrections made. Fig. 7 and its text use the VLM edges as the reference graph, with precision defined as the share of CMI edges in the VLM graph and recall as the share of VLM edges recovered.

Related Revised Sections: Sec. 3.3 (conditional-dependence formulation); Sec. 5.3 (CMI validation and Fig. 7); Appendix C (causal assumptions and identifiability); Appendix E.4 (Push-T interventions); Appendix F.1 (CMI estimator and threshold sensitivity).


Response 3 (W1 c, Q3): the shared embedding and what the architecture guarantees

  • W1(c). Each object state is a linear projection of the shared embedding, s^(n) = W_n z. We agree that grounding a projection to box state does not by itself exclude information from other objects, and the revised paper makes no such claim.

  • The guarantee, as now stated: each transition head reads only the projected states of its parents (Eq. 8). This is a property of the architecture. App. E.2 no longer says that a grounding error stays confined to one object; it states what each head reads.

  • What this structure gives: the set of inputs behind every prediction is explicit and can be checked, and it is checked against CMI and Push-T interventions. Related work points the same way: C-SWM reports interpretable object-level abstractions from a relational graph (Kipf et al., 2020), and a recent survey argues that counterfactual reasoning needs a modular causal decomposition of the latent state (arXiv:2606.00133, Sec. 5.3.3).

Related Revised Sections: Sec 4.2, App. E.2, Sec 2.


AuthorsOpenReview ↗

Yvre_response_4

Response 4 (W2, Q4): isolating the benefit, complexity, alternative graphs and seeds

  • Where the model plans (Push-T, Cube). Push-T success: dense 90.0 ± 3.1, object-centric 90.0 ± 4.3, causal 89.5 ± 2.7. Cube: dense 75.9, causal 76.0. Evaluation uses 8 seeds, and the causal mode adds +1.07% parameters.

  • Complexity. LIBERO-Goal has more tracked objects than Push-T or Cube, and it is also a different task, dataset and protocol. The three environments are therefore not a matched complexity ablation, and Sec 5.2 says so.

  • Pretraining and BC-head seeds. We extend the LIBERO-Goal evaluation to three paired encoder-pretraining configurations, each evaluated using three independently trained BC-head seeds. The dense/causal success rates are 0.369/0.411, 0.449/0.353, and 0.333/0.497, respectively. Across configurations, the descriptive means are 0.384 for dense and 0.420 for causal, with causal encoders outperforming dense encoders in two of three comparisons. The results therefore retain a positive average effect while showing substantial variability across pretrained encoders.

Dense vs. Causal Encoder Comparison (3 head seeds)

ComparisonDense MeanCausal MeanΔ (Causal − Dense)
Paper encoders0.3690.411+0.042
Pretraining pair 10.4490.353−0.096
Pretraining pair 20.3330.497+0.164
Overall descriptive mean0.3840.420+0.037
  • Alternative graph controls. We additionally evaluate a fully connected graph and a random graph on Push-T, obtaining 89.6 ± 2.1% and 89.8 ± 2.4% planning success, respectively, compared with 89.5 ± 2.7% for the VLM graph over eight evaluation seeds. Thus, planning performance is largely insensitive to the chosen graph topology in this setting. The VLM graph is not presented as a source of improved Push-T success, but as an explicit, interpretable structure whose dependencies can be assessed against observational and interventional evidence.

Benefit of Causal structure:

  • An auditable dependency structure: each object's next state is predicted only from its listed parents, checked against CMI (recall 0.73–0.93 on Cube, 0.80 on Push-T, 0.88 on LIBERO) and Push-T interventions.
  • Over dense predictors such as LeWM and DINO-WM, whose dependencies are implicit: an explicit structure at +1.07% parameters with matched planning success.
  • Over C-JEPA, which induces a causal bias by masking without an explicit graph and whose authors note that its influence neighbourhoods are not validated against explicit temporal causal graphs (Nam et al., 2026, Remark 2 and Conclusion): a graph that is explicit and validated.

Related Revised Sections: Sec. 1 (contribution and evaluation scope); Sec. 2 (motivation and related work); Sec. 5.1 (planning protocol and evaluation seeds); Sec. 5.2 (complexity qualification and LIBERO results, Table 2); Appendix E.6 (alternative graph controls, Table 6).


References

  • Kipf, T., van der Pol, E., and Welling, M. (2020). Contrastive Learning of Structured World Models. https://arxiv.org/abs/1911.12247.
  • Nam, H., Le Lidec, Q., Maes, L., LeCun, Y., and Balestriero, R. (2026). Causal-JEPA: Learning World Models through Object-Level Latent Masking. https://arxiv.org/abs/2602.11389.
  • Schölkopf, B., Locatello, F., Bauer, S., Ke, N. R., Kalchbrenner, N., Goyal, A., and Bengio, Y. (2021). Towards Causal Representation Learning. https://arxiv.org/abs/2102.11107.
  • Wang, Z., Xiao, X., Xu, Z., Zhu, Y., and Stone, P. (2022). Causal Dynamics Learning for Task-Independent State Abstraction. https://arxiv.org/abs/2206.13452.
  • Richens, J., and Everitt, T. (2024). Robust Agents Learn Causal World Models. ICLR.
  • World Models: A Comprehensive Survey of Architectures, Methodologies, Reasoning Paradigms, and Applications. (2026). arXiv:2606.00133.
AuthorsOpenReview ↗

Thank you and we are happy to take further questions if any

Dear Reviewer YVrE,

Thank you once again for your insightful review. We have provided a detailed response along with related discussions and revisions in the paper.

With the rebuttal deadline approaching, please feel free to let us know if you have any further questions or concerns. We will be happy to address them in the revision before the deadline.

We sincerely appreciate your time, effort, and valuable comments.

Thanks and regards,

Submission 20795 Authors

Reviewer_CUCYOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes a framework that adds object-level and causal structure to a JEPA-style world model. A VLM lists the objects in the first frame, and SAM3 tracks them to get their bounding boxes. The encoder is trained with a linear readout that predicts each object’s bounding box from the latent. The VLM is then shown frames where objects move and decides which objects and actions directly cause other objects to move, producing a sparse causal graph. In the causal mode, each object’s next state is predicted only from the objects and actions that the graph says affect it. They test on three environments (Push-T, Cube and LIBERO-Goal). Results on Push-T and Cube are similar to the LeWM baseline, while on LIBERO-Goal, a benchmark of cluttered kitchen manipulation tasks, their method achieves a higher success rate.

Critical strengths

Object and causal structure without manual labels. A VLM and SAM3 ground the objects and propose a causal graph with no hand-labelled masks, and the graph is checked with a statistical test. This makes the approach practical to apply to new environments.

A clean, controlled ablation. The dense, object-centric and causal versions share the same encoder, predictor backbone and planner. Only the structure of the transition function changes, so any difference can be attributed to that structure rather than to architecture or training differences.

Critical weaknesses

OCCAM only beats LeWM on LIBERO-Goal (0.570 vs. 0.512), from a single run. On LIBERO, the encoder is frozen, and only a behaviour-cloning head is trained, which the authors say tests the encoder, not the predictor (4.2). So it’s unclear whether the causal predictor helps at all.

There is no control with per-object networks and a fully connected or random graph, and no comparison to simple graphs or the CMI graph. On LIBERO, there is also no object-centric baseline (e.g., C-JEPA).

Questions for authors

Is the causal predictor used on LIBERO?

How were the CMI thresholds chosen? They differ per environment, and in Push-T accepted edges sit just above the threshold. How do precision and recall change across a range of chosen thresholds?

Could you add more baselines to compare your method on LIBERO? Can you add a control with per-object networks using a fully connected graph, and one using a random graph of the same sparsity?

Minor presentation issues:

  • Some variables in the equations are not defined properly in the text.
  • The sentence starting at L41 seems to be incomplete.
5 comments
AuthorsOpenReview ↗

Cucy_Response_1

Reviewer CUCY

We thank the reviewer for the constructive feedback. We address each concern below and have incorporated the corresponding clarifications and additional experiments into the revised manuscript and appendix.

Response 1 (W1, Q1): what LIBERO-Goal tests, and whether the causal predictor matters there

Q1: the causal predictor is not part of the LIBERO-Goal pipeline. LIBERO-Goal gives the task as a language instruction and no goal observation at test time (Yuan et al., 2026, App. A.2). A latent planner that ranks rollouts by distance to a goal latent (LeWM; Maes et al., 2026) has no goal to rank against, and RC-aux likewise trains an action-prediction head on this benchmark instead of using its planner (Li et al., 2026, Sec. 4.5).

Protocol. The LeWM, object-centric and causal encoders are pretrained on the LIBERO-Goal demonstrations with their own objectives, then frozen. One OFT-style head (chunk 8, 4 layers, hidden size 1024, 30 epochs, 50 rollouts per task) is trained on each. Only the pretraining objective differs between arms.

W1, single run.

Dense vs. Causal Encoder Comparison (3 head seeds)

ComparisonDense MeanCausal MeanΔ (Causal − Dense)
Paper encoders0.3690.411+0.042
Pretraining pair 10.4490.353−0.096
Pretraining pair 20.3330.497+0.164
Overall descriptive mean0.3840.420+0.037

Causal encoders outperform dense encoders in 2 out of 3 comparisons. The paper's original encoders, evaluated across three head seeds, show a causal advantage of +0.042. We further evaluate two additional pretraining configurations, each comparing independently trained dense and causal encoders across three head seeds. One favors causal encoders (+0.164), while the other favors dense encoders (−0.096). Overall, the descriptive average favors causal encoders by +0.037.

W1, encoder-only test: the measured effect on the representation.

  • Object-state probing from the frozen latent (linear probe to each object's box center, random frame-level split): sign test across objects p = 0.004 (Sec 5.2). Linear-probe r by arm (dense, object-centric, causal): Cube 0.835, 0.851, 0.854; LIBERO 0.738, 0.758, 0.768. On Cube the gain sits on the manipulated object (cube-x r 0.749, 0.819, 0.833).

  • Planning evidence. Push-T and Cube directly evaluate goal-conditioned planning with the learned world model. On Push-T, the causally trained model plans using the autoregressive predictor and achieves 89.5 ± 2.7% success, compared with 90.0 ± 3.1% for the dense baseline. On Cube, planning directly uses the graph-conditioned branch, achieving 76.0% versus 75.9% for the dense baseline. These results establish planning parity under the respective rollout protocols, with only 1.07% additional parameters in the causal model.

Motivation for causal structure beyond planning parity:

  • Causal models are argued to support robust transfer (Schölkopf et al., 2021); a robust agent under distribution shift must have learned an approximate causal model (Richens & Everitt, 2024); sparse causal dynamics generalize to unseen states in the benchmarks of Wang et al. (2022).

  • It is an explicit, auditable dependency structure: each object's next state is predicted only from its listed parents (Sec 4.2). Dense predictors such as LeWM and DINO-WM have implicit dependencies.

  • It is checked against two independent references: a CMI estimator (recall 0.73–0.93 on Cube, 0.80 on Push-T, 0.88 on LIBERO) and simulator interventions on Push-T.

  • The closest object-centric method, C-JEPA, induces a causal bias by masking and uses no explicit causal graph; its authors state that its influence neighbourhoods are not validated on datasets with explicit temporal causal graphs (Nam et al., 2026, Remark 2 and Conclusion).

Related Revised Sections: Abstract and Sec. 1 (contribution and evaluation scope); Sec. 2 (related work); Sec. 4.2 (predictor training and rollout selection); Sec. 5.1 (planning protocol and parameter counts); Sec. 5.2 (LIBERO protocol and Table 2); Sec. 5.3 (object-state probing).


AuthorsOpenReview ↗

Cucy_response_2

Response 2 (W2, Q3): controls and baselines

Fully connected, random controls on Push-T.

We evaluate two alternative graph structures under the same training and evaluation protocol. The VLM graph achieves 89.5 ± 2.7% planning success, compared with 89.6 ± 2.1% for the fully connected graph and 89.8 ± 2.4% for the random graph, across eight evaluation seeds. These results show that the particular VLM-selected edges do not confer a planning-success advantage on Push-T. Instead, the graph's contribution is to provide an explicit semantic dependency hypothesis that can be independently examined using CMI and simulator interventions. We include these controls in the revised appendix.

Object-centric baseline. The object-centric variant (same encoder, readout, data and protocol as OCCAM) is the object-centric baseline in every LIBERO comparison (Table R1). C-JEPA reports Push-T and CLEVRER results and no LIBERO-Goal result (Nam et al., 2026).

Published LeWM-family results on LIBERO-Goal, each measured with its own protocol:

  • LeWM + OFT head, encoder trained with the head: 0.712 (Li et al., 2026) and 0.777 ± 0.005 (PSG-JEPA, arXiv:2608.06799).

  • LeWM, frozen encoder, flow-matching policy: 0.534 ± 0.053 (Liu et al., 2026).

  • Object slots on the LeWM encoder with its own flow-matching head and selector: 0.56, against 0.51 for a monolithic LeWM latent (SR-WM; Cheng et al., 2026). SR-WM supervises roles and predicates with simulator state.

  • Table R3 lists the protocol beside each number. OCCAM's rows come from our pipeline, in which all arms share one protocol.

Advantages of OCCAM over these comparable methods:

  • Over dense predictors such as LeWM and DINO-WM, whose dependencies are implicit: an explicit dependency structure at +1.07% parameters, with planning success matching our dense arm on Push-T and Cube.

  • Over C-JEPA: an explicit graph validated against CMI and simulator interventions, where C-JEPA's influence neighbourhoods are implicit (Nam et al., 2026).

  • Over SR-WM: the graph comes from a VLM given 8 sampled episodes, with no simulator labels; SR-WM's role and predicate labels come from simulator state (Cheng et al., 2026, Table S5).

  • Over CMI-based discovery: the VLM graph needs 8 sampled episodes, where the CMI estimator used 500 annotated episodes (about 100,000 transitions) on Cube, and the estimator then confirms most VLM edges.

Related Revised Sections: Sec. 2 (comparison with object-centric and causal world models); Sec. 5.1 (evaluation protocol and predictor selection); Sec. 5.2 (LIBERO results and Table 2); Appendix E.6 (Push-T graph-structure controls, Table 6).


AuthorsOpenReview ↗

Cucy_response_3

Response 3 (Q2): the CMI estimator and the threshold

Definition MSE gap versus CMI. Appendix F.1 previously described the leave-one-out MSE difference directly as CMI. We clarify that this difference first measures incremental predictive value. Under a Gaussian predictive-likelihood approximation, we convert the full and leave-one-out prediction errors into a log-likelihood-based dependency score, following the motivation of Wang et al. (2022). The resulting score approximates conditional mutual information under the stated predictive-model assumptions; it is not an assumption-free causal identification statistic.

CMIj→i=12log⁡(1+Δj→iMSEfull)\mathrm{CMI}{j\to i}=\tfrac{1}{2}\log\Big(1+\frac{\Delta{j\to i}}{\mathrm{MSE}_{\text{full}}}\Big)

Sensitivity to ε (precision / recall against the VLM graph; 5 estimator seeds on Cube and Push-T; on LIBERO, edges kept in at least 50% of the tasks with CMI):

Cube (5 estimator seeds)

EpsilonPrecisionRecall
0.020.830.93
0.030.880.93
0.040.930.87
0.05 (ours)1.000.73
0.101.000.73
0.201.000.67
0.301.000.40

Push-T (5 estimator seeds)

EpsilonPrecisionRecall
0.0100.601.00
0.0120.600.90
0.015 (ours)0.570.80
0.0160.670.80
0.0180.730.60
0.0200.530.40
0.0300.600.30
0.040 and above0.000.00

LIBERO (5 seeds per task, union over tasks)

EpsilonCMI edgesPrecisionRecall
0.010220.361.00
0.015200.350.88
0.020 (ours)190.370.88
0.030160.440.88
0.050130.460.75
0.080110.550.75
0.12080.500.50
0.20030.330.12
  • On Cube, recall of the VLM edges is 0.73–0.93 for ε from 0.02 to 0.10; on Push-T it is 0.80 at ε = 0.015–0.016; on LIBERO it is 0.88 at ε = 0.015–0.03.

  • The VLM graph is built from 8 sampled episodes; the estimator uses 500 annotated episodes on Cube (about 100,000 transitions).

Presentation issues. We thank the reviewer for identifying the undefined variables and incomplete sentence. We have clarified the relevant notation and corrected the sentence in the revised manuscript.

Related Revised Sections: Sec. 3.3 (conditional-dependence formulation and assumptions); Sec. 5.3 (CMI graph validation and Fig. 7); Appendix C (causal assumptions and identifiability); Appendix E.4 (controlled simulator interventions); Appendix F.1 (Gaussian CMI estimator, five-seed variability, and threshold sensitivity, Tables 9–11).


References

  • Kipf, T., van der Pol, E., and Welling, M. (2020). Contrastive Learning of Structured World Models. https://arxiv.org/abs/1911.12247.

  • Nam, H., Le Lidec, Q., Maes, L., LeCun, Y., and Balestriero, R. (2026). Causal-JEPA: Learning World Models through Object-Level Latent Masking. https://arxiv.org/abs/2602.11389.

  • Schölkopf, B., Locatello, F., Bauer, S., Ke, N. R., Kalchbrenner, N., Goyal, A., and Bengio, Y. (2021). Towards Causal Representation Learning. https://arxiv.org/abs/2102.11107.

  • Wang, Z., Xiao, X., Xu, Z., Zhu, Y., and Stone, P. (2022). Causal Dynamics Learning for Task-Independent State Abstraction. https://arxiv.org/abs/2206.13452.

  • Richens, J., and Everitt, T. (2024). Robust Agents Learn Causal World Models. ICLR.

  • Liu et al. (2023). Causal Triplet: An Open Challenge for Intervention-centric Causal Representation Learning. CLeaR.

  • World Models: A Comprehensive Survey of Architectures, Methodologies, Reasoning Paradigms, and Applications. (2026). arXiv:2606.00133.

  • Yuan et al. (2026). Representation World Model: Learning States, Transition and Executable Plans in Representation. arXiv:2609.29171.

  • Li et al. (2026). Predictive but Not Plannable: RC-aux for Latent World Models. https://arxiv.org/abs/2605.07278.

  • Maes et al. (2026). LeWorldModel: Stable End-to-End Joint-Embedding Predictive Architecture from Pixels. https://arxiv.org/abs/2603.19312.

  • Cheng et al. (2026). Beyond Instance Slots: Semantically Rich World Models for Physical Interaction Planning. https://arxiv.org/abs/2608.22294.

  • Liu et al. (2026). Temporally Centered SIGReg Improves LeWorldModel Representations for Robot Policy Learning. https://arxiv.org/abs/2607.26924.

  • Is Forward Prediction Enough? Physical State Grounding for JEPA World Models (PSG-JEPA, 2026). https://arxiv.org/abs/2608.06799.

Comments on the paper

2 comments
AuthorsOpenReview ↗

General response_2

Expanded LIBERO Evaluation and Representation Analysis

  • Additional pretraining and BC-head seeds. We expand the original LIBERO comparison to three paired encoder-pretraining configurations, each evaluated with three independently trained BC-head seeds. The dense and causal success rates are 0.369/0.411, 0.449/0.353, and 0.333/0.497, respectively. The descriptive means are 0.384 for dense and 0.420 for causal, an average difference of +0.037, with causal encoders outperforming dense encoders in two of the three comparisons. These additional experiments expose the variability across pretrained encoders while retaining a positive average result.

  • Object-state decodability. We further clarify the frozen-encoder linear-probe analysis. Mean object-state correlations increase from 0.835 to 0.854 on Cube and from 0.738 to 0.768 on LIBERO when comparing dense and causal representations. We also specify the random frame-level split used in this analysis and distinguish object-state decodability from generalization to unseen trajectories.

  • Qualitative analysis and grounding errors. We retain the original LIBERO rollout and bowl-task analyses while clarifying that the graph-conditioned architecture restricts transition-head inputs, not the information content of the shared latent representation. Consequently, these experiments provide evidence about learned representations and downstream behavior, rather than a guarantee that grounding errors cannot propagate.

Related revised sections: Sec. 5.2–5.3 (expanded LIBERO results and probing), Appendix E (qualitative analysis and grounding limitations).


CMI Validation, Threshold Sensitivity, and Simulator Interventions

  • CMI estimator and assumptions. We clarify the distinction between the raw leave-one-out prediction-error difference and its Gaussian predictive-likelihood transformation into a CMI-style score. We also specify the causal Markov, faithfulness, and state-sufficiency assumptions under which the population conditional-dependence criterion identifies temporal dependencies. The reported VLM–CMI precision and recall measure agreement between graph-construction procedures, not recovery against a known physical ground-truth graph.

  • Threshold and estimator-seed sensitivity. We add threshold sweeps and five-seed estimator analyses. On Cube, VLM-edge recall ranges from 0.73 to 0.93 over the examined thresholds; the robot-to-cube dependency has score 0.043 ± 0.036 and is recovered in 1/5, 3/5, and 4/5 estimator runs at thresholds 0.05, 0.04, and 0.03, respectively. On Push-T, the pusher-to-block dependency scores 0.022 ± 0.006, compared with 0.010 ± 0.005 in the reverse direction. On LIBERO, the CMI graph contains 19 edges, the VLM graph contains 8, and 7 are shared, yielding precision 0.37, recall 0.88, and F1 0.52 at the selected threshold.

  • Controlled Push-T interventions. We add simulator interventions across 600 initial states with a five-step horizon. The action-induced block displacement decreases as the initial pusher-to-block distance increases, from 14.48 pixels in the nearest distance bin to 0.37 pixels in the farthest. When the pusher trajectory is held fixed, the effect falls below 0.002 pixels. These experiments support a pusher-mediated interpretation of the action's effect on the block and help explain the observational collinearity between action and pusher state.

Related revised sections: Sec. 3.3 and Sec. 5.3 (dependency interpretation and graph validation), Appendix C (causal assumptions), Appendix E (interventions), Appendix F.1 (CMI estimator, seed variability, and threshold-sensitivity tables).


We believe these additional experiments and clarifications strengthen the empirical characterization of OCCAM and make the scope of its contributions more explicit. In particular, the revised results distinguish planning performance, representation transfer, and the interpretability and validation of the proposed graph, while preserving the central contribution of introducing explicit, language-grounded interaction structure into latent world models.

We thank the Area Chair and reviewers again for their time and constructive suggestions. We hope that our detailed responses and the additional evidence address the main concerns.

AuthorsOpenReview ↗

General response_1

General Response

We sincerely thank the Area Chair and all reviewers for their careful reading, constructive feedback, and valuable suggestions. We have addressed the individual concerns in our reviewer responses and incorporated the corresponding clarifications and additional experiments into the revised manuscript and appendix. In particular, we have expanded the evaluation of graph structure, examined sensitivity to random seeds and CMI thresholds, and clarified the distinct roles of the causal predictor and the learned representation. Below, we summarize the main points and corresponding revisions.


Comparison with Alternative Graph Structures and the Role of Causality

  • Fully connected and random graph controls. To isolate the contribution of the VLM-proposed dependency structure, we evaluate alternative graph topologies on Push-T. The VLM graph achieves 89.5 ± 2.7% planning success, compared with 89.6 ± 2.1% for the fully connected graph and 89.8 ± 2.4% for the random graph, across eight evaluation seeds. These controls show that the specific VLM-selected edges do not provide an additional planning-success advantage on Push-T. OCCAM's contribution is instead the introduction of an explicit, language-grounded dependency structure that can be independently inspected and evaluated, while preserving competitive planning performance.

  • Relationship to existing object-centric and causal world models. We clarify OCCAM's distinction from dense latent predictors such as LeWM and DINO-WM, and from object-centric approaches such as C-JEPA. While object-centric representations and masking can introduce structural biases, OCCAM explicitly specifies which object and action variables each transition head receives. We further examine the proposed dependencies through CMI estimation and simulator interventions, rather than relying solely on downstream success as evidence of graph quality.

Related revised sections: Sec. 1–2 (motivation and related work), Sec. 5.2–5.3 (interpretation of planning and graph validation), Appendix E (graph-structure controls).


Clarification of Training, Planning, and LIBERO Evaluation

  • Autoregressive and graph-conditioned predictors. We clarify that the autoregressive predictor (P_θ) is retained in all three training modes. In the causal mode, the graph-conditioned branch is trained through distillation toward the autoregressive prediction, with loss weight (λ_gr = 0.05). At planning time, Cube uses the graph-conditioned predictor, whereas Push-T uses the autoregressive predictor of the causally trained model. The dense and object-centric variants use the autoregressive predictor in both environments. These distinctions are now made explicit in the methodology and training algorithm.

  • Planning performance and model capacity. OCCAM preserves planning success relative to the dense baseline on Push-T (89.5 ± 2.7% vs. 90.0 ± 3.1%) and Cube (76.0% vs. 75.9%), with only 1.07% additional parameters. The results establish that explicit graph-conditioned dynamics can be introduced without sacrificing the planning performance of the underlying world model.

  • LIBERO-Goal evaluation protocol. We clarify that LIBERO-Goal evaluates frozen world-model encoders using separately trained behavior-cloning heads, rather than directly planning with the causal predictor. The language-specified goals do not provide the goal observations required by the latent-distance planner. The LIBERO experiments therefore examine the effect of causal pretraining on downstream representation transfer.

Related revised sections: Sec. 4.2 (training objective and predictor selection), Sec. 5.1 (evaluation protocols and model capacity), Sec. 5.2 (planning and LIBERO results), Appendix D (training algorithm).


From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.