acceptodds
Under review as a conference paper at ICLR 2027

Causality-Inspired Object-Centric Models for Planning with Monte Carlo Tree Search

Abstract

We introduce COMET (Causality-Inspired Object-Centric Models for Planning with Monte Carlo Tree Search), a model-based reinforcement learning algorithm that performs Monte Carlo Tree Search in a slot-structured latent space. COMET pairs a frozen unsupervised object-centric encoder with a transformer-based world model, in which actions are bound to objects through a novel action–slot fusion mechanism that is used in slot transition prediction. Policy and value heads use causality-inspired attention, modulating token interactions by learned per-slot relevance scores so that decision-making concentrates on task-relevant entities. COMET adds an explicit object-level inductive bias to MuZero-style latent planning. Across eight visually and dynamically diverse tasks from the Object-Centric Visual RL benchmark, ManiSkill, Robosuite, and VizDoom, COMET achieves a higher mean normalized score during the early stages of training compared to object-centric and monolithic baselines.

Primary area:
reinforcement learning
Keywords:
Reinforcement learning, world model, object-centric representations, Monte Carlo tree search

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_EJE4OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces COMET, an object-centric, model-based reinforcement learning algorithm that performs MuZero-style Monte Carlo Tree Search within a slot-structured latent space. The method freezes an unsupervised object-centric encoder, employs a simple action-slot fusion mechanism to associate each object's latent information with the action via a shared projector, and utilizes causality-inspired attention mechanisms within the policy and value transformers to focus on task-relevant objects. Across eight tasks with diverse visual and dynamic characteristics, COMET achieves higher average normalized scores than both monolithic and object-centric baselines during the early stages of training while maintaining comparable performance in the later stages.

Critical strengths

This paper integrates MCTS based planning into an object centric latent space, a direction that is both timely and valuable for compositional reasoning and data efficiency, and the experiments cover a relatively comprehensive set of tasks (This point is important for my rating).

Critical weaknesses

1: Causality inspired attention has not been eliminated; the paper does not quantify its specific contribution to COMET's improvements, and results on the continuous control subset show that COMET underperforms compared to OCDreamer and UniZero; the paper's main argument is limited to the "early stage," with its advantages diminishing by the end of training on the full suite.

2: Recent work on object centric MCTS style planning, such as ObjectZero (slot based encoder plus GNN dynamics plus MuZero and EfficientZero style MCTS), has been overlooked; this would have served as a highly relevant and potentially stronger baseline for discrete and continuous tasks (This point is important for my rating).

Questions for authors

1: Is it possible to eliminate the causal heuristic attention within the value head and quantify its contribution across a representative subset of tasks?

2: Is it possible to conduct a direct comparison with object-centric MCTS planners (e.g., ObjectZero-style) to distinguish the respective advantages of Transformer dynamics combined with action-slot binding versus message-passing dynamics combined with action broadcasting?

2 comments

We thank the reviewer for a fair and precise assessment. Our responses are provided below.

Q1: Is it possible to eliminate the causal heuristic attention within the value head and quantify its contribution across a representative subset of tasks?

Causality-inspired attention modifies the standard transformer self-attention mechanism in the policy and value heads. To isolate its contribution, we compare COMET with a variant in which causality-inspired attention is replaced by standard self-attention.

Table 4 presents the results of this ablation across five tasks from the Object-Centric Visual RL benchmark, including both discrete and continuous action spaces. All five tasks use the SLATE encoder.

The results show that causality-inspired attention substantially improves performance on Object Reaching, while yielding comparable final performance on Object Goal, Object Comparison, and Property Comparison. On Object Interaction, the variant without causality-inspired attention performs better. These findings suggest that the effectiveness of causality-inspired attention depends on the task.


Table 4. Ablation of causality-inspired attention

Performance is reported as mean ± standard deviation at different environment interaction steps.

Object Reaching

Method0100000200000300000400000480000
COMET (w/o causality-inspired attention)0.233 ± 0.0540.521 ± 0.0150.523 ± 0.0330.624 ± 0.0620.745 ± 0.0430.691 ± 0.062
COMET0.300 ± 0.0270.755 ± 0.0150.757 ± 0.0320.769 ± 0.1650.874 ± 0.0620.855 ± 0.059

Object Goal

Method050000100000150000200000
COMET (w/o causality-inspired attention)0.200 ± 0.0270.928 ± 0.0160.989 ± 0.0160.978 ± 0.0211.000 ± 0.000
COMET0.222 ± 0.0570.961 ± 0.0160.989 ± 0.0160.994 ± 0.0080.989 ± 0.016

Object Interaction

Method0500000100000015000001950000
COMET (w/o causality-inspired attention)0.000 ± 0.0000.466 ± 0.0590.661 ± 0.0670.767 ± 0.0190.790 ± 0.041
COMET0.000 ± 0.0000.292 ± 0.0180.551 ± 0.0880.671 ± 0.0880.714 ± 0.015

Object Comparison

Method050000100000150000200000250000
COMET (w/o causality-inspired attention)0.244 ± 0.0160.655 ± 0.0550.900 ± 0.0540.972 ± 0.0210.955 ± 0.0310.979 ± 0.007
COMET0.211 ± 0.0570.715 ± 0.0900.966 ± 0.0000.945 ± 0.0161.000 ± 0.0000.961 ± 0.008

Property Comparison

Method050000100000150000200000250000
COMET (w/o causality-inspired attention)0.178 ± 0.0680.665 ± 0.0500.944 ± 0.0560.962 ± 0.0200.978 ± 0.0310.961 ± 0.032
COMET0.256 ± 0.0570.544 ± 0.0670.888 ± 0.0160.928 ± 0.0410.933 ± 0.0530.966 ± 0.013

Thank you for the additional ablation experiments. However, my second major concern remains unaddressed. In particular, the lack of discussion or comparison with closely related object-centric MCTS methods such as ObjectZero makes it difficult to assess the specific contribution of COMET's architecture. The limited advantages on continuous-control tasks and the diminishing performance gains at later training stages also remain concerns.

Overall, while the new ablation strengthens the empirical evaluation, I do not think it sufficiently resolves my main concerns regarding the method's novelty and comparative effectiveness. I therefore maintain my original rating.

Reviewer_M3XmOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

In this paper, the authors introduce COMET, a model-based RL method that performs MuZero-style Monte Carlo Tree Search in a slot-structured latent space. The method maps observations to slots with a frozen pretrained object-centric encoder, predicts next-step slots and rewards with a UniZero-style transformer world model in which the action embedding is concatenated with each slot, and uses policy and value heads with causality-inspired attention that reweights attention by learned per-slot relevance scores. Across eight tasks from the Object-Centric Visual RL benchmark, ManiSkill, Robosuite, and VizDoom, COMET achieves the highest mean normalized score during early training.

Critical strengths

[Critical Strengths]

  1. The authors propose an action-slot binding that conditions each slot on the action. This simple design removes the bottleneck of predicting all next-step slots from a single action token, and the ablation shows a sharp performance drop without it.

  2. The motivation is intuitive and well grounded. In many object-oriented environments, the reward depends on interactions among objects, yet only a few objects matter at each step, so bringing an explicit object-level inductive bias into MuZero-style planning is a natural direction. The approach pays off on the relational OCVRL tasks, where COMET converges faster than all baselines on Object Comparison and Property Comparison (Fig. 4) and achieves the highest mean normalized score at early training stages (Table 1).

[Minor Strengths]

  1. The evaluation covers eight tasks from four benchmarks with both discrete and continuous action spaces, and the authors candidly report the tasks where COMET underperforms.

  2. The appendices provide trajectory rollouts, per-slot score visualizations, and an online encoder fine-tuning study that addresses the frozen-encoder limitation.

Critical weaknesses

[Critical Weakness] (ordered by importance)

  1. The method largely combines existing components, namely frozen object-centric encoders, the UniZero pipeline, and STICA’s causality-inspired attention. The action–slot binding, the only part explicitly called novel, is closely related to Biza et al. (2022), as the authors themselves note. The authors should clarify what is new beyond this integration.

  2. The causality-inspired attention, the paper’s titular component, is never ablated. Please compare it with standard attention.

  3. STICA, the closest prior work, is not included as a baseline, so it is unclear what MCTS planning adds. Please compare with STICA or explain why this is not possible.

  4. Appendix K does not support the claim that COMET identifies task-relevant objects. In Object Goal, the target object scores lower than an irrelevant slot at every shown step (Figs. 12–13). Please tone down the causality framing or evaluate the scores quantitatively.

  5. The claim of consistent performance across tasks is not supported. On the continuous-control tasks, COMET’s mean score is below UniZero’s(Table 3), and at the end of training OCDreamer and OCRL perform better with the same encoders. Please analyze the cause.

  6. The aggregate scores have no confidence intervals although the gaps between top methods are small. Please report with confidence intervals in table 1 and 3.

  7. It is unclear whether OCDreamer uses slot pre-initialization, which is critical for COMET. Its near-zero success on Object Comparison and Property Comparison, despite strong continuous-control results, also raises concerns about its configuration. Please verify the baseline settings.

[Minor Weakness]

  1. Computational cost is reported only for COMET. Please compare training time and planning latency with other baselines.

  2. Please extend the ablations to the visually complex continuous-control tasks (Block Lifting, Cube Pushing), where COMET underperforms.

Questions for authors
  1. Why does COMET underperform on the continuous-control tasks?

  2. How are the scores α computed (architecture and output activation)?

  3. What exactly is the "w/o action-slot binding" variant in Fig. 7? Does it predict all n next-step slots from a single action token?

3 comments

We thank the reviewer for the thoughtful critique. We address the concerns and questions below and provide additional experimental results.

W2: The causality-inspired attention, the paper’s titular component, is never ablated. Please compare it with standard attention.

Causality-inspired attention modifies the standard transformer self-attention mechanism in the policy and value heads. To isolate its contribution, we compare COMET with a variant in which causality-inspired attention is replaced by standard self-attention.

Table 3 presents the results of this ablation across five tasks from the Object-Centric Visual RL benchmark, including both discrete and continuous action spaces. All five tasks use the SLATE encoder.

The results show that causality-inspired attention substantially improves performance on Object Reaching, while yielding comparable final performance on Object Goal, Object Comparison, and Property Comparison. On Object Interaction, the variant without causality-inspired attention performs better. These findings suggest that the effectiveness of causality-inspired attention depends on the task.

W3: STICA, the closest prior work, is not included as a baseline, so it is unclear what MCTS planning adds. Please compare with STICA or explain why this is not possible.

We initially planned to include STICA as an additional baseline. We attempted to train STICA using its official GitHub implementation, both on our tasks and on the tasks considered in the original STICA paper. However, we were unable to obtain successful training runs in either setting. We also contacted the authors of STICA for assistance but have not received a response. Consequently, we could not obtain reliable results for a direct comparison with STICA. We acknowledge that such a comparison would be valuable for isolating the contribution of MCTS-based planning and consider its absence a limitation of our evaluation.

W4: Appendix K does not support the claim that COMET identifies task-relevant objects. In Object Goal, the target object scores lower than an irrelevant slot at every shown step (Figs. 12–13). Please tone down the causality framing or evaluate the scores quantitatively.

In Figures 12 and 13, the causality-inspired attention mechanism assigns high relevance scores to the agent, represented by the red circle. This is consistent with the agent's role in the environment: its actions drive state transitions and directly influence the expected return.

Importantly, the learned causality scores should be interpreted as object-relevance weights for policy and value prediction rather than direct measures of an object's semantic importance or independently identified causal effects.

Moreover, the transformer backbone processes interactions among object slots before passing their representations to the policy and value heads. Therefore, information about the target object may already be incorporated into other slot representations, even when the target itself receives a low relevance score in the subsequent attention mechanism.

W7: It is unclear whether OCDreamer uses slot pre-initialization, which is critical for COMET. Its near-zero success on Object Comparison and Property Comparison, despite strong continuous-control results, also raises concerns about its configuration. Please verify the baseline settings.

We use slot pre-initialization for all object-centric methods except OCRL in the original experiments. Thus, OCDreamer and COMET receive object-centric representations with the same slot pre-initialization procedure. We did not perform task-specific hyperparameter tuning for OCDreamer or the other baselines. Instead, we followed the hyperparameter configurations provided in their original publications and official implementations, using a fixed configuration across tasks. We acknowledge that task-specific hyperparameter tuning could potentially improve OCDreamer's performance, particularly on Object Comparison and Property Comparison.

Q3: What exactly is the "w/o action-slot binding" variant in Fig. 7? Does it predict all n next-step slots from a single action token?

Yes. The "w/o action–slot binding" variant follows the original UniZero-style approach, in which a single action token is processed by the transformer backbone. Its resulting latent representation is then used to predict latent representations for all objects. In contrast, COMET concatenates the action embedding with each object slot independently and processes the resulting vectors through a shared MLP projector. This produces N slot-conditioned action embeddings, allowing the model to predict the next state of each slot from its corresponding action-conditioned representation.

The ablation therefore evaluates whether explicitly conditioning each object slot on the action improves transition modeling compared to predicting all object slots from a single action-token representation.

Table 3. Ablation of causality-inspired attention

Performance is reported as mean ± standard deviation at different environment interaction steps.

Object Reaching

Method0100000200000300000400000480000
COMET (w/o causality-inspired attention)0.233 ± 0.0540.521 ± 0.0150.523 ± 0.0330.624 ± 0.0620.745 ± 0.0430.691 ± 0.062
COMET0.300 ± 0.0270.755 ± 0.0150.757 ± 0.0320.769 ± 0.1650.874 ± 0.0620.855 ± 0.059

Object Goal

Method050000100000150000200000
COMET (w/o causality-inspired attention)0.200 ± 0.0270.928 ± 0.0160.989 ± 0.0160.978 ± 0.0211.000 ± 0.000
COMET0.222 ± 0.0570.961 ± 0.0160.989 ± 0.0160.994 ± 0.0080.989 ± 0.016

Object Interaction

Method0500000100000015000001950000
COMET (w/o causality-inspired attention)0.000 ± 0.0000.466 ± 0.0590.661 ± 0.0670.767 ± 0.0190.790 ± 0.041
COMET0.000 ± 0.0000.292 ± 0.0180.551 ± 0.0880.671 ± 0.0880.714 ± 0.015

Object Comparison

Method050000100000150000200000250000
COMET (w/o causality-inspired attention)0.244 ± 0.0160.655 ± 0.0550.900 ± 0.0540.972 ± 0.0210.955 ± 0.0310.979 ± 0.007
COMET0.211 ± 0.0570.715 ± 0.0900.966 ± 0.0000.945 ± 0.0161.000 ± 0.0000.961 ± 0.008

Property Comparison

Method050000100000150000200000250000
COMET (w/o causality-inspired attention)0.178 ± 0.0680.665 ± 0.0500.944 ± 0.0560.962 ± 0.0200.978 ± 0.0310.961 ± 0.032
COMET0.256 ± 0.0570.544 ± 0.0670.888 ± 0.0160.928 ± 0.0410.933 ± 0.0530.966 ± 0.013

Thank you for the detailed response. In particular, the ablation study comparing causality-inspired attention with standard self-attention has resolved my concerns about causality-inspired attention. However, I could not find a response addressing my concern about novelty. I also remain concerned that no comparison has been made with STICA, which is one of the most closely related approaches. In addition, the mean and variance should be computed over multiple independent seeds, rather than over results measured at different time steps. For these reasons, I will maintain my original score.

Reviewer_banyOpenReview ↗
Rating
2
Confidence
3
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

The paper proposes COMET, a MuZero/UniZero-style model-based RL agent that plans with MCTS in a slot-structured latent space. Observations are encoded by a frozen, separately pretrained object-centric encoder (SLATE, DINOSAUR, or Slot Contrast, depending on the task), with slot initialization propagated across timesteps for temporal consistency. A nanoGPT-style transformer backbone processes slots and "slot-conditioned action embeddings," formed by concatenating the action with each slot and passing the result through a shared MLP, to predict next-step slots and reward. Policy and value heads are transformers that reuse STICA's causality-inspired attention, which modulates attention by learned per-slot relevance scores. Experiments cover eight tasks from the OCRL benchmark, ManiSkill, Robosuite, and VizDoom, comparing against OCRL, OCDreamer, SOLD, and UniZero. The headline result is a higher mean normalized score at 33% and 66% of the training budget, supported by ablations of action–slot binding and slot pre-initialization on one or two tasks and an encoder fine-tuning experiment under distribution shift.

Critical strengths
  • Bringing slot-structured latents into a UniZero-style search-based planner is a reasonable and underexplored direction, as most prior object-centric MBRL work builds on Dreamer-style imagination. The ablation in Figure 7 convincingly shows that per-slot action conditioning substantially outperforms a single action token. This is a concrete and transferable design insight for anyone building slot-based transformer world models, and it is the paper's most consequential technical contribution.

  • The evaluation spans 2D/3D OCRL tasks, ManiSkill, Robosuite, and VizDoom, with both discrete and continuous action spaces, and compares against object-centric and monolithic, model-based and model-free baselines. This breadth gives a realistic picture of where the approach helps (visually simple, single-target discrete tasks) and where it does not (Block Lifting and Cube Pushing). I also appreciate that the authors report the unfavorable results candidly in Table 3 and Section 4.

Critical weaknesses
  • The central claims are not yet isolated by the evidence. Without an ablation replacing the causality-inspired attention with standard self-attention, it is difficult to attribute gains to this mechanism specifically. The aggregate advantage is also narrow: COMET is not best at the final budget (0.758 vs. 0.778 for OCRL), the margin at 66% is small, and only three seeds are used. On the continuous-control subset, COMET trails OCDreamer substantially (0.499 vs. 0.798), suggesting the aggregate is driven mainly by the visually simple discrete tasks.

  • The methodological novelty is limited, as most components are inherited from prior work and the action–slot fusion closely resembles existing soft action–object binding. This places more weight on the empirical comparison, where some baseline results raise questions. OCDreamer stays near zero on two tasks despite sharing the same encoder, and I wonder whether its configuration transfers well to these tasks. In addition, if the baselines do not receive the slot pre-initialization that the ablation shows is essential for COMET, part of the reported advantage may stem from input preprocessing rather than the method itself.

Questions for authors

Q1. Could you ablate the causality-inspired attention by replacing it with standard self-attention, at least on the three SLATE discrete tasks?

Q2. Do the object-centric baselines receive slots with the same pre-initialization as COMET? If not, could you rerun OCRL and OCDreamer with identical slot inputs?

Q3. Why does OCDreamer stay near zero on Object Comparison and Property Comparison, and were its hyperparameters verified or tuned for these tasks?

Q4. Could you report results with at least five seeds and bootstrap confidence intervals or IQM for Table 1?

Q5. In Figure 12, the target object receives near-zero scores throughout. How does this square with the claim that the scores identify task-relevant objects?

4 comments

We thank the reviewer for the detailed and constructive feedback. We address the questions and concerns below and provide additional experimental results.

Q1: Could you ablate the causality-inspired attention by replacing it with standard self-attention, at least on the three SLATE discrete tasks? W1: The central claims are not yet isolated by the evidence. Without an ablation replacing the causality-inspired attention with standard self-attention, it is difficult to attribute gains to this mechanism specifically.

Causality-inspired attention modifies the standard transformer self-attention mechanism in the policy and value heads. To isolate its contribution, we compare COMET with a variant in which causality-inspired attention is replaced by standard self-attention.

Table 1 presents the results of this ablation across five tasks from the Object-Centric Visual RL benchmark, including both discrete and continuous action spaces. All five tasks use the SLATE encoder.

The results show that causality-inspired attention substantially improves performance on Object Reaching, while yielding comparable final performance on Object Goal, Object Comparison, and Property Comparison. On Object Interaction, the variant without causality-inspired attention performs better. These findings suggest that the effectiveness of causality-inspired attention depends on the task.

Q2: Do the object-centric baselines receive slots with the same pre-initialization as COMET? If not, could you rerun OCRL and OCDreamer with identical slot inputs? W1: In addition, if the baselines do not receive the slot pre-initialization that the ablation shows is essential for COMET, part of the reported advantage may stem from input preprocessing rather than the method itself.

We use slot pre-initialization for all object-centric methods except OCRL in the original experiments. To address the reviewer's concern, we additionally evaluate OCRL with and without slot pre-initialization on two discrete-control and two continuous-control tasks.

Table 2 presents the results. OCRL achieves comparable performance with and without slot pre-initialization across two tasks with discrete action space (Object Comparison, Object Interaction) and two tasks with continuous action space (Object Reaching and Block Lifting).

This result is consistent with our expectations. The primary purpose of slot pre-initialization is to maintain temporal consistency of object representations by preventing slot permutations between consecutive observations within an episode. This property is particularly important for model-based RL algorithms, which learn transition models that predict the future states of individual object slots.

In contrast, OCRL is a model-free algorithm and does not learn an explicit slot-level transition model. Consequently, its performance is less sensitive to temporal slot permutations, which explains why slot pre-initialization has little effect on its learning performance.

These additional experiments indicate that the absence of slot pre-initialization does not substantially disadvantage OCRL in our comparisons.

Thus, the low score assigned to the target object does not necessarily imply that its information is ignored during decision-making. Nevertheless, the visualization primarily demonstrates that the mechanism emphasizes the agent, rather than establishing that it assigns high scores to every semantically relevant object.

Table 1. Ablation of causality-inspired attention (Q1, W1)

Performance is reported as mean ± standard deviation at different environment interaction steps.

Object Reaching

Method0100000200000300000400000480000
COMET (w/o causality-inspired attention)0.233 ± 0.0540.521 ± 0.0150.523 ± 0.0330.624 ± 0.0620.745 ± 0.0430.691 ± 0.062
COMET0.300 ± 0.0270.755 ± 0.0150.757 ± 0.0320.769 ± 0.1650.874 ± 0.0620.855 ± 0.059

Object Goal

Method050000100000150000200000
COMET (w/o causality-inspired attention)0.200 ± 0.0270.928 ± 0.0160.989 ± 0.0160.978 ± 0.0211.000 ± 0.000
COMET0.222 ± 0.0570.961 ± 0.0160.989 ± 0.0160.994 ± 0.0080.989 ± 0.016

Object Interaction

Method0500000100000015000001950000
COMET (w/o causality-inspired attention)0.000 ± 0.0000.466 ± 0.0590.661 ± 0.0670.767 ± 0.0190.790 ± 0.041
COMET0.000 ± 0.0000.292 ± 0.0180.551 ± 0.0880.671 ± 0.0880.714 ± 0.015

Object Comparison

Method050000100000150000200000250000
COMET (w/o causality-inspired attention)0.244 ± 0.0160.655 ± 0.0550.900 ± 0.0540.972 ± 0.0210.955 ± 0.0310.979 ± 0.007
COMET0.211 ± 0.0570.715 ± 0.0900.966 ± 0.0000.945 ± 0.0161.000 ± 0.0000.961 ± 0.008

Property Comparison

Method050000100000150000200000250000
COMET (w/o causality-inspired attention)0.178 ± 0.0680.665 ± 0.0500.944 ± 0.0560.962 ± 0.0200.978 ± 0.0310.961 ± 0.032
COMET0.256 ± 0.0570.544 ± 0.0670.888 ± 0.0160.928 ± 0.0410.933 ± 0.0530.966 ± 0.013

Table 2. Ablation of slot pre-initialization for OCRL (Q2, W1)

Performance is reported as mean ± standard deviation at different environment interaction steps.

Object Comparison

Method050000100000150000200000250000
OCRL (w/o pre-initialization)0.247 ± 0.0130.256 ± 0.0130.333 ± 0.0560.451 ± 0.0450.576 ± 0.0380.766 ± 0.018
OCRL0.259 ± 0.0190.265 ± 0.0230.318 ± 0.0210.432 ± 0.0580.588 ± 0.0510.756 ± 0.029

Object Interaction

Method0500000100000015000001950000
OCRL (w/o pre-initialization)0.003 ± 0.0030.063 ± 0.0320.303 ± 0.0930.598 ± 0.2140.791 ± 0.091
OCRL0.005 ± 0.0030.083 ± 0.0450.333 ± 0.0780.562 ± 0.1890.802 ± 0.109

Object Reaching

Method0100000200000300000400000480000
OCRL (w/o pre-initialization)0.207 ± 0.0170.273 ± 0.0350.629 ± 0.0960.845 ± 0.0080.890 ± 0.0260.910 ± 0.013
OCRL0.163 ± 0.0370.251 ± 0.0550.649 ± 0.0730.823 ± 0.0290.899 ± 0.0310.922 ± 0.021

Block Lifting (cumulative return)

Method0100000200000300000400000480000
OCRL (w/o pre-initialization)11.9 ± 1.125.7 ± 0.630.1 ± 1.936.9 ± 2.539.7 ± 2.340.1 ± 1.4
OCRL13.1 ± 1.022.5 ± 0.933.1 ± 2.633.5 ± 2.942.9 ± 3.142.2 ± 2.1

Q3: Why does OCDreamer stay near zero on Object Comparison and Property Comparison, and were its hyperparameters verified or tuned for these tasks? W2: OCDreamer stays near zero on two tasks despite sharing the same encoder, and I wonder whether its configuration transfers well to these tasks.

We did not perform task-specific hyperparameter tuning for OCDreamer or the other baselines. Instead, we followed the hyperparameter configurations provided in their original publications and official implementations, using a fixed configuration across tasks. We acknowledge that task-specific tuning could potentially improve its performance.

Q5: In Figure 12, the target object receives near-zero scores throughout. How does this square with the claim that the scores identify task-relevant objects?

In Figure 12, the causality-inspired attention mechanism assigns high relevance scores to the agent, represented by the red circle. This is consistent with the agent's role in the environment: its actions drive state transitions and directly influence the expected return.

Importantly, the learned causality scores should be interpreted as object-relevance weights for policy and value prediction rather than direct measures of an object's semantic importance or independently identified causal effects.

Moreover, the transformer backbone processes interactions among object slots before passing their representations to the policy and value heads. Therefore, information about the target object may already be incorporated into other slot representations, even when the target itself receives a low relevance score in the subsequent attention mechanism.

Thank you for the thoughtful response and the new experiments.

The pre-initialization results resolve that concern, and the Fig. 12 clarification helps. The ablation suggests the causality-inspired attention helps mainly on Object Reaching, so its role could be framed more modestly. The seed count, the OCDreamer tuning, and the continuous-control gap remain open.

I will keep my score, and I find the direction promising.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.