- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper introduces COMET, an object-centric, model-based reinforcement learning algorithm that performs MuZero-style Monte Carlo Tree Search within a slot-structured latent space. The method freezes an unsupervised object-centric encoder, employs a simple action-slot fusion mechanism to associate each object's latent information with the action via a shared projector, and utilizes causality-inspired attention mechanisms within the policy and value transformers to focus on task-relevant objects. Across eight tasks with diverse visual and dynamic characteristics, COMET achieves higher average normalized scores than both monolithic and object-centric baselines during the early stages of training while maintaining comparable performance in the later stages.
Critical strengths
This paper integrates MCTS based planning into an object centric latent space, a direction that is both timely and valuable for compositional reasoning and data efficiency, and the experiments cover a relatively comprehensive set of tasks (This point is important for my rating).
Critical weaknesses
1: Causality inspired attention has not been eliminated; the paper does not quantify its specific contribution to COMET's improvements, and results on the continuous control subset show that COMET underperforms compared to OCDreamer and UniZero; the paper's main argument is limited to the "early stage," with its advantages diminishing by the end of training on the full suite.
2: Recent work on object centric MCTS style planning, such as ObjectZero (slot based encoder plus GNN dynamics plus MuZero and EfficientZero style MCTS), has been overlooked; this would have served as a highly relevant and potentially stronger baseline for discrete and continuous tasks (This point is important for my rating).
Questions for authors
1: Is it possible to eliminate the causal heuristic attention within the value head and quantify its contribution across a representative subset of tasks?
2: Is it possible to conduct a direct comparison with object-centric MCTS planners (e.g., ObjectZero-style) to distinguish the respective advantages of Transformer dynamics combined with action-slot binding versus message-passing dynamics combined with action broadcasting?
2 comments
Thank you for the additional ablation experiments. However, my second major concern remains unaddressed. In particular, the lack of discussion or comparison with closely related object-centric MCTS methods such as ObjectZero makes it difficult to assess the specific contribution of COMET's architecture. The limited advantages on continuous-control tasks and the diminishing performance gains at later training stages also remain concerns.
Overall, while the new ablation strengthens the empirical evaluation, I do not think it sufficiently resolves my main concerns regarding the method's novelty and comparative effectiveness. I therefore maintain my original rating.
We thank the reviewer for a fair and precise assessment. Our responses are provided below.
Q1: Is it possible to eliminate the causal heuristic attention within the value head and quantify its contribution across a representative subset of tasks?
Causality-inspired attention modifies the standard transformer self-attention mechanism in the policy and value heads. To isolate its contribution, we compare COMET with a variant in which causality-inspired attention is replaced by standard self-attention.
Table 4 presents the results of this ablation across five tasks from the Object-Centric Visual RL benchmark, including both discrete and continuous action spaces. All five tasks use the SLATE encoder.
The results show that causality-inspired attention substantially improves performance on Object Reaching, while yielding comparable final performance on Object Goal, Object Comparison, and Property Comparison. On Object Interaction, the variant without causality-inspired attention performs better. These findings suggest that the effectiveness of causality-inspired attention depends on the task.
Table 4. Ablation of causality-inspired attention
Performance is reported as mean ± standard deviation at different environment interaction steps.
Object Reaching
Object Goal
Object Interaction
Object Comparison
Property Comparison