- Rating
- 3
- Confidence
- 3
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes ECLAIR, an entity-centric exploration framework for sparse-reward visual RL. It combines unsupervised object-centric representations with latent dynamics prediction to construct entity-level intrinsic rewards. The paper also connects prediction errors to pseudo-count-based exploration and analyzes the benefits of factored reward structures. Experiments across MiniHack, SpriteWorld, and OGBench show promising improvements over established baselines.
Critical strengths
-
Effective integration of entity-centric representations and latent dynamics. ECLAIR combines object-centric representations with latent dynamics to provide structured exploration signals. Its online training mechanism allows representations and dynamics models to adapt to newly encountered states, extending approaches that rely on fixed pretrained representations.
-
Strong empirical results and useful analyses. ECLAIR achieves promising results across diverse environments, particularly on challenging long-horizon tasks. The ablations support the importance of online fine-tuning and KL clipping, while the visualizations suggest that intrinsic rewards align with meaningful object interactions.
Critical weaknesses
-
The contribution of entity-centric factorization could be better isolated. Although patch-based and slot-based variants are evaluated, differences in model architectures and experimental settings make it difficult to isolate the benefits of entity-centric factorization from those of the underlying dynamics model. More controlled comparisons would strengthen the central claim.
-
Limited cross-domain validation of key design choices. While the paper evaluates alternative representations across several tasks, the main component-level ablations, including online fine-tuning and prediction horizon, are primarily conducted on Scene-T1. It remains unclear whether these findings generalize consistently across different domains. Extending these ablations to other environments would strengthen the conclusions.
Questions for authors
-
Relationship to prior entity-centric RL methods. Could the authors clarify how ECLAIR differs from ECRL (2024) and HECRL (2026), particularly given that HECRL also addresses sparse-reward manipulation? A brief discussion emphasizing ECLAIR's focus on online exploration would help clarify its contribution.
-
Relationship to JEPA-style predictive models. How does ECLAIR relate to JEPA-style latent predictive models? Could similar exploration signals be obtained from a JEPA-style predictor, and what specific advantages does entity-centric factorization provide over monolithic latent prediction?
3 comments
Author's response (continued)
Relationship to ECRL and HECRL.
ECRL and HECRL use DLP-style entity representations inside the policy and value function for goal-conditioned manipulation. ECLAIR addresses a different problem, which is exploration in sparse-reward tasks without goals or demonstrations, and uses entity structure in the intrinsic reward.
The second key difference is that ECLAIR fine-tunes its entity representation online. An exploring agent keeps reaching configurations that its initial data does not cover, so the representation must adapt online. This is a major challenge that ECRL and HECRL do not address, as both prior works learn a frozen representation from a fixed offline dataset and never fine-tune it online. Our ablations confirm that this adaptation matters in every domain (Table R2).
Finally, the two lines of work are complementary. We can leverage ECLAIR to provide an exploration signal for ECRL/HECRL-style learners. We will leave this for future work. We have added this comparison to Appendix B.
Relationship to JEPA-style predictors.
JEPA-style models (LeCun, 2022; Assran et al., 2023) predict in a learned latent space without reconstruction. ECLAIR's entity model is trained with reconstruction, which anchors each entity latent to objects in the image better. When learning online from a changing data distribution, this mitigates the problem of latent space drifting or collapsing. The closest object-centric JEPA we are aware of is C-JEPA (Nam et al., 2026), which we found difficult to adapt to our online exploration setting. We leave the incorporation of JEPA-related world models to future work and have added JEPA and C-JEPA to Appendix B.
We hope to have addressed all the concerns from the reviewer, and we welcome any further comments.
References
- Assran, M., et al. (2023). Self-Supervised Learning from Images with a Joint-Embedding Predictive Architecture. CVPR.
- Daniel, T., & Tamar, A. (2022). Unsupervised Image Representation Learning with Deep Latent Particles. ICML.
- Haramati, D., Daniel, T., & Tamar, A. (2024). Entity-Centric Reinforcement Learning for Object Manipulation from Pixels. ICLR.
- Haramati, D., et al. (2026). Hierarchical Entity-centric Reinforcement Learning with Factored Subgoal Diffusion. ICLR.
- Kipf, T., et al. (2022). Conditional Object-Centric Learning from Video. ICLR.
- LeCun, Y. (2022). A Path Towards Autonomous Machine Intelligence. OpenReview preprint.
- Mosbach, M., et al. (2025). SOLD: Slot Object-Centric Latent Dynamics Models for Relational Manipulation Learning from Pixels. ICML.
- Nam, H., et al. (2026). Causal-JEPA: Learning World Models through Object-Level Latent Masking. ICML.
Thanks to the authors for the response.
The authors have addressed most of my concerns. I will keep my positive score.
We thank the reviewer for their positive assessment of the paper and their constructive feedback. We address each concern below.
We clarify that the patch variant is the closest controlled comparison. Architecturally, it changes only the encoder, from object-centric particles to a fixed grid of image patches. The dynamics module, the number of latents, and the latent dimension are the same as in ECLAIR. The prediction horizon H is tuned separately for each variant over {1,5,10}, and we report the best setting. For patches, H = 1 performed best, so the reported result reflects its strongest configuration, at a higher compute cost. Similarly, for every method, we search the intrinsic reward scale β over {0.1,1.0,2.0} per domain and report the value with the best performance, which is β=0.1 for patches. Under this comparison, the entity-centric encoder improves the reward from 0.89 to 0.99 on MultiRoom-N4 and from 0.78 to 0.89 on MultiRoom-N6 (Table 2).
For slots, we made a substantial effort to incorporate a second entity-centric approach into the framework. No latent world model for slots comparable to LPWM exists, so we used the existing SOLD implementation (Mosbach et al., 2025), which pairs a SAVi encoder (Kipf et al., 2022) with a deterministic slot dynamics model. Its bonus is therefore the prediction error (MSE) in slot space. Under equal, fixed isotropic variance, the KL divergence between two Gaussians reduces to a scaled squared error, so slot-space MSE is the fixed-variance special case of our latent-KL bonus rather than a different objective.
The goal of these variants is to show that the framework applies across entity-centric representations, not to rank the representations themselves. We have clarified this in Appendix C.1.
Following the reviewer's suggestion, we extended the ablations to MultiRoom-N4, MultiRoom-N6, and Scene-T2 (Table R2 in the general response; Appendix G). The main conclusions generalize across domains. Online fine-tuning matters in every domain. KL clipping is important in the Scene manipulation tasks and neutral in MultiRoom. ECLAIR is robust to the prediction horizon on MultiRoom and Scene-T1, while a longer horizon helps on Scene-T2. We thank the reviewer for suggesting these experiments, which we believe strengthen the paper.