- Rating
- 3
- Confidence
- 2
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper studies whether factorizing object representations into explicit attributes helps policy learning, beyond simply separating objects. The authors introduce COMB, an object-centric world model built on particle based model and extended to online model-based RL. They evaluate it on eight 2D and 3D control tasks, including held-out attribute combinations. They also introduce an entanglement dosing experiment that applies invertible transformations to frozen representations, preserving information while changing how easily policy heads can access individual attributes. The results suggest that nonlinear entanglement hurts policy learning and compositional transfer, while linear transformations have little effect.
Critical strengths
The main strength is the controlled analysis in Sections 6 and 7. Rather than relying only on end-to-end comparisons, the authors use invertible transformations to change representation structure while preserving information content. The placebo and conditioning-matched linear controls make the results more convincing. I also appreciate the matched-skill OOD evaluation, which helps distinguish poor generalization from slower policy learning.
Also the paper offers a useful insight which is that structually disentangling attributes is not sufficient by itself but what matters is whether the downstream policy can easily read them. I find this analysis more valuable than the architectural contribution of COMB itself.
Critical weaknesses
While I find the analysis interesting, I have some concerns.
-
Limited architectural novelty : I think COMB largely builds on existing factorized particle representations and combines them with standard components for model-based RL. The main architectural contribution seems to be extending DDLP to online policy learning. I therefore see the contribution primarily in the experimental analysis, not the proposed architecture.
-
The interpretation of the entanglement experiments : The invertible transformations preserve information, but nonlinear scrambling also makes the representation harder for the policy heads to decode. While the linear controls address conditioning, they did not rule out increased function-approximation or optimization difficulty as an explanation for the performance gap. So I think it is less clear how broadly this conclusion holds with more expressive heads or different optimization settings.
Questions for authors
-
The analysis focuses on position and appearance, which have relatively clear boundaries in the current representation. How would the proposed findings extend to attributes that interact more strongly, such as object geometry, contact state, or occlusion? In particular, what aspects of the current conclusions depend on the explicit factorization already provided by DDLP?
-
The entanglement experiments show that representation format matters even when information is preserved. However, to what extent should this be interpreted as a benefit of factorization itself, rather than compatibility between the representation coordinates and the downstream policy architecture? How should this distinction affect the broader claim that factorized representations are preferable for policy learning?
-
The entanglement intervention starts from a successfully learned COMB representation and applies an invertible nonlinear transformation. How far can the resulting performance differences be generalized to representations that are learned in an entangled form from the outset? Does the intervention primarily characterize the cost of nonlinear coordinate transformations, rather than the broader benefits of learning factorized representations?
2 comments
Thank you for the detailed rebuttal and additional experiments. I appreciate the authors' efforts to address my concerns, particularly the additional analyses on head capacity, training budget, and data coverage. The clarification that the main benefit lies in the linear readability of task-relevant attributes, rather than factorization alone, is also helpful.
While some limitations remain regarding architectural novelty and generalization to representations learned in an entangled form, I find the controlled analysis of representation format and its impact on policy learning to be a meaningful contribution.
Overall, the rebuttal has clarified several of my concerns, and I maintain my positive rating of 3.
We thank the reviewer for their time and effort, and we appreciate the relevant questions, which we address below.
Novelty (W1). We agree that the controlled study is the paper's main contribution, as now indicated in the introduction. We see COMB as a secondary contribution: to the best of our knowledge, it is the first world model with factorized per-object attributes trained with online RL (Tab. 2). This is not a direct extension of DDLP, which is trained offline on fixed video. For online RL, the policy must act on an unordered set of particles, and reward gradients must not write relations between objects into individual particles. This required action-conditioned particle dynamics, a transformer aggregator for each head, a reward stop-gradient, and an actor–critic with MPPI (App. C). COMB achieves the best or tied-best return on all eight tasks and all six held-out splits (Fig. 2, Tab. 1).
Format or optimization difficulty? (W2).
Offline (App. H.3): Using the test from Prop. 1, we varied head size, training steps, and the amount of data. At both doses, the large head fits the training data (error ≈10−4), so fitting is not the issue. At the strongest dose, error on unseen combinations stops improving once the head reaches width 256 and is unchanged even with 16× more training steps. Beyond that, only more data reduces the error, and slowly: the unscrambled code trained on 1/16 of the data still transfers far better (0.004) than the strongly scrambled one trained on all of it (0.071). At the milder dose, every MLP head already transfers well (0.004–0.008, against 0.12 for a linear head). Capacity alone does not close the gap; the format determines how much data the head needs. This is consistent with Redhardt et al. (NeurIPS 2025), where large networks overcome a nonlinear encoding when training covers the task space well; in RL, that coverage is exactly what is limited.
Online (App. H.4): On OCRL-OOO-Hard, we increased the actor, critic, and reward heads by 2.75×, at the strongest dose and the paper's training budget. Both arms improved by a similar amount (+11.5 and +9.8 points), so the absolute gap remained (31.6 points, against 29.9 with the original heads; p=0.005). On held-out combinations at matched skill, the dosed policy still loses 37% of COMB's return (46% with the original heads), compared with 3% for COMB (p=2⋅10−4). For both head sizes, the deficit remains stable between 100k and 250k steps (Tab. 10, App. H.4).
Interacting attributes (Q1). Contact is a relation between objects: COMB's dynamics model captures it through attention across particles, and the aggregator computes it for the heads, rather than storing it in an individual particle. Geometry and rotation share the appearance code with colour, so transfer for these attributes depends on the encoder (App. G.1). Occlusion violates the one-particle-per-object assumption and is outside our scope (Sec. 8). The readability criterion itself, however, does not depend on DDLP.
Factorization or compatibility with the head? (Q2). We read it as compatibility with the head. What matters is that the task-relevant attributes are linearly readable by the head's first layer (Prop. 1b). Standard heads begin with a linear layer, so this applies broadly, and recent theory reaches a closely related requirement for linear readouts (Uselis et al., ICML 2026). This does narrow the broader claim. A factorized representation is preferable when its factorization makes the task-relevant attributes linearly readable, as COMB's does for position. A factorization the head cannot read linearly does not help: warping each attribute within its own coordinates keeps the code factorized, yet still hurts transfer (Tab. 7). We have made this explicit in the abstract and conclusion.
Scrambled vs. learned-entangled representations (Q3). The entanglement dial measures the cost of changing coordinates while keeping the information fixed; it does not measure the full benefit of learning a factorized representation. We now state this explicitly. As a link to learned representations, we applied the probe to the baselines, which are entangled from the outset (App. H.2):
On the multi-object tasks, position readability follows the same ordering as final in-distribution return (Fig. 2). Colour is harder for all models, but COMB still reads it best. We view this as a consistency check, not a causal result: the baselines also differ in their encoders and training objectives.