- Rating
- 3(+1)
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 1
Read review
Paper summary
The paper studies evaluation of world models, with the goal of separating systematic generalization from behavioral shortcut learning. The authors view the problem from the perspective of compositional generalization. The authors formalize this as a setting where the test set trajectories remains within the Cartesian product of the training marginal supports of the individual latent factors and actions and targets the (state, action) pairs which are absent in the training data. The authors instantiate the framework empirically as a benchmark AnyGoal containing 4 control environments with 8 test variants each containing 400 episodes. For measuring the coverage the authors use some standard measures (coordinate range, distance, etc) and Gaussian KDE for the joint state-action novelty. To demonstrate how AnyGoal can be used in practice, the authors benchmark a few different types of world models based on the rollout MSE as well as CEM based MPC planning success rate. The empirical results on AnyGoal provide a few interesting observations. Even with ground truth physical observations, world models fail to capture interaction dynamics resulting on planning and rollout prediction on contact-rich goal shifts. End-to-end visual models perform even more poorly due to observation level compositional shifts. The paper concludes with a theoretical result on the sufficient conditions for compositional generalization showing that if each latent factor's transition depends only on a sparse subset of local coordinates represented in training then the global compositional error is bounded.
Critical strengths
- I like the broader question that the paper is exploring and also agree with the compositional generalization framing being appropriate for systematic evaluations. The goal shifted environments which act on specific marginal factors are a clean and intuitive way to test generalization. I also think the experiments decoupling the representation and dynamics failures via oracle state-predictors are quite insightful.
- I think some of the analysis is also quite insightful. For eg. the counterfactual action-intervention analysis is a useful diagnostic for action sensitivity. The experiment in Section 5.3 also nicely isolates representation geometry effects when visual observations are strictly in-distribution, as well as how rollout MSE and MPC planning success can diverge depending on task tolerance and representation space.
Critical weaknesses
- There seems to be a critical issue in the empirical results by using KDE for measuring the combinatorial gap. The authors use fit an unregularized Gaussian KDE to vectors in 18 and 33D reporting low joint in-distribution overlap (Fig 4b, Table 4). However, there is no control reported for a held-out test set from the base task itself. In 33D, even an i.i.d. held-out split from the base task would yield near-0% overlap under this estimator. On the other hand in Appendix C, high-dimensional articulated states are sliced into independent 1D marginals for the marginal support check ignoring kinematic configurations and joint coordination. Overall, despite the generally correct formal setting, the actual implementation seems lacking.
- Another major issue in my opinion is the theoretical result (Prop 1) seems trivial with major assumptions. Assumption 4 provides a learned model with the exact decoupled structure and Assumption 4 ensures every test input is inside the training support, making the result trivial. It is also very directly follows from prior work [1, 2, 3].
[1] Compositional Generalization from First Principles, Wiedemer et al. NeurIPS 2023.
[2] Efficient Reinforcement Learning in Factored MDPs, Kearns and Koller, IJCAI 1999.
[3] Provably Efficient Causal Model-Based Reinforcement Learning for Systematic Generalization. Mutti et al. AAAI 2023.
Questions for authors
- Could you evaluate a held-out set from the base task with the Gaussian-KDE?
- According to Table 5 the 10D oracle state for
Push-Tincludes the 4D goal pose (Goal position (x,y), goal orientation). Why is the task goal included in the state input of a dynamics predictor, and what exact values are passed into those 4 coordinates during evaluation onAvoid-T(where the goal is a corner region, not a T-block pose),Stack-T, andPush-T Antigoal? Does removing the 4D goal from thePush-Toracle state change the degradation onStack-TandPush-T Antigoal?
3 comments
Response 2
Q2. 4D goal components for Push-T oracle experiment.
Thank you for catching a flaw in our oracle-state experimental setup. The 4D goal pose should not have been included in a predictor meant to test physical dynamics given the state and action. Those were goal block related values which were constant throughout experiment. We apologize for this mistake. Your observation prompted us to rerun the experiment without the goal input, making this analysis more rigorous. We report the corrected results below.
- Comparison of planning success rate
| Task | 10D AdaLN | 10D Action Dup. | 10D Toy-Drift | 6D AdaLN | 6D Action Dup. | 6D Toy-Drift |
|---|---|---|---|---|---|---|
| Push-T | 92.7±1.2 | 92.0±2.0 | 89.3±3.1 | 91.3±1.2 | 91.3±2.3 | 88.0±2.0 |
| Avoid-T | 100.0±0.0 | 100.0±0.0 | 99.3±1.2 | 100.0±0.0 | 100.0±0.0 | 91.3±2.3 |
| Stack-T | 70.7±3.1 | 66.7±7.0 | 54.0±2.0 | 72.7±4.2 | 69.3±9.2 | 33.3±6.4 |
| Push-T Antigoal | 79.3±5.0 | 81.3±3.1 | 75.3±5.0 | 80.7±1.2 | 82.0±5.3 | 40.0±8.0 |
For AdaLN and Action Dup., the planning results change very little. Toy-DriftWorld shows a decline. Notably, it does not change our overall interpretation of the experimental results. We will update those results.
- Comparison of one-step (5 environment step) rollout error.
| variant | model | 10d_mse | 10d_normalized_mse | 6d_mse | 6d_normalized_mse |
|---|---|---|---|---|---|
| pusht | adaln | 0.0000050 | 0.0000005 | 0.0000075 | 0.0000013 |
| pusht | action_duplicate | 0.0000280 | 0.0000028 | 0.0000331 | 0.0000055 |
| pusht | toy-drift | 0.0000660 | 0.0000066 | 0.0007132 | 0.0001189 |
| avoidt | adaln | 0.0000063 | 0.0000006 | 0.0000082 | 0.0000014 |
| avoidt | action_duplicate | 0.0000350 | 0.0000035 | 0.0000272 | 0.0000045 |
| avoidt | toy-drift | 0.0000871 | 0.0000087 | 0.0001539 | 0.0000257 |
| stackt | adaln | 0.0025057 | 0.0002506 | 0.0048609 | 0.0008102 |
| stackt | action_duplicate | 0.0031794 | 0.0003179 | 0.0046214 | 0.0007702 |
| stackt | toy-drift | 0.0046358 | 0.0004636 | 0.0158269 | 0.0026378 |
| pusht_antigoal | adaln | 0.0002737 | 0.0000274 | 0.0004030 | 0.0000672 |
| pusht_antigoal | action_duplicate | 0.0003598 | 0.0000360 | 0.0006294 | 0.0001049 |
| pusht_antigoal | toy-drift | 0.0013174 | 0.0001317 | 0.0095641 | 0.0015940 |
We also report rollout errors. Because the four goal coordinates are fixed, we expected them to contribute little to the MSE. The unnormalized errors are indeed largely unchanged, except for Toy-DriftWorld, much like the planning results. Since the state dimensions differ, we also provide errors normalized by dimension. We will update the reported results accordingly. Thank you again for catching our mistake. We hope these additional results address your concern.
Finally, we appreciate that you recognized the broader question we are exploring, the compositional-generalization framing, and the value of the oracle-state, action-intervention, and representation-geometry analyses. Your comments also helped us identify several places where the paper needed more careful qualification, clearer reporting, and tighter scope, and they have significantly improved the revised version.
More broadly, our goal with AnyGoal is to provide a practical bridge between theoretical questions about compositional generalization and the empirical evaluation of current world models. We recognize that this middle ground inevitably leaves some factors difficult to isolate perfectly, and your review helped us clarify where our evidence supports a diagnostic interpretation rather than a causal one. At the same time, we believe that systematically exposing these failure modes under controlled goal shifts can provide a useful foundation for the world-model community to develop stronger evaluations, mechanisms, and methods in future work.
We hope that the additional results, corrections, and scope clarifications provided in our rebuttal have addressed the main soundness concerns underlying the current assessment. We very much appreciate the time and care you put into the review. If there are any remaining points that you would like us to clarify or discuss further, we would be happy to address them.
Response to rebuttal
Thanks for the detailed rebuttal and response to my questions. I appreciate the experiments to address my concerns. I am raising my score as the experiments address my empirical questions, however I believe the lack of theoretical novelty and limitations in the implementation of the proposed framework remain.
Response 1
We thank the reviewer for their detailed feedback and constructive suggestions. Here, we treasure the opportunity to address your concerns and improve the quality of our work.
W1, Q1. Base-task KDE control experiment.
We thank the reviewer for this important methodological consideration. We sampled 400 episodes from the base episodes using each of three random seeds (0/1/2) and used them as held-out test sets. The original number of episodes for each base task is as follows: PushT: 18,685; Cube: 10,000; Peg: 1,461; and TwoRoom: 10,000.
Table 1. Base-task KDE overlap (%), mean ± standard deviation over three 400-episode splits.
Encouragingly, the held-out base-task control shows that the joint-support estimator behaves as expected in most environments. For TwoRoom and PushT, held-out base trajectories remain almost entirely in-distribution even when evaluating the full state-action vector, with overlap above 95%. Cube also retains high overlap at the state level and 80% overlap for action + all state. This supports the use of the joint-KDE score as a meaningful diagnostic in these environments.
The main exception is Insert-Peg, as reviewer expected, where the held-out overlap drops sharply as dimensionality increases. This suggests that the full-dimensional KDE becomes unreliable in this low-sample, high-dimensional setting. We therefore do not interpret the Insert-Peg joint-overlap percentage quantitatively, and will clarify that this score is a diagnostic rather than a definitive support test.
However, importantly, it does not affect the two core design underlying AnyGoal: the tasks are still controlled goal shifts where the semantic factors remain individually within the training support. At worst, the Insert-Peg joint overlap may be higher than our estimator suggests, meaning that some test combinations are more in-distribution than estimated rather than involving factor extrapolation.
W1. Clarification of marginal support check.
Thank you for your cafeful evaluation of paper. We would like to respectfully clarify that Appendix C does not slice the full articulated state into independent 1D marginals. The support checks are applied to semantically defined factor groups, including 2D/3D positions and 4D quaternions. Only inherently scalar quantities such as individual joint angles or motor commands are checked separately. This distinction is intentional. Our marginal test is meant to verify that individual physical factors remain within training support while allowing their joint configuration to be novel. Requiring the full articulated configuration or joint coordination itself to remain in support would largely collapse the distinction between marginal factor support and the combinatorial gap we aim to evaluate. We will make it clearer in the revision.
W2. Proposition 6 and Section 6
Thank you for raising this point and providing the opportunity to clarify our contribution. We would like to clarify that our original intention of Section 6 is not to provide a complete solution to compositional generalization, but first identify two open question that needs to be addressed, and then formalize one favorable regime. We agree that Proposition 1 is intentionally a simple sufficient condition, and its proof is immediate once the structural and support assumptions hold.
Its purpose is not to solve compositional generalization or mechanism identification, but to formalize one distinction motivated by our experiments: a test point can be globally novel while remaining within the support of every local mechanism relevant to prediction. Importantly, we do not assume away mechanism identification as a solution to the overall problem. Section 6 explicitly separates mechanism identification from extrapolation as two distinct challenges, and states that identifying mechanisms from biased offline trajectories remains open.
We thank the reviewer for pointing out the connection to factored MDPs and causal MBRL. We will cite these works and clarify that Proposition 1 is a synthesis/illustration in our world-model setting rather than a claim of a novel theoretical principle.