- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper studies planning when multiple observations correspond to the same successful outcome. The authors keep the LeWorldModel encoder and dynamics frozen and train a projector to map successful observations toward a common center. Distance to this center is then used as the planning cost. The training includes goal compactness and a regularizer on ordinary observations. The full nonlinear model also uses temporal and action preferences from successful trajectories. The authors evaluate linear and nonlinear projectors on Push-T, Two-Room, and OGBench-Cube, comparing them with costs based on a single goal image or a set of goal images under the same planner. The nonlinear model improves success on all three environments, with the largest gains on Push-T and Two-Room.
Critical strengths
-
The paper points out a problem with matching a single goal image. The planner may try to match details such as the agent position even when they do not determine success. Learning a separate cost allows the world model to keep these details for predicting dynamics.
-
The MLP reaches 90.8% success on Push-T and 100.0% on Two-Room, compared with 75.7% and 73.3% for latent 1-NN, across three world-model seeds. The image-only linear projector reaches 81.0% on Push-T. This is a useful result since it does not use the trajectory supervision of the full model.
Critical weaknesses
- The full model uses successful trajectories and action preferences, while the main goal-set baselines only use goal images. Removing the two trajectory losses drops Push-T success from 90.8% to 47.5%. To me, this comparison does not show what is gained by mapping successful states to a common center, beyond learning a cost from the additional supervision. Learning visual objectives from successful examples has already been studied in [1], which the paper discusses. I would like to see a scalar cost network trained with the same data and comparable capacity. The current ablations also keep goal compactness throughout, so they do not answer this question.
- Each environment uses only one task definition. This does not really test the motivation of using the same predictive model for different tasks, since only one goal head is trained per environment. The gains are also mostly on Push-T and Two-Room. On OGBench-Cube, the MLP reaches 52.3%, compared with 47.7% for the best reported k-NN variant. I would find experiments with several different goals on the same frozen world model more informative.
- The trajectory losses assume that later states are better and demonstrated action sequences are preferable to sampled or perturbed alternatives. A different action sequence could also succeed, particularly when several outcomes are valid goals. The authors acknowledge this, but do not check how often these preferences agree with simulator outcomes. The preference loss matters to the result, since removing it lowers Push-T success from 90.8% to 80.8%.
- Figure 10 changes the initial agent position and measures planning success. This changes the control problem as well, so it does not directly show that the cost ignores irrelevant changes in agent position. Figure 9 also aggregates costs over states. I would like to see the costs for different agent positions at the same block pose, including held-out successful poses and nearby failures.
[1] Few-Shot Goal Inference for Visuomotor Learning and Planning, CoRL 2018.
Questions for authors
- How does the method compare with a scalar cost network using the same supervision and comparable capacity? What happens if goal compactness is removed while the trajectory losses are kept?
- Can different goal heads be trained on the same frozen world model for several task definitions?
- How often do the temporal and action preferences agree with simulator outcomes?
- How much does the cost vary with agent position when the block pose is fixed? Does it still distinguish successful poses from nearby failures?