- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
This paper introduces LAGO, a latent world model for language-conditioned model-based planning. The same predictor is trained in two modes: predicting the next latent state given an action, and predicting a future latent subgoal given the current state, a natural-language instruction, and a continuous completion variable. At inference time, several subgoals corresponding to different values of the continous variable are generated independently from the current observation. CEM then optimizes action sequences using a soft-minimum trajectory-to-subgoal alignment cost, with subgoals regenerated after every MPC. The method is evaluated on TwoRoom, a newly proposed Kingdom navigation domain, OGBench Cube and OGBench Scene. The paper compares against flat and hierarchical latent world models, oracle-subgoal variants, language reward models, and a goal-conditioned behavior-cloning baseline. It additionally presents LAGOSET/LAGO++, showing that language-conditioned subgoal prediction can benefit from action-free video-language data.
Critical strengths
The strongest aspect of the paper is not merely “language goals instead of image goals.” It is that dynamics prediction and task grounding become the same mathematical operation: predict a target latent from the present latent, with the conditioning variable determining whether the prediction represents a one-step transition or a longer-horizon semantic target. Both are optimized using the same latent-regression objective. This is considerably cleaner than attaching a separate reward model or generative visual-goal model.
Critical weaknesses
The paper claims a notable architectural virtue: one predictor, shared parameters, one latent objective. However, I don't think the experiments isolate this contribution. LAGO(O.S.) versus DINO-WM(O.S.) shows that exposing the model to the subgoal-prediction objective improves downstream planning dynamics. That is an interesting result. But it does not tell us whether the benefit comes specifically from sharing one predictor, versus simply learning from additional future-state supervision.
Questions for authors
What happens if dynamics prediction and subgoal prediction use separate predictors with identical supervision and matched capacity? This seems necessary to establish the value of the shared-predictor contribution.
Official Comment by Authors
Dear Reviewer,
We would like to thank you for the time and consideration you have given to reviewing our work. As the rebuttal period draws to a close, we would welcome any remaining feedback or questions you may have. Of course, should any issues remain unresolved, we would be happy to provide further explanations or additional experimental results.