- Rating
- 1
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
This paper investigates whether the hierarchical latent world model is benefitted from 1) macro action space or 2) coarse level planner.
Critical strengths
- The paper provides a deep analysis between two published models: PLDM and HWM to show how exactly hierarchical model performs better than flat planning model.
Critical weaknesses
- I am not sure how I should feel about this paper. Abstract is not abstract, introduction uses unintroduced terms, and figures are barely figurable. I try not to weigh too much on the paper’s presentation when reviewing, but this paper is on un-ignorable level.
- The paper has only 13 references and misses rich literature context behind the latent world model. I do not even see JEPA-based world models.
- This work verify its hypothesis using two pretrained checkpoints under maze setups. The scope is too narrow to draw meaningful and generalizable lesson over these scenarios.
- No clear explanations provided for how two models (PLDM, HWM) differ in their mechanisms; at least provide how the inputs are given, how these model process the inputs, and how their planning differs, in a thorough way.
- The section names are not organized and inconsistent.
Questions for authors
- Could you restate the question the whole paper is dealing with in a plain language? Planning on the macro-action space & using coarse-level planning model do sound very similar for me, and I could not clearly understand what this paper tries to discover. If a model is applied on a coarse timestep in the micro-action representation space, it actually will have to return the most similar micro-action that the model has to head to, which essentially is a sort of macro-action (especially in this maze setup, where the action is configured only by 2-dimentional directions).
- Could you expand the experiments over the maze dataset? The maze environment makes sense in that micro actions often lead to pitfalls, but essentially this setup could be the only beneficial setup where having coarse-level decision timesteps often leads model closer to the goal. Again, the generalizability problem emerges.
- Similarly, could you generalize the idea over PLDM and HWM?
- What’s the lesson the research community can take away from this paper? How will this be used to improve, motivate, and inspire other hierarchical world model attempts in the future? Should they stop creating a macro-action representation space (specifically for maze scenario)? Is this really a lesson that is not constrained to solving maze in a virtual environment?
2 comments
Post-rebuttal comments
Thanks for the honest answers. Although questions are answered, most weaknesses remain. I maintain my rating as it is.
The question in plain words, how the two planners differ, and what the results support
Thank you for the review. You are right about the presentation, the references and the scope. The main limits (one environment, a missing comparison described below, the organisation of the paper) need more than the discussion period allows, so we have not uploaded a revised PDF.
The question in plain words (your first question). A hierarchical planner differs from a flat one in two ways at once. It rolls each candidate plan forward with a second predictor that jumps ten environment steps per call (the "coarse" model of Table 2), where the flat planner chains 250 one-step predictions (the "fine" model). And it searches over learned macro-actions: an 8-number code that a learned action encoder computes from a ten-step chunk of actions. A common reading credits the second. The paper asks which of the two the gain comes from.
You are right that the two overlap: any ten-step predictor must be told what happens during its ten steps, and that input is a macro-action of some kind. So the test cannot remove macro-actions; it can only make the search over them crude. The crudest search we ran is over one held 2-D action per ten steps (two numbers), which the released action encoder then turns into the code the ten-step predictor takes. On the same 20 hard trials, a planner that searches only these two numbers solves 5 when its plans are rolled through the one-step predictor and 19 through the ten-step predictor; 14 trials are solved only with the ten-step predictor and none only with the one-step one (Table 2). Searching the full learned 8-number space solves 17, which 20 trials cannot tell apart from 19. So the planner does not need to search its learned macro-action space.
Three qualifications:
How the two planners differ. Both encode the image observation with the same encoder into one latent space; the encoder and the one-step predictor are bit-for-bit identical in the two released checkpoints (Section 2). The flat planner (PLDM) samples 250-step action sequences, rolls each through the one-step predictor, scores it by the squared distance between the last predicted latent and the encoded goal, moves its plan toward the low-cost samples, executes the first actions and replans. The hierarchy (HWM) has no second observation encoder (Section 2); it adds an action encoder, which turns a ten-step chunk into the 8-number macro-action, and a predictor that advances the same latent ten steps per macro-action. Its top level samples sequences of 47 macro-actions in the release (470 steps; 25 in Table 2) and rolls them through that predictor; the predicted latents become subgoals that the one-step planner tracks.
Presentation and references. Agreed. The abstract is a list of results with their test statistics, and the abstract and introduction use terms before they are defined ("level-1 weights", "harness", "discordant", "coverage", κ). The section titles mix questions, claims and labels and follow the order of the experiments, so the decisive comparison arrives only in Section 7. Figure 1's label line runs outside its box and an axis label of Figure 4 is cut off. The paper has thirteen references; it never says that the two models it studies are themselves JEPA world models, and it cites little of the latent world-model literature.
Scope (your second and third questions). Agreed. Every result is on one released model pair and one maze family (Section 9); the release ships the hierarchy only for this maze model. We have no result outside mazes, so we cannot say whether they are a favourable case. Within them, coarse decisions alone show no sign of helping: the held actions solve 5 of 20 through the one-step predictor, against 9 to 12 for the unrestricted planner.
The lesson (your last question). On this model pair, searching the learned macro-action space is not what makes the hierarchy work: a search over two numbers per macro-action could not be told apart from the full search. The practical use is a control: run the crude search before crediting a learned action space. The results do not say that builders should stop learning a macro-action space, since the action encoder is still used here, nor that any of this holds outside this pair.