- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 2
Read review
Paper summary
This paper introduces Hierarchical Residual Prediction (HRP), a latent world model in which the upper level predicts not the future, but the prediction error (residual) of the lower level's k-step rollout. The residual is sparse without any sparsity penalty — it vanishes wherever the lower level is already correct — and, being a vector, it records how the lower level failed rather than merely that it failed. Building on this, the paper proposes two readings of the residual: its magnitude locates the states where the lower level fails, while its dependence on the action (clusterability) separates genuine decision points, where the agent's action settles the outcome, from uncontrolled branching driven by dynamics the agent cannot control. The planner escalates to coarse search only where the residual is both large and clusterable. Experiments span navigation, manipulation, and open-ended survival tasks of varying difficulty; restricting coarse search to the detected decision points improves long-horizon success while reaching the best success of prior hierarchies at 2.4× fewer model rollout steps.
Critical strengths
- The overall concept of Hierarchical Residual Prediction for latent world models is novel.
- The key contributions are each supported by fairly detailed experimental validation.
Critical weaknesses
- The paper is difficult to read, and several key concepts are not explained clearly or rigorously — "decision points" being a prominent example.
- The entire motivation rests on the premise that a level 1 trained with squared-error regression necessarily outputs the mean of possible successors (the "averaged state"). If level 1 were instead a stochastic or distributional dynamics model, it could sample from the branches directly, potentially rendering the residual signal unnecessary. The paper only compares against deterministic hierarchical methods and lacks this class of closely related baselines, which leaves the claim that "the residual is a necessary signal" incompletely supported.
- The method's premise is that prediction error is concentrated at a small set of states and that the agent's action resolves them. The authors themselves note: "On Push-T, contact makes the dynamics branch continuously, mean clusterability is 0.04 against 0.29 at GraphMaze junctions, and HWM stays ahead at every horizon." In other words, the method is currently effective only for environments with discrete branching — precisely the class of problems where hierarchical planning already works reasonably well — while offering no benefit for the genuinely difficult case of continuous contact-rich manipulation.
Questions for authors
- How does the set of detected decision points change with the training level / capacity of level 1, and with the environment?
- Does the failure on Push-T stem from an architectural limitation of the regression-based level 1, or from a limitation of the estimator? Could distributional or generative predictions (e.g., quantiles, flow matching) for Δ or for level 1 extend residual-based detection to continuously branching environments?
- The Crafter evaluation provides goals in a topological order of the achievement dependency graph. What is HRP's score without this guidance? How much of the 9.2% comes from the hierarchical model itself, and how much from the goal ordering?
- Regarding the claim of "separating real decision points from random events that the agent cannot control": a comparative analysis against methods such as ICM (intrinsic curiosity model), which are known to be attracted to uncontrollable stochasticity, would make the design choices of HRP considerably clearer.