- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper studies the cost used to compare predicted futures when planning with a frozen latent world model. It proposes success-aligned distance, which predicts the number of steps along a recorded trajectory until the task’s success condition first holds. The labels are computed from logged simulator states, while the cost networks receive pairs of encoded observations.
SAD-CEM combines the predictions of five cost networks, adds a penalty for their disagreement, and uses cross-entropy search to choose actions. For more distant goals, it checks the first predicted action block against paths sampled from a reused LeWM++ generator. If that check fails, an additional search tracks a selected waypoint instead. Experiments on five simulated tasks compare the complete planner and its simpler direct mode with other planners.
Critical strengths
1. The controlled comparisons support the usefulness of the success-aligned cost.
The evidence provided is very extensive. In Table 2, success-aligned labels increase the four-environment mean from 74.8% to 79.9% in direct mode and from 79.5% to 83.5% with the waypoint branch. The separate cost-and-schedule comparison also shows that the improvement over latent Euclidean distance is not explained only by more frequent replanning. Moreover, the direct mode already reaches 74.2% mean success across the five tasks without using generated waypoints. Together, these results establish that the learned cost contributes useful improvements independently of the full planner’s additional machinery.
2. The additional experiments help identify which components matter and where their benefits are limited.
The paper discusses many other facets, including ensemble averaging, the disagreement penalty, candidate sources, waypoint selection, search budget, and goal distance. These comparisons make the study more informative. In particular, the candidate-source experiment executes each proposed waypoint from the same initial state, helping separate the quality of the available waypoints from the ability to rank them.
The paper also reports results that qualify its design choices. Averaging the networks accounts for most of the ensemble improvement, while the disagreement penalty adds less. Similarly, the waypoint branch’s advantage becomes much smaller after each planner selects its search budget on validation data. Reporting such findings helps the reader identify the strongest part of the approach and understand the limits of the complete system.
Critical weaknesses
1. The baseline comparisons leave some uncertainty about the specific advantage of the proposed training target.
The success-aligned labels use the task’s success condition evaluated on logged simulator states, whereas most comparison methods do not receive this information during training, making the broad comparison combine the proposed target with access to task-specific supervision. The large overall improvement, therefore, does not, by itself, establish how much comes from the particular time-to-success target rather than from making the success condition available to a learned cost.
The binary reachability control partly addresses this question by using the same success condition. However, it predicts only whether success occurs within ten steps. At the label level, all examples requiring more than ten steps receive the same value, so the target cannot distinguish how much longer different journeys take. Additionally, the LeWM++ baseline excludes its action chunk prior. This omission is marked in Table 1, but the reason for excluding it is not explained.
2. The additional waypoint branch offers a smaller practical advantage once the direct planner’s search budget is selected separately.
The added value of the waypoint branch is less clear than the benefit of the learned cost. Both the full planner and direct mode use the proposed success-aligned cost. Although the waypoint branch improves the main five-environment average from 74.2% to 79.4%, the difference falls to one percentage point when each version selects its search iterations on validation data. The branch does make performance less sensitive to the search setting, but the results provide weaker evidence that it substantially improves success over a well-configured direct planner. This limits the demonstrated contribution of the additional branch.
Questions for authors
-
Why was the action chunk prior excluded from LeWM++? Are the results with the complete configuration already available?
-
What simulator-state variables and tolerances define the success condition used to construct the labels in each environment, particularly Cube and Scene? This would clarify the task information available during training.
-
For the validation-selected configurations in Table L.2, are corresponding planning-time measurements available for the full planner and direct mode?