acceptodds
Withdrawn from ICLR 2027

Plan Toward Success, Not Similarity: Success-Aligned Costs on Frozen Latent World Models

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

A frozen latent world model serves control through test-time search, where an optimizer seeks the plan of lowest cost. We argue that the cost is the main lever, and that latent and temporal distances measure the wrong quantity, embedding similarity or elapsed steps. We propose success-aligned distances, heads that regress the logged time until the task predicate first holds, a label that is zero exactly on successful states. SAD-CEM optimizes a pessimistic ensemble of these heads toward the encoded goal with cross-entropy search, and consults generated waypoints when its plan leaves the generated paths. With frozen LeWM backbones, SAD-CEM has the highest or tied-highest mean success on each of five goal-reaching environments, and a five-environment mean of 0.794 against 0.630 for the strongest of fifteen external planners. The success-aligned cost alone, without generated waypoints, already reaches 0.742, while a step-gap label or one head in place of the ensemble lowers success significantly. Code and demos are available at anonymous.4open.science/r/sad-cem.

Primary area:
applications to robotics, autonomy, planning
Keywords:
world models, latent planning, model predictive control, learned cost functions, goal-conditioned control, cross-entropy method, ensembles

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_CzFSOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies the cost used to compare predicted futures when planning with a frozen latent world model. It proposes success-aligned distance, which predicts the number of steps along a recorded trajectory until the task’s success condition first holds. The labels are computed from logged simulator states, while the cost networks receive pairs of encoded observations.

SAD-CEM combines the predictions of five cost networks, adds a penalty for their disagreement, and uses cross-entropy search to choose actions. For more distant goals, it checks the first predicted action block against paths sampled from a reused LeWM++ generator. If that check fails, an additional search tracks a selected waypoint instead. Experiments on five simulated tasks compare the complete planner and its simpler direct mode with other planners.

Critical strengths

1. The controlled comparisons support the usefulness of the success-aligned cost.

The evidence provided is very extensive. In Table 2, success-aligned labels increase the four-environment mean from 74.8% to 79.9% in direct mode and from 79.5% to 83.5% with the waypoint branch. The separate cost-and-schedule comparison also shows that the improvement over latent Euclidean distance is not explained only by more frequent replanning. Moreover, the direct mode already reaches 74.2% mean success across the five tasks without using generated waypoints. Together, these results establish that the learned cost contributes useful improvements independently of the full planner’s additional machinery.

2. The additional experiments help identify which components matter and where their benefits are limited.

The paper discusses many other facets, including ensemble averaging, the disagreement penalty, candidate sources, waypoint selection, search budget, and goal distance. These comparisons make the study more informative. In particular, the candidate-source experiment executes each proposed waypoint from the same initial state, helping separate the quality of the available waypoints from the ability to rank them.

The paper also reports results that qualify its design choices. Averaging the networks accounts for most of the ensemble improvement, while the disagreement penalty adds less. Similarly, the waypoint branch’s advantage becomes much smaller after each planner selects its search budget on validation data. Reporting such findings helps the reader identify the strongest part of the approach and understand the limits of the complete system.

Critical weaknesses

1. The baseline comparisons leave some uncertainty about the specific advantage of the proposed training target.

The success-aligned labels use the task’s success condition evaluated on logged simulator states, whereas most comparison methods do not receive this information during training, making the broad comparison combine the proposed target with access to task-specific supervision. The large overall improvement, therefore, does not, by itself, establish how much comes from the particular time-to-success target rather than from making the success condition available to a learned cost.

The binary reachability control partly addresses this question by using the same success condition. However, it predicts only whether success occurs within ten steps. At the label level, all examples requiring more than ten steps receive the same value, so the target cannot distinguish how much longer different journeys take. Additionally, the LeWM++ baseline excludes its action chunk prior. This omission is marked in Table 1, but the reason for excluding it is not explained.

2. The additional waypoint branch offers a smaller practical advantage once the direct planner’s search budget is selected separately.

The added value of the waypoint branch is less clear than the benefit of the learned cost. Both the full planner and direct mode use the proposed success-aligned cost. Although the waypoint branch improves the main five-environment average from 74.2% to 79.4%, the difference falls to one percentage point when each version selects its search iterations on validation data. The branch does make performance less sensitive to the search setting, but the results provide weaker evidence that it substantially improves success over a well-configured direct planner. This limits the demonstrated contribution of the additional branch.

Questions for authors
  1. Why was the action chunk prior excluded from LeWM++? Are the results with the complete configuration already available?

  2. What simulator-state variables and tolerances define the success condition used to construct the labels in each environment, particularly Cube and Scene? This would clarify the task information available during training.

  3. For the validation-selected configurations in Table L.2, are corresponding planning-time measurements available for the full planner and direct mode?

Reviewer_1QXYOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies goal-reaching by test-time search over a frozen latent world model (LeWM). It argues that the cost the search minimizes is the main lever, and that latent Euclidean and temporal distances measure the wrong quantity. It proposes success-aligned distances (SAD): three-layer networks, called heads, trained on pairs of encoded states. They regress the logged time until the environment's success predicate χ\chi first holds, yij=1Tmin⁡{τ∈{0,…,j−i}:χ(si+τ,sj)=1}y_{ij} = \frac{1}{T} \min\{\tau \in \{0, \dots, j-i\} : \chi(s_{i+\tau}, s_j) = 1\}, which is zero exactly when the first state already satisfies the goal. SAD-CEM minimizes a pessimistic five-head ensemble (mean plus one standard deviation) toward the encoded goal with the cross-entropy method (CEM), re-planning after every 5-step block. For goals estimated to be more than ten steps away, it also samples eight latent paths from a flow-matching generator (from LeWM++). If the planned first block leaves these paths, the ensemble picks a path end as a waypoint, which the planner tracks with the latent Euclidean cost. Formal results bound the optimism of picking the minimum of NN noisy cost estimates, and show that ensemble spread cannot detect errors shared by all heads. The experiments compare SAD-CEM with 15 planners in five simulated environments, with the same checkpoints, goals and 50-step budget. SAD-CEM's five-environment mean success is 0.794, vs. 0.630 for the runner-up (LeWM++ without its action chunk prior), and 0.742 without the waypoint branch. Ablations swap the label (step gap, fixed-horizon reachability) and the ensemble (one head), and other experiments vary the search budget, goal distance and backbone.

Critical strengths
  1. The empirical comparison is thorough. In each environment, all 16 planners share the checkpoint, goals, start states, success predicate and 50-step budget. Where the paper selects settings, it uses a validation split, and it reports the main results on a disjoint final split (Section 4.1 and Appendix D). Paired tests in the text are goal-clustered, and the paper links anonymous code. This makes the comparison useful on its own, and the reported gains more credible.

  2. The method performs well and consistently across environments. In Table 1, every baseline scores 0.412 or less in some environment, while SAD-CEM's lowest score is 0.628 (Scene).

Critical weaknesses
  1. Most of the gain over the baselines on Table 2's four environments appears to come from choices other than success alignment, the margin over the runner-up comes mostly from Reacher, and the waypoint branch adds little at tuned budgets.

Table 2 replaces the success-aligned label with a step gap, keeping everything else fixed. The four-environment mean drops by 0.05 in direct mode (0.799 to 0.748) and by 0.04 for SAD-CEM (0.835 to 0.795). These drops are significant, though the step gap is slightly better on TwoRoom. But the step-gap variant does not use the task predicate, and its 0.748 already exceeds every external planner's four-environment mean (at most 0.665, for LeWM++). Yet the title and abstract emphasize success alignment. Without Reacher, though, the step-gap variant trails LeWM++ (three-environment means 0.684 vs. 0.749). There, success-aligned heads raise direct mode to 0.739, still below LeWM++, and SAD-CEM from 0.741 to 0.784, above it.

In Table 1, TRM, ACID and RC-aux re-plan after every block only on Scene (Table N.1(b)), are never run with an ensemble, and keep "the hyperparameters that their authors selected" (Appendix D). TRM uses the same head architecture and a step-gap label, so its remaining differences from the step-gap control include the ensemble, label scaling, pair sampling and schedule. TRM also searches with fewer samples than SAD-CEM on Reacher and PushT (Table N.1(b)), but with the CEM row's search size it does no better (Appendix D). For the success-aligned cost, the schedule matters mainly on PushT, as the paper notes: in Table 3 (validation split, one head, mean over three seeds), re-planning after every block instead of every five raises PushT from 0.367 to 0.527.

In Table 1, the margin over the runner-up, LeWM++ without its action chunk prior, comes mostly from Reacher, where LeWM++ scores 0.412, below plain CEM (0.788). Without Reacher, the mean margin falls from 0.16 to 0.06. Against the best baseline in each environment, SAD-CEM's margin is at most 0.012, except on Reacher (0.156). On Cube and PushT, the goal-clustered bootstrap intervals for SAD-CEM minus LeWM++ include zero (Appendix K).

The waypoint branch takes up most of the formal results and much of the appendix. With each variant's search budget chosen on validation, it adds only 0.010 over five environments (Table L.2; p = 0.25, Appendix L), as the conclusion acknowledges. Its larger gain at the default 30 CEM iterations mostly reflects direct mode degrading (Table L.1, four environments), as Appendix L notes (lines 1064–1065).

  1. The label uses privileged supervision, and the paper does not discuss pre-2026 work on learned goal-reaching distances.

The label needs the environment's success predicate on logged simulator states (Equation 2), the same predicate that scores the evaluation. The planner rows and the other learned costs train without it (Section 4.1), and the conclusion calls the case without it an open question (lines 483–484). The only other label that uses it is binary reachability at h=10h = 10, which cannot distinguish a state 15 steps from success from one 50 steps away. Its poor result (Table 2: a four-environment mean of 0.629 in direct mode, 0.180 on PushT) does not show that time until success is the best use of the predicate. The paper claims only that time until success "outperforms this fixed-horizon binary label" (lines 358–359), and notes that other horizons remain untested (lines 484–485).

Prior work learns temporal or dynamical distances for goal reaching, e.g., Search on the Replay Buffer (Eysenbach et al., 2019), Dynamical Distance Learning (Hartikainen et al., 2020), self-supervised functional distances (Tian et al., 2021), LEXA (Mendonca et al., 2021) and quasimetric RL (Wang et al., 2023). The paper cites none of these, and its related work on learned costs cites only 2026 work on latent world models. Its novelty claim (lines 090–092) is limited to planners on frozen latent world models. This literature also raises a design question: regressing logged times estimates how long the data-collection policy took, not the shortest time to success, which Q-learning-style and quasimetric methods target.

  1. The presentation is dense, and the formal results are elementary.

Many terms are used before they are defined, are defined only in the appendix, or have no definition I could find, e.g., "task predicate", "heads", "generated paths", "waypoint branch", "support test", "raised goals", "stall entry", and most of the ablation labels in Figure 4. Results switch between four- and five-environment means (e.g., 0.799 and 0.742 for direct mode), which the paper usually labels but which are hard to compare. Proposition 1 applies the standard maximal inequality for sub-Gaussian variables, as the paper notes, and Lemma 1.1, Proposition 2, Lemma 2.1 and Lemma 2.2 have proofs of four to seven lines. As the paper also notes, Proposition 1 does not bound CEM itself (lines 253–254), yet three paragraph titles in Section 4.2 cite it for experiments that measure success, not the quantities it bounds. Since the paper says Proposition 1 and Lemma 1.1 "motivate" the cost (line 213), I suggest presenting them more briefly and using the space for the method and the attribution experiments.

I would raise my score if the authors show, under matched schedules, that the label explains a substantial part of the gain over the learned-cost baselines, and compare against the full LeWM++.

Minor points:

Figure 1 sits above the abstract, which is highly unusual and departs from the ICLR template. The abstract should say that the "task predicate" is the environment's success predicate. Line 156 says the planners "execute the first block of the final mean", but in Table N.1 the CEM row executes all five planned blocks on four environments. The Table 3 caption mentions the CEM row's schedule (lines 336–337), but please correct line 156 and state each row's schedule there. Table 1's markers come from t-tests over five seeds on a fixed set of 50 goals, so they ignore variation across goals, and the caption mentions no correction for the 90 comparisons. Please use the goal-clustered tests used elsewhere in the paper. Table 3 lacks the cell for the latent Euclidean cost under receding 5, but the CEM row selected receding 5 on validation for Reacher, TwoRoom, Cube and PushT. Figure 1(b) compares the costs only under receding 1, where the latent Euclidean cost gets 0.333 on Reacher, vs. 0.788 for the CEM row in Table 1 (different splits and seeds). So Figure 1(b) may overstate the effect of the cost alone. The conclusion says the cost "keeps its lead on DINO-WM and for goals 100 steps ahead" (lines 477–478). But the DINO-WM experiment covers only PushT and omits LeWM++, which scores higher than direct mode on LeWM PushT (means 0.684 vs. 0.644, Tables 1 and 2). At 100 steps, Table M.2 tests the full SAD-CEM rather than the cost alone, and its four-environment lead over LeWM++ comes from Reacher. The appendices use "pre-registered" 13 times; please point to a timestamped record, or say "fixed before any run" (line 1040) instead.

Questions for authors
  1. With step-gap heads, SAD-CEM (direct mode) reaches 0.748 on Table 2's four environments, vs. 0.595 for TRM, which uses the same head architecture and a step-gap label (Table 1). How much of this gap comes from the ensemble, the re-planning schedule, label scaling and pair sampling? Please report TRM (ideally also ACID and RC-aux) re-planning after every block, and SAD-CEM with step-gap heads executing all five planned blocks.

  2. Why was the action chunk prior removed from LeWM++, and how does the full method perform? Why does LeWM++ without it score 0.412 on Reacher, below plain CEM (0.788)?

  3. Can you compare against other uses of the same success predicate, such as reachability at several horizons, budget-conditioned reachability as in RC-aux, or a goal-conditioned value learned by TD with the success indicator as reward? This would show whether the gain comes from the time-until-success label or from access to the predicate. Also, which state variables does χ\chi check in each environment?

My suggestion for future work (not part of my assessment):

  1. Learning the distance with Q-learning-style or quasimetric objectives would target the shortest time to success, which may matter when the logged trajectories are far from optimal.
Reviewer_1ymVOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

The authors propose learning the number of steps until success, as opposed to the number of steps between two states, to guide search of a frozen world model. In addition, they combine this with a pessimistic ensemble and waypoint search. This is compared to other planning methods on tasks such as Reacher and Cube which results in superior coverage (percentage of instances solved).

Critical strengths

The idea to train based on whether or not a goal state has been reached is straight-forward and draws from a long history of success on constructing heuristic functions in heuristic search. It is good to see this perspective be adopted in modern machine learning methods.

Critical weaknesses

The paper assumes access to the underlying simulator to compute \Chi, which is the goal test function. However, the trained world model is not particular to these simulated domains and, to the best of my knowledge, the methods to which the authors compare do not assume the same access to the simulator exists. It would be different if, instead, a success aligned metric were learned in observation or latent space. However, this is not the case. Therefore, the methods presented in the paper cannot readily be adopted by these pre-trained world models as they are domain-specific.

The paper presented several additions to the cost-aligned training, such as ensembles and waypoint search. The ablations in this paper were minimal and confusing. For example, in Figure 4, the bars have names that are not defined anywhere, such as “point distance” and “paths of other states”. It is not immediately obvious what this means. Also, it appears that Figure 4 is just comparing SAD-CEM to one removed component. What about with only one component and none of the others. Namely, the success aligned cost without the ensemble and waypoint search. How does this compare to some other cost?

Questions for authors

How can this be generalized if it requires access to the underlying simulator?

How does the success aligned cost directly compare to other costs without the other components of this approach?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.