acceptodds
Under review as a conference paper at ICLR 2027

SAGE: Subgoal-Conditioned Action Generation for Latent World Model Planning

Abstract

Latent world models have emerged as a powerful planning paradigm by learning action-conditioned predictive dynamics and using them as internal simulators to imagine and evaluate candidate action sequences. However, as the planning hori zon grows, performance becomes increasingly constrained by proposal quality: a fixed candidate budget must search an exponentially larger action space, making it difficult to expose the world model to high-quality candidate futures for evalua tion. In this paper, we introduce SAGE, a prior-conditioned planner that replaces random proposal initialization with structured guidance. At each planning stage, a goal-conditioned generator predicts the next intermediate latent subgoal for a specified duration, which is then used to condition the generation of candidate action sequences. To capture semantic information across temporal scales, we use subgoals of varying durations as priors, balancing fine-grained local control with higher-level long-horizon progress. Then the frozen world model evaluates these proposals against the same subgoal and guides their refinement before ex ecution. Experiments on PushT and OGBench Cube show that coupling latent subgoal decomposition with prior-conditioned action generation substantially im proves long-horizon planning while preserving strong short-horizon performance. To be specific, when the target offset is 150, it raises PushT success from 4.7% to 64.7% and OGBench Cube success from 20.7% to 67.3%. We further extend latent world-model planning to LIBERO, where SAGE improves full-episode success from 0% with the vanilla LeWM planner to 48.7% on Scene2 and 58% on Caddy.

Primary area:
unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords:
World models, long-horizon planning

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_tSheOpenReview ↗
Rating
2
Confidence
2
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

The paper addresses the failure of latent world model (LeWM/DINO-WM) planning on distant image goals, attributing it to two factors: under a fixed candidate budget, randomly initialized action proposals provide increasingly poor coverage of an exponentially growing action space; and a distant final goal offers only weak guidance for evaluating local action candidates. SAGE freezes the world model and learns two components: (1) a multi-duration subgoal generator that, given the current history, the final goal, the remaining goal horizon Δ\Delta, and a requested option duration τ\tau, predicts the latent subgoal τ\tau steps ahead; (2) a subgoal-conditioned action prior — a Transformer with a trajectory-level GMM (8 modes) — that proposes τ\tau-step action candidates conditioned on the predicted subgoal to initialize the online search (CEM refinement on PushT/Cube). Experiments are conducted on PushT, OGBench Cube, and LIBERO.

Critical strengths
  1. The problem decomposition is accurate and the core design is coherent. The paper separates the long-horizon planning bottleneck into two independent dimensions — proposal quality and evaluation target — and lets the same temporally matched subgoal constrain both.
Critical weaknesses
  1. Limited novelty, and the most relevant baselines are missing, making it difficult to determine whether the gains come from "world-model planning" or from "hierarchical imitation on expert data".
  • A subgoal generator plus a goal-conditioned action prior is essentially hierarchical goal-conditioned imitation (Play-LMP, HIRO-style subgoals, Guider, FF-JEPA — all discussed in the related work). Both components are trained with supervision on expert or oracle trajectories.
  • The paper's own data show that world-model search sometimes helps little, and can even hurt: on DINO-WM Cube, Generator+PriorTop reaches 141/150 at H=150H{=}150, above SAGE's 132/150 (Table 15).
  • The paper does not compare against goal-conditioned BC or hierarchical imitation policies trained on the same data (e.g., GCBC, HIQL, Diffusion Policy), nor against FF-JEPA, the closest concurrent work.
  • PRISM, the only learned baseline, uses fixed 25-step chunks whose execution cadence differs from SAGE's, so it is not a matched comparison.
  • Consequently, the central claim that subgoal-conditioned action generation substantially improves world-model planning lacks sufficient controlled support.
  1. The evaluation protocol may favor SAGE, and some baselines are weak, which discounts the strength of the conclusions.
  • The subgoal generator takes the true remaining goal horizon Δ\Delta as input, and goals are future frames of held-out trajectories from the same distribution — in effect, waypoint tracking along demonstration trajectories. Validity on out-of-distribution goals and multimodal paths is untested (the generator is a deterministic regressor, with the attendant risk of predicting an "averaged" latent corresponding to no valid state).
  • The LIBERO planning configuration is weak overall: all methods use only 64 candidates with a single ranking pass (vs. 300 candidates × 30 CEM rounds on PushT/Cube), and Base LeWM's proposal is a residual random walk with a fixed scale of 0.05, which limits the strength of the "0% → 48.7%/58.0%" comparison.
Questions for authors
  1. Can you add the following comparisons, all trained on the same data? (a) Goal-conditioned BC or hierarchical imitation policies (e.g., GCBC + subgoals, HIQL, Diffusion Policy); (b) FF-JEPA (subgoals used only as optimization targets). If such methods — which do not use the world model — achieve success rates close to SAGE, the paper's contribution would need to be repositioned.
  2. What would Base LeWM achieve on LIBERO with the same 300-candidate × 30-round CEM configuration as on PushT/Cube (or under equal compute)? And what would PRISM achieve under the same execution schedule as SAGE?
  3. Can you provide a failure attribution: do failures stem mainly from unreachable or inaccurate subgoals, or from world-model prediction/scoring errors?
  4. Does SAGE remain effective when trained on non-expert or random play data, or when the goal lies outside the training distribution (i.e., not on any recorded trajectory)?
  5. Could a duration selector be learned to adaptively choose τ\tau from the current state, removing the dependence on preset per-HH schedules?
  6. How were the per-HH execution schedules in the main experiments determined? Were the evaluation manifests, or data overlapping with them, used in this process?
  7. In Table 1, PriorOnly on Cube drops sharply from 75.3% at H=100H{=}100 to 1.3% at H=150H{=}150. What explains this?
11 comments
AuthorsOpenReview ↗

Response to Reviewer tShe (1/6)

We thank the reviewer for the detailed and constructive review. We will first address each question in the following, and then discuss each weakness.


Q1(a): Can you add goal-conditioned BC or hierarchical imitation baselines?

Thank you for this suggestion. We have added goal-conditioned behavior cloning (GCBC) and hierarchical behavior cloning baselines, all trained on the same data as SAGE. These policies are executed without world-model search, allowing us to evaluate how much can be achieved through direct imitation alone. We first consider two GCBC variants: (1) GCBC (frozen encoder) uses LeWM's pretrained visual encoder with frozen parameters to extract visual features, followed by a goal-conditioned action policy; (2) GCBC (end-to-end visual) takes visual observations as input and directly predicts goal-conditioned action distributions. We further include Hierarchical BC, which models actions through a subgoal generator and a subgoal-conditioned GCBC, evaluating both sampling from its learned action mixture and deterministic selection of the most likely mode.

Table R1: Success rates (%) over 150 queries for behavior cloning baselines and SAGE. PushT and Cube are evaluated at H=150, while Scene2 and Caddy report full-episode LIBERO success rates.

MethodPushTCubeScene2Caddy
GCBC (frozen LeWM encoder, sampled)8.028.721.352.7
GCBC (end-to-end visual, sampled)1.322.06.024.0
Hierarchical BC (sampled)14.08.729.346.7
Hierarchical BC (top mode)16.01.335.338.7
SAGE64.767.348.758.0

As shown in the table, SAGE achieves the highest success rate across all four tasks, outperforming the best BC baseline by 48.7, 38.7, 13.3, and 5.3 percentage points on PushT, Cube, Scene2, and Caddy. These results indicate that direct imitation of training trajectories alone is insufficient to achieve SAGE's performance, highlighting the importance of combining subgoal-conditioned planning with a latent world model. The results have been added in Appendix A.24 of the revised version.


Q1(b): How does SAGE compare with the concurrent work FF-JEPA?

Thank you for your suggestion. We reimplement FF-JEPA to enable a direct comparison. On PushT and Cube, both methods use fixed 25-step action chunks at H=150; on Scene2 and Caddy, we train FF-JEPA's diffusion variant on the same 8,000 demonstrations and use the same full-episode queries, 15-step commitment, and recovery budget as SAGE (Appendix A.32).

Table R2: Success rates (%) of FF-JEPA and SAGE under matched execution settings. PushT and Cube use H=150 with fixed 25-step planning and execution; Scene2 and Caddy report full-episode LIBERO success.

MethodPushTCubeScene2Caddy
FF-JEPA (our reimplementation)13.36.70.00.0
SAGE55.368.048.758.0

As shown in the table, SAGE significantly outperforms FF-JEPA on these benchmarks. Besides, we also consider FF-JEPA's closest goal-conditioned analogue in our setup, i.e., Subgoal Scoring, which achieves 51.3% success on Cube at H=150. These results further highlight the effectiveness of our method.


AuthorsOpenReview ↗

Response to Reviewer tShe (2/6)

Q2: What would Base LeWM achieve on LIBERO with the same 300-candidate × 30-round CEM configuration as on PushT/Cube (or under equal compute)? And what would PRISM achieve under the same execution schedule as SAGE?

Thank you for these suggestions. We conduct two additional experiments to address these questions.

First, we evaluate Base LeWM on LIBERO with both SAGE's original search budget (64 candidates and one ranking pass) and the CEM budget as on PushT(300 candidates and 30 rounds).

Table R3: Success rates (%) of Base LeWM and SAGE on full-episode LIBERO tasks under different search budgets.

MethodSearch budgetScene2Caddy
Base LeWM64 candidates, 1 ranking pass0.00.0
Base LeWM300 candidates, 30 CEM rounds0.00.0
SAGE64 candidates, 1 ranking pass48.758.0

Under both configurations, Base achieves 0.0% success on Scene2 and Caddy, compared with 48.7% and 58.0% for SAGE on the same queries. The results have been added to Appendix A.7 of the revised version.

Second, we re-evaluate PRISM's released action prior using CEM under SAGE's exact queries, search budget, and execution schedule, ensuring a controlled comparison.

Table R4: Success rates (%) of SAGE and PRISM-CEM under matched search budgets and execution schedules.

HPushT (SAGE / PRISM)Cube (SAGE / PRISM)
5081.3 / 52.076.0 / 62.0
10072.7 / 11.385.3 / 71.3
15064.7 / 14.067.3 / 43.3

As shown in the table, SAGE consistently outperforms PRISM across all evaluated horizons, with particularly large improvements at longer horizons. The results for all six horizons have been added to Appendix A.25. These results further suggest that SAGE's improvements cannot be achieved simply by increasing the search budget or changing the execution schedule.


Q3: Can you provide a failure attribution: do failures stem mainly from unreachable or inaccurate subgoals, or from world-model prediction/scoring errors?

Thank you for this suggestion. To disentangle the contributions of subgoal errors and world-model prediction errors, we conduct a controlled intervention study on the first 15-step decision. Specifically, we investigate two factors: (1) Subgoal accuracy: replacing SAGE's predicted first subgoal with the corresponding ground-truth subgoal from the expert trajectory; and (2) Prediction accuracy: keeping the subgoal, candidate actions, and latent distance unchanged, while replacing the world model's predicted endpoint of each candidate with the endpoint actually reached by executing that candidate in the simulator, encoded by the same frozen encoder. This corresponds to scoring candidates with a perfect predictor. By combining these two interventions, we examine the individual and joint effects of the two error sources.

We collect a run on LeWM Cube at H=150, achieving 104/150 successes, and verify that replay reproduces all 150 outcomes, including every CEM round of the first decision. For each of the 46 failures, we restart from the saved state and rerun the full first decision (300 candidates, 30 CEM rounds), modifying only the subgoal and/or the candidate endpoints used in every CEM round. We then continue execution using SAGE.

Table R5: Number of failures rescued by correcting the subgoal and/or the candidate endpoints at the first decision on LeWM Cube at H=150.

SubgoalCandidate endpoints in CEMRescued failures
Generated subgoalWorld-model prediction0/46
Expert subgoalWorld-model prediction22/46
Generated subgoalSimulator endpoint23/46
Expert subgoalSimulator endpoint19/46

The results show that correcting either the subgoal or the world-model predictions rescues a similar number of failures (22/46 and 23/46, respectively), suggesting that both error sources contribute comparably. The two single corrections rescue 15 failures in common, indicating that the two error sources often co-occur. Overall, these results indicate that failures arise from both inaccurate subgoal predictions and world-model prediction errors, rather than being dominated by either factor alone. The results have been added to Appendix A.34 of the revised version.

AuthorsOpenReview ↗

Response to Reviewer tShe (3/6)

Q4: Does SAGE remain effective when trained on non-expert or random play data, or when the goal lies outside the training distribution (i.e., not on any recorded trajectory)?

Thank you for this question. We conduct two additional experiments to evaluate SAGE's effectiveness beyond expert demonstrations and its generalization to out-of-distribution (OOD) goals.

First, we evaluate SAGE on two datasets collected without expert policies. Specifically, the training process on PointMaze-Large uses goal-agnostic random exploration, where uniformly sampled actions are executed without access to goals or the map. TwoRoom uses the released LeWM dataset collected by a noisy heuristic controller.

Table R6: Success rates (%) of Base LeWM and SAGE using datasets collected without expert policies.

BenchmarkData collectionBaseSAGE
PointMaze-Large (H=150)Goal-agnostic random exploration0.056.0
TwoRoom (H=75)Noisy heuristic controller30.090.0

SAGE achieves 56.0% success on PointMaze-Large and 90.0% on TwoRoom, substantially outperforming Base LeWM on both benchmarks. The results have been added to Appendices A.29 and A.30.

Second, to evaluate whether SAGE can reach goals that are not represented in the training trajectories, we conduct an additional experiment on PushT with OOD goals. Specifically, the joint pusher–block configurations of these goals lie 1.25–2.0 normalized units away from the training data support. We compare SAGE against Base LeWM, GCBC, and Hierarchical BC.

Table R7: Success rates (%) on original and moderate OOD PushT goals.

MethodOriginal goalsModerate OOD goals
Base4.75.3
GCBC6.70.7
Hierarchical BC14.04.0
SAGE62.040.0

SAGE succeeds on 93/150 original goals (62.0%) and 60/150 moderate OOD goals (40.0%) in two runs on the same 150 recorded queries. SAGE's success rate drops by 22.0 percentage points and remains above the evaluated baselines. Appendix A.28 reports the evaluation protocol and results.

These results indicate that SAGE can remain effective when trained on non-expert data and generalize beyond the recorded trajectories.


Q5: Could a duration selector be learned to adaptively choose τ\tau from the current state, removing the dependence on preset per-schedules?

Thank you for this suggestion. To investigate whether action durations can be selected adaptively rather than relying on preset schedules, we implement a lightweight learned duration selector. It consists of two linear classifiers, one for 20-step and one for 25-step execution, which take the current latent state, the goal difference, and the remaining planning horizon as inputs. Each classifier predicts whether switching from 15 steps to that duration at the current decision improves, preserves, or worsens final task success; the selector chooses the option with the largest positive expected gain and defaults to 15 steps otherwise. Training labels come from paired interventions on 64 PushT H=150 pilot episodes disjoint from the test set, and the trained selector is frozen for all horizons.

Table R8: Success rates (%) of fixed 15-step execution and the learned duration selector on PushT. The fixed baseline differs from the mixed-duration schedule used in the main results.

HorizonFixed 15-stepLearned selectorGain
H=10073.375.3+2.0 %
H=15064.066.0+2.0 %

As shown in the table, the learned selector slightly improves success rates by 2.0 percentage points across different horizons. These results demonstrate the feasibility of learning adaptive action durations from the current state, reducing reliance on manually preset duration schedules. Further details have been added to Appendix A.27.

Reviewer_SvExOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies why latent world model planning degrades as the goal moves farther into the future, attributing the failure to the proposal mechanism rather than the dynamics model, since a fixed candidate budget drawn from a generic random distribution covers an exponentially larger action space increasingly poorly. It proposes SAGE, which keeps the encoder and latent dynamics frozen and adds two learned components trained on recorded trajectories: a variable-duration subgoal generator that predicts an intermediate latent target a requested number of steps ahead, and a subgoal-conditioned action prior parameterized as a trajectory Gaussian mixture that proposes candidate action sequences toward that target. The same generated subgoal serves both as the conditioning signal for proposals and as the target the frozen world model scores against during refinement. It is tested on PushT and OGBench Cube across a range of goal offsets, where success improves over the base planner and the advantage widens as the target moves farther away, and on a custom image-goal LIBERO setup where the base planner fails entirely on full episodes while SAGE completes a substantial fraction. A three-way component analysis separates the proposal distribution, the scoring target, and the use of world-model search, with supporting experiments on a second backbone, multiple training seeds, a visual-only variant, and an estimated rather than reference goal offset.

Critical strengths
  • The component analysis is designed well enough to answer the question a skeptical reader would ask, which is whether the gain comes from the learned proposal or from somewhere else. By varying proposal, scoring target, and world-model search independently on a fixed backbone and budget, the paper shows that the subgoal must serve as both the conditioning signal and the evaluation target, since conditioning proposals on a subgoal while still scoring against the final goal drops PushT from 64.7 to 6.7 percent. It also shows the world model contributes beyond the prior, since removing search costs 20 to 29 successes on LIBERO full episodes.

  • The supporting experiments go beyond what is typical for this kind of paper. The method is demonstrated on a second, independently trained backbone, across three component training seeds, with a visual-only variant that removes low-dimensional state conditioning, and with an estimated goal offset replacing the reference offset so the planner does not depend on knowing how far away the goal is. Decision latency is reported rather than omitted. Together these address the most obvious alternative explanations for the gains.

Critical weaknesses
  • The central comparisons are made against an untuned random proposal, so the most prominent numbers are not informative. Base LeWM samples from a zero-mean Gaussian whose zero does not correspond to a zero physical action, and from a residual random walk on LIBERO, which is why it reaches 0 percent on full episodes. Reporting 0 to 48.7 percent as the headline result establishes that random action search fails at multi-stage manipulation rather than that subgoal conditioning is the right fix.

  • SAGE's action prior is trained on scripted oracle demonstrations while no policy-learning method is run on the same data, so the obvious alternative explanation for the gains, namely that imitation on expert trajectories supplies most of the benefit, is never tested. Prior Only and Generator plus Prior Top are the closest controls and do leave room for the world model, but each executes a single mixture mode without sampling or search, making them weak stand-ins for a goal-conditioned behavior cloning or offline reinforcement learning baseline on the same 8,000 episodes.

Questions for authors

The action prior is trained with likelihood on scripted oracle demonstrations, yet no policy-learning method is evaluated on the same data. What full-episode LIBERO success does a goal-conditioned behavior cloning policy, sampled and executed without any world model, reach on the same 8,000 training episodes?

4 comments
AuthorsOpenReview ↗

Response to Reviewer SvEx

We thank the reviewer for the positive assessment and careful review. We understand that your main concern is whether SAGE's improvements come from subgoal-conditioned planning rather than weak baselines or imitation of training trajectories. Below, we will address your main concerns point by point.


W1: The central comparisons are made against an untuned random proposal, so the most prominent numbers are not informative.

Thank you for raising this point. To provide a more comprehensive evaluation, we have added several stronger baselines beyond the original random proposals. In the following table, we compare SAGE with (1) two Gaussian-proposal baselines (Base LeWM), one centered on the training-set mean action, which is LeWM's original proposal on PushT and Cube, and one centered on zero physical action, (2) PRISM-CEM, which uses PRISM’s released action prior on PushT and Cube, and a PRISM-style Gaussian head trained on the same data on LIBERO, and (3) a representative goal-conditioned behavior cloning (GCBC) policy. We also provide more results of different behavior cloning policies in our response to Weakness 2. For all search-based comparisons, we keep the frozen world model, query set, search budget, and execution schedule consistent across methods.

Table R1. Success rates (%) over 150 queries. On PushT and Cube, search-based methods use 300 candidates and 30 CEM rounds; on LIBERO, they use 64 candidates and one ranking pass.

MethodSearchPushT H50PushT H100PushT H150Cube H50Cube H100Cube H150Scene2 (full)Caddy (full)
Base LeWM (Gaussian, training-mean center)✓56.08.74.756.058.720.70.00.0
Base LeWM (Gaussian, zero-action center)✓41.37.36.754.058.022.70.00.0
PRISM-CEM✓52.011.314.062.071.343.31.34.7
GCBC (frozen LeWM encoder, sampled)✗20.08.08.061.382.728.721.352.7
SAGE✓81.372.764.776.085.367.348.758.0

The results show that SAGE consistently outperforms LeWM baselines, PRISM-CEM and GCBC across all reported horizons. In particular, the advantage becomes more significant at longer horizons, where subgoal-conditioned planning is increasingly important. This suggests that SAGE's gains are not simply due to the failure of random proposals, but reflect the benefit of subgoal-conditioned planning.


W2 / Q1: Does imitation on expert trajectories supply most of the benefit? What full-episode LIBERO success does a goal-conditioned behavior cloning policy, sampled and executed without any world model, reach on the same 8,000 training episodes?

To evaluate how much of SAGE's performance can be attributed to imitation of training trajectories, we compare it against several behavior cloning (BC) policies. In particular, we train goal-conditioned behavior cloning (GCBC) policies on the same 8,000 LIBERO training episodes and execute them by sampling actions, without using a world model or search. We first consider two GCBC variants: (1) GCBC (frozen encoder) uses LeWM's pretrained visual encoder with frozen parameters to extract visual features, followed by a goal-conditioned action policy; (2) GCBC (end-to-end visual) takes visual observations as input and directly predicts goal-conditioned action distributions. We further include Hierarchical BC, which models actions through a subgoal generator and a subgoal-conditioned GCBC, evaluating both sampling from its learned action mixture and deterministic selection of the most likely mode. All policies are evaluated on the same queries with matched eval budgets.

Table R2: Success rates (%) over 150 queries for behavior cloning baselines and SAGE. PushT and Cube are evaluated at H=150, while Scene2 and Caddy report full-episode LIBERO success rates.

PolicyPushTCubeScene2Caddy
GCBC (frozen encoder, sampled)8.028.721.352.7
GCBC (end-to-end visual, sampled)1.322.06.024.0
Hierarchical BC (sampled)14.08.729.346.7
Hierarchical BC (top mode)16.01.335.338.7
SAGE64.767.348.758.0

On full-episode LIBERO evaluation, SAGE achieves 48.7% success on Scene2 and 58.0% on Caddy, outperforming all behavior cloning baselines. These results suggest that imitation of training trajectories cannot fully explain SAGE's performance, highlighting the benefit of its subgoal-conditioned planning mechanism.


We hope these results could address the reviewer's concerns. We are looking forward to your reply and please let us know if there is more to clarify.

Thanks for the rebuttal and the additional experiments, which have addressed my concerns. I will maintain my positive rating.

AuthorsOpenReview ↗

Additional Results Related to Your Original Questions

Dear Reviewer SvEx,

Thank you again for your positive assessment. During the discussion, we obtained new results that relate directly to your two original questions, and we would like to share them for completeness.

Does imitation of expert trajectories supply most of the benefit? Trained on non-expert data, SAGE still clearly outperforms Base. In all three settings, the world model, subgoal generator, and action prior are trained only on this data (successes out of 150):

BenchmarkTraining dataBaseSAGE
PointMaze-Large, H=150 (Appendix A.30)Goal-agnostic random exploration084
Cube, H=150 (Appendix A.36)OGBench task-agnostic play data1244
TwoRoom, H=75 (Appendix A.29)Released LeWM data from a noisy heuristic controller45135

SAGE's advantage over imitation also grows under distribution shift: on PushT goals far from the training data, Hierarchical BC drops to at most 4.7%, while SAGE retains 30.7–42.0% (Appendix A.28).

Is the Base comparison informative? To verify that Base's 0% on LIBERO full episodes reflects task difficulty rather than an implementation issue, we evaluated Base on tail queries that start near the end of each episode (success %, Appendix A.32):

HorizonScene2Caddy
Tail, H=251.320.0
Tail, H=502.715.3
Full episode0.00.0

Base succeeds when the goal is near but fails as the horizon grows to full multi-stage episodes, which is the regime SAGE addresses.

If you find that these results further strengthen the paper, we would be grateful if you would take them into account in your final assessment.

Authors

Thanks for sharing the additional results. I appreciate the authors' effort on the new experiments, which further supports the proposed method. I will maintain my positive rating.

Reviewer_MYefOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

SAGE uses predicted subgoals to guide planning with a frozen world model. It first trains a subgoal generator, then trains an action generator conditioned on the predicted subgoals. During planning, the same subgoal guides both action proposals and the evaluation of their predicted outcomes. The method supports different action durations and is evaluated on PushT, OGBench Cube, and a customized image-goal LIBERO setup.

Critical strengths
  1. Using the same subgoal for action generation and scoring makes sense. A useful preparatory action may not immediately approach the final goal, so scoring it against that goal can discard a good proposal. The sharp performance drop when SAGE switches back to final-goal scoring supports this design choice.

  2. The method improves long-horizon planning without changing the underlying world model. Training the action generator on predicted subgoals also matches the input it receives during execution. The gains on PushT and Cube, together with the LIBERO results, show that the approach works beyond a single task.

Critical weaknesses
  1. The comparison between Final-goal Prior and SAGE changes both the proposal conditioning and the scoring target. It therefore does not fully separate the benefit of feeding subgoals to the action generator from that of using them for scoring. On PushT at H=150, subgoal scoring alone reaches 58.7%, versus 64.7% for SAGE; the corresponding gap is larger on Cube, at 51.3% versus 67.3%. Testing a final-goal-conditioned prior with subgoal scoring would clarify this contribution.

  2. Both action generation and scoring depend on the predicted subgoal. If that target is inaccurate, world-model search may find actions that approach it without helping the final task. The results show that the approach is useful, but the phrase “reachable local targets” is stronger than the training objective establishes. The paper should qualify this wording and discuss how replanning handles subgoal errors.

  3. Figure 1 also leaves out important connections. It does not show the subgoal being used to score world-model predictions or where candidate selection and CEM refinement occur. These steps are central to the method and should be visible in the diagram.method’s effectiveness, the characterization of its outputs as “reachable local targets” needs more direct evidence or more qualified wording.

Questions for authors
  1. Could you evaluate the existing final-goal-conditioned action prior using subgoal-based scoring, particularly at long horizons, while keeping the execution schedule and search budget fixed?

  2. What happens when the predicted subgoal is inaccurate? A representative example showing whether later replanning corrects the error would be helpful.

  3. In the DINO-WM Cube experiment at H=150, direct prior execution outperforms SAGE. What explains this result, and when does world-model selection or refinement make the chosen action worse?

4 comments
AuthorsOpenReview ↗

Response to Reviewer MYef (1/2)

We thank the reviewer for the careful review and for the insightful suggestions. Below, we will address your main concerns point by point.


W1: The comparison between Final-goal Prior and SAGE changes both the proposal conditioning and the scoring target. It therefore does not fully separate the benefit of feeding subgoals to the action generator from that of using them for scoring.

Thanks for pointing this out. We have run the suggested variant on PushT under the same execution schedule and search budget.

Table R1. Final-goal-conditioned vs. subgoal-conditioned proposal on PushT (H=150), with identical subgoal scoring, execution schedules, and search budgets.

Method (H=150)ProposalScoring targetSuccess Rate
Final-goal prior + subgoal scoringFinal-goal priorSubgoal57.3
SAGESubgoal priorSubgoal64.7

As shown in the table, SAGE improves the success rate from 57.3% to 64.7% compared with final-goal-conditioned proposals, suggesting that conditioning proposals on the predicted subgoal can provide additional benefits under the same subgoal-scoring setup.


W2: The phrase "reachable local targets" is stronger than the training objective establishes. The paper should qualify this wording and discuss how replanning handles subgoal errors.

Thank you for pointing this out. We have replaced "reachable local targets" with "local targets predicted from demonstrations." We have also clarified in Section 3 of the revised version that these targets are supervised using recorded future observations and are therefore not necessarily physically reachable.

Regarding how replanning handles subgoal prediction errors, SAGE adopts an iterative replanning strategy that allows subsequent subgoals to be adjusted based on the actual execution outcome. Specifically, after executing a τ\tau-step option toward the current predicted subgoal, SAGE observes the resulting state and predicts the next subgoal conditioned on this updated observation. This allows SAGE to replan based on its actual state, rather than continuing to follow a potentially inaccurate subgoal prediction.

Our experimental results further demonstrate this ability to handle subgoal prediction errors. Specifically, we measure first-subgoal deviation by the distance between the predicted first subgoal and the corresponding expert trajectory. We then rank all queries, including both successful and failed ones, based on this distance. Among the 132 successful queries on DINO-WM Cube at H=150, 32 exhibit large deviations at the first subgoal and fall within the top 25% in terms of distance from the expert trajectory, yet still successfully complete the task (Appendix A.26). The results indicate that replanning can help SAGE recover from initial subgoal deviations.

Besides, Figure 7 in Appendix A.31 shows a representative example of recovery, illustrating how subsequent replanning accounts for an early deviation and enables successful task completion.


W3: Figure 1 does not show the subgoal being used to score world-model predictions or where candidate selection and CEM refinement occur. The characterization of its outputs as "reachable local targets" needs more direct evidence or more qualified wording.

Thanks for pointing this out. We have revised Figure 1 to explicitly show (i) the subgoal being used to score world-model predictions and (ii) where candidate selection and CEM refinement occur. Besides, we have replaced "a reachable local target" with "predicts a local target".


Q1: Could you evaluate the existing final-goal-conditioned action prior using subgoal-based scoring, particularly at long horizons, while keeping the execution schedule and search budget fixed?

Thanks for your suggestion. We have run the suggested variant on PushT under the same execution schedule and search budget.

Table R2. Final-goal-conditioned vs. subgoal-conditioned proposal on PushT (H=150), with identical subgoal scoring, execution schedules, and search budgets.

Method (H=150)ProposalScoring targetSuccess Rate
Final-goal prior + subgoal scoringFinal-goal priorSubgoal57.3
SAGESubgoal priorSubgoal64.7

As shown in the table, SAGE improves the success rate from 57.3% to 64.7% compared with final-goal-conditioned proposals, suggesting that conditioning proposals on the predicted subgoal can provide additional benefits under the same subgoal-scoring setup.

AuthorsOpenReview ↗

Response to Reviewer MYef (2/2)

Q2: What happens when the predicted subgoal is inaccurate? A representative example showing whether later replanning corrects the error would be helpful.

Thanks for your suggestion. We note that SAGE adopts an iterative replanning strategy that allows subsequent subgoals to be adjusted based on the actual execution outcome. Specifically, after executing a τ\tau-step option toward the current predicted subgoal, SAGE observes the resulting state and predicts the next subgoal conditioned on this updated observation. This allows SAGE to replan based on its actual state, rather than continuing to follow a potentially inaccurate subgoal prediction.

Our experimental results further demonstrate this ability to handle subgoal prediction errors. Specifically, we measure first-subgoal deviation by the distance between the predicted first subgoal and the corresponding expert trajectory. We then rank all queries, including both successful and failed ones, based on this distance. Among the 132 successful queries on DINO-WM Cube at H=150, 32 exhibit large deviations at the first subgoal and fall within the top 25% in terms of distance from the expert trajectory, yet still successfully complete the task (Appendix A.26). The results indicate that replanning can help SAGE recover from initial subgoal deviations. Besides, Figure 7 in Appendix A.31 shows a representative example of recovery, illustrating how subsequent replanning accounts for an early deviation and enables successful task completion. However, when the sampled subgoal deviates too far, subsequent replanning may not be sufficient to recover. Figure 8 in Appendix A.31 illustrates such a failure case.


Q3: In the DINO-WM Cube experiment at H=150, direct prior execution outperforms SAGE. What explains this result, and when does world-model selection or refinement make the chosen action worse?

To better understand why direct prior execution outperforms SAGE in this case, we take a closer look at the queries where SAGE fails despite the prior succeeding. Although SAGE rescues 6 queries where the prior fails, it also causes 15 previously successful queries to fail, reducing the overall success rate from 141/150 to 132/150.

To investigate why world-model selection can hurt performance on these 15 queries, we compare SAGE and Prior Top under identical initial conditions on the 15 failed DINO-WM Cube queries at H=150. Specifically, for each query, we execute only the first 15 actions of each method from the same initial state and compare their resulting endpoints.

Table R3. First-segment endpoint comparison on the 15 failed DINO-WM Cube queries (H=150). Physical distances within 1 mm count as ties.

Endpoint metricSAGEPrior Top
Closer to generated subgoal (latent)123
Closer to expert frame at step 15 (latent)510
Closer to expert gripper position at step 15213

As shown in the table, world-model selection produces endpoints closer to the generated subgoal in 12 of 15 queries, but closer to the expert frame in latent space in only 5 of 15 queries. In physical space, world-model selection leaves the gripper farther from the expert gripper position in 13 of 15 queries. These results show that although world-model selection successfully drives the system closer to its generated subgoal, it also moves the system further away from the expert trajectory. Consequently, when the generated subgoal is highly inaccurate, world-model selection can further increase the deviation from the expert trajectory by steering the system toward the wrong target, making the resulting error too large for subsequent replanning to recover.


We hope these additions could address the reviewer's concerns. We are looking forward to your reply and please let us know if there is more to clarify.

AuthorsOpenReview ↗

Your invaluable input is needed

Dear Reviewer MYef,

We have carefully prepared a detailed response to address each of your questions. Would you please take a look and let us know whether you find it satisfactory? Your invaluable input is greatly appreciated. Thank you once again, and we hope you have a wonderful day!

Authors

Thanks to the authors for addressing my concern. I will maintain my score.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Rebuttal Summary

Dear Program Chairs, Senior Area Chairs, Area Chairs, and Reviewers,

Below, we summarize the main concerns, the additional experiments and clarifications we provided, and the outcome of the discussion. All revisions are highlighted in blue in the updated PDF.

Main concerns. The reviewers' questions centered on (1) whether SAGE's gains hold against stronger, matched baselines rather than an untuned random proposal; (2) whether the gains come from imitation of demonstrations rather than world-model planning; and (3) how the method behaves under subgoal errors, non-expert data, and out-of-distribution goals. In response, we conducted a series of additional experiments and clarifications following their suggestions, and added the results to the revised paper.

Additional experiments.

  • Same-data imitation baselines (frozen-encoder and end-to-end GCBC, sampled and top-mode Hierarchical BC) on all four benchmarks, in Table 3 and Appendix A.24 (Reviewers SvEx, tShe).
  • PRISM under SAGE's exact queries, budget, and execution schedule at six horizons, in Figure 2 and Appendix A.25 (Reviewers SvEx, tShe).
  • Calibrated and zero-action-centered Base variants, including a 300-candidate, 30-round CEM budget on LIBERO, in Appendices A.7 and A.19, and LIBERO tail-query checks in Appendix A.32 (Reviewers SvEx, tShe).
  • Final-goal prior with subgoal scoring, separating proposal conditioning from the scoring target, in Section 4.3 (Reviewer MYef).
  • Analysis of when world-model search hurts on DINO-WM Cube, in Appendix A.26, and recovery and failure examples after early subgoal deviations, in Appendix A.31 (Reviewer MYef).
  • An FF-JEPA reimplementation on all four benchmarks, in Appendix A.32 (Reviewer tShe).
  • A controlled first-decision failure attribution on LeWM Cube, in Appendix A.34 (Reviewer tShe).
  • Training on non-expert data: PointMaze-Large with goal-agnostic random exploration (Appendix A.30), TwoRoom (Appendix A.29), and Cube with OGBench play data (Appendix A.36) (Reviewers tShe, SvEx).
  • OOD goals at four levels of distribution shift, in Appendix A.28 (Reviewer tShe).
  • Estimated goal offsets, a learned duration selector, Prior Only at intermediate horizons, and a stochastic flow-matching subgoal generator, in Appendices A.20, A.27, A.33, and A.35 (Reviewer tShe).

Additional clarifications.

  • Confirmed that the 0% success of Base and FF-JEPA on full LIBERO episodes reflects long-horizon difficulty rather than an implementation issue, as both solve some short tail queries, in Appendix A.32 (Reviewer tShe).
  • Clarified how the per-horizon execution schedules were fixed before test evaluation without using evaluation data (Reviewer tShe).
  • Explained why world-model search can hurt when the prior is near ceiling on DINO-WM Cube, in Appendix A.26 (Reviewers MYef, tShe).
  • Explained the choice of PointMaze-Large for random exploration, in Appendix A.30, and how SAGE tolerates imperfect subgoals through replanning, in Appendices A.26 and A.31 (Reviewer tShe).

Writing improvements.

  • Replaced "reachable local targets" with "local targets predicted from demonstrations" and clarified in Section 3 that these targets are not guaranteed to be physically reachable (Reviewers MYef, tShe).
  • Revised Figure 1 to show how the subgoal generator, action prior, and frozen world model interact during planning (Reviewers MYef, tShe).

Overall, in this paper, we propose SAGE, a subgoal-guided planning framework for latent world models. SAGE combines variable-duration subgoal generation, subgoal-conditioned action proposals, and world-model-based optimization, using the subgoal to guide both action generation and scoring. Extensive experiments on PushT, OGBench Cube, and LIBERO demonstrate significant improvements in long-horizon planning, supported by comprehensive ablations and evaluations across different world-model backbones and settings. We believe that our additional experiments and clarifications during the rebuttal have effectively addressed the concerns raised by all reviewers.

Thank you once again for your dedication to the community.

Best regards,

Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.