acceptodds
Under review as a conference paper at ICLR 2027

Metro-WM: Long-Horizon Latent Planning with Realisable Sub-Goals

Abstract

Model-predictive control with Joint-Embedding Predictive Architectures (JEPAs) provides a strong zero-shot goal-reaching planner, but it is only effective over short planning horizons. Hierarchical extensions attempt to bridge this gap by learning a macro planner to predict intermediate latent sub-goals to guide the micro planner. In this work, we demonstrate that unconstrained latent sub-goal prediction is fundamentally flawed. A rigorous evaluation reveals that a leading state-of-the-art macro planner routinely emits physically unrealisable sub-goals. To resolve this, we introduce Metro-WM, a hierarchical framework that issues sub-goals by retrieving genuine states from prior experience rather than generating ungrounded latent vectors. Specifically, Metro-WM constructs a graph whose vertices are observed frames from offline expert demonstrations or random-action trajectories, allowing frames from different episodes to be connected and stitched into routes to the goal. Planning over the full graph also makes the system highly robust to execution errors: if the micro planner drifts off course, Metro-WM instantly finds a new optimal path from the current state. Our experiments show that Metro-WM achieves superior long-horizon success rates of up to percentage points over the next best hierarchical approach while being up to faster, requiring both less offline compute and fewer tuned hyperparameters. Additional analysis reveals that Metro-WM finds shorter paths than the offline demonstrations, outperforms an oracle relying on the query's own demonstration, and maintains robust performance under extremely sparse dataset conditions.

Primary area:
applications to robotics, autonomy, planning
Keywords:
world models, hierarchical planning, model-predictive control, joint-embedding predictive architectures

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_6CTPOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper argues that hierarchical JEPA planners can produce latent sub-goals that do not correspond to any physically realisable state, and introduces a decoder-free realisability residual to measure this. It proposes METRO-WM, a training-free macro planner that builds a graph over encoded frames from recorded episodes, connects consecutive frames within an episode and visually similar frames across episodes, and selects sub-goals by shortest-path search so that every sub-goal is a recorded state. Experiments on PushT, OGBench Cube, TwoRoom, and Reacher with LeWM and PLDM as micro planners report higher long-horizon success and lower online latency than flat and hierarchical baselines.

Critical strengths
  • At the longest goal distance, METRO-WM improves clearly over both flat planning and the hierarchical baselines evaluated, on all four environments and with both micro planners, while running faster online. The long-horizon gain is therefore real and not specific to one environment or one world model.

  • The macro planner requires no training and only two hyperparameters, so it is cheap to apply on top of any frozen latent world model.

Critical weaknesses
  • The paper does not discuss the assumptions the method depends on or the conditions under which it stops working. METRO-WM can only issue sub-goals that exist in the recorded data and can only stitch episodes where latent-similar frames are physically interchangeable. For example, the method may not work when the goal lies in a region of the state space that no demonstration has visited, since the graph then contains no usable sub-goal and the method falls back to flat planning, whose behaviour is not evaluated. It is also exposed to perceptual aliasing, where physically different states have similar latents and a bridge edge connects them with a cost the micro planner cannot meet; this is a known problem in retrieval-based planning that Emmons et al. (2020), cited by the paper, explicitly address, and the paper does not discuss it. The discussion of limitations in the paper falls far short of what the method requires. A more detailed discussion of the assumptions and applicability of the approach, together with experiments that show where it breaks, is required.

  • The ablations do not test the factor the paper attributes its gains to. The central claim is that retrieved sub-goals succeed because they are physically realisable states, but METRO-WM differs from the hierarchical baselines in several ways at once, such as planning over a precomputed shortest-path cost for every recorded frame, replacing the learned macro predictor and its CEM search with a non-parametric lookup, and stitching routes across demonstrations, and no experiment varies realisability alone. The paper therefore shows that the retrieval planner works, but not that realisability is the reason it works.

Questions for authors
  • Can the authors provide the additional analysis requested above: an explicit statement of the assumptions under which METRO-WM applies, and experiments that show how it behaves when these assumptions are violated?

  • The related work in the main text is limited to two paragraphs, and the extended discussion in Appendix F covers hierarchical latent planning only within the JEPA family. Latent sub-goal generation in learned world models has a longer history outside JEPA, including work that addresses the same realisability concern. Can the authors provide a more thorough discussion of how METRO-WM relates to this broader line of work?

4 comments
AuthorsOpenReview ↗

Response to Reviewer 6CTP

R3-a. Assumptions, limitations, where it breaks. Thank you for the suggestion. We added a new appendix section (App. I), detailing both assumptions and limitations. In summary, Metro assumes:

  • (A1) The corpus covers the goal region and a route to it. Our new fallback ablation (App. E.4) removes from the graph every episode that passes within a growing radius of the goal, up to the whole graph, so Metro must fall back to flat planning near the goal. Table 9 shows that Metro degrades gracefully where the micro planner can cover the uncovered stretch (Reacher 80–98%, Cube 76–90%, vs flat 62.00 and 55.33), and breaks where it cannot (PushT and TwoRoom at the largest radius drop to the level of flat planning or below). Table 5 also shows that Metro stays above flat planning with 1% of the corpus.
  • (A2) Most bridges are accurate: frames close in the world model's latent space are a few frames apart. Metro does not detect perceptual aliasing, a property of the world model's representation that the micro planner's own objective shares. Re-entry recovers from occasional wrong bridges (Table 3), but Metro breaks when many are wrong. Learning a bridge distance, as SGM (Emmons et al., 2020) does, could filter them at the cost of training; we leave this to future work.
  • (A3) The micro planner is reliable within H frames. Metro's performance ceiling is set by the micro planner: a stronger world model raises its success (App. E.2).
  • (A4) Latent distance tracks elapsed frames at short gaps. Bridge prices are clipped at H because the latent metric saturates beyond it (App. D).
  • (A5) Scenes recur across episodes. Metro handles FourRoom in the new H-JEPA setup (see general response; Sec. 6.3, App. H), where a different door is closed in each episode (98.67% success), but cannot stitch where layouts rarely repeat.

R3-b. Is realisability the reason it works? We do not claim that realisability alone explains the gain, and intuitively it cannot. A realisable sub-goal that leads away from the goal, or lies beyond the micro planner's reach, does not help. A good sub-goal must be both realisable and useful, "both within π_lo's reach and on a route to meet the goal" (Sec. 5). Retrieval provides the first; the shortest-path search over recorded experience provides the second.

To vary realisability alone, as requested, we kept Hi-LeWM-C's macro and micro planners unchanged and replaced each of its sub-goals with the nearest realisable state, found by the same r(z) search as in Sec. 4. Success (%), d = 75, at Hi-LeWM-C's budget T = 150

CubePushT
Hi-LeWM-C73.33 ± 4.1634.00 ± 3.46
Hi-LeWM-C (nearest realisable sub-goals)75.33 ± 2.3140.67 ± 13.01

Making Hi-LeWM-C's sub-goals realisable gives only a small gain that is not significant, far below Metro-LeWM’s performance: realizability alone is not sufficient.

R3-c. Related work beyond JEPA. Thank you for the suggestion. To clarify, App. F already covers two lines of work outside JEPA:

  • sub-goal generation in pixel space: TAP, HVF, GCP, LEAP, SuSIE, UniPi, VLP;
  • retrieval and graph planning in RL and navigation: SPTM, ViNG, SoRB, SGM, L3P, HIGL, DHRL, Plan2vec, HTM, TTGS.

We agree that latent sub-goal generation outside of JEPA deserved its own discussion. We added a paragraph in App. F, and extended the pointer to it in the main text (Sec. 2), on methods that generate sub-goals, in a learned latent space or in state space, and constrain them towards realisability (the sub-goal encodes a real state), reachability (the lower level can attain it), or both. They do so through:

  • a learned prior over the latent space: LEAP, Director;
  • dynamics constraints on planned latent states: LatCo;
  • adjacency constraints: HRAC;
  • value-based high levels: LEAP, RIS;
  • verification by the low-level policy: HAC, kSubS.

These mechanisms make generated sub-goals more likely to be realisable or reachable, and most rely on additional trained components, but none guarantees realisability.

AuthorsOpenReview ↗

Follow-up on our response

Dear Reviewer, thank you again for your review. As the discussion period closes on Nov 18, we wanted to check whether our response has addressed your concerns. We would be happy to clarify anything further or provide additional details. If our responses have resolved your concerns, we would be grateful if you would consider updating your assessment.

Reviewer_6CTPOpenReview ↗

Follow-up Questions After Rebuttal

Thank you for the detailed response and the new appendix on assumptions. The fallback ablation, the experiment in R3-b, and the expanded related work address several of the points I raised. However, the result of R3-b changes how I read the contribution, and the new assumptions raise further questions about the design. My remaining concerns are about what the paper contributes and whether the method is internally consistent, rather than about missing experiments. The questions below ask for explanation rather than new experiments, and whether I adjust my rating will depend on the answers. I am keeping my score at 2 for now.

  1. R3-b shows that making Hi-LeWM-C's sub-goals realisable gives only a small, non-significant gain, and you conclude that realisability alone is not sufficient and attribute the rest to shortest-path search over recorded experience. Shortest-path retrieval over recorded states with cross-episode connections and re-planning is the structure of SoRB, SPTM, and SGM. Setting realisability aside, what is the technical contribution of Metro-WM relative to this line of work? The title, abstract, and the end of Section 4 still present the method as a fix for unrealisable sub-goals; please clarify how the framing will be revised to match the finding.

  2. A2 acknowledges that bridges can connect physically different states with similar latents, and offers three mitigations: the ε cap, frame-based pricing, and re-entry. All three are functions of latent distance. Aliased pairs have small latent distance by definition, so they pass the ε cap, receive a low price, and re-entry attaches an aliased current state to the wrong node. How do these mitigations address errors caused by aliasing, as opposed to errors where the bridge is correct but the micro planner fell short?

  3. A4 assumes that latent distance grows with the number of frames at short gaps, yet Eq. 5 requires a running maximum to make the empirical quantile Λ(d) monotone, which means the assumption does not hold on the paper's own data even at small d. How should this be reconciled, and how is Eq. 5 defined where Λ(d+1) = Λ(d)?

  4. When the graph is disconnected, Section 5.3 routes to the reachable vertex closest to the goal in latent space. Appendix D shows that latent distance saturates at long range, so this choice is uninformative in exactly the case it is meant to handle. Could the authors explain this design, and whether it accounts for the no-bridge variant in R1-e falling below flat planning on PushT, TwoRoom, and Reacher?

AuthorsOpenReview ↗

Response to Follow-up Questions

We are grateful for your detailed follow-up and constructive feedback. We answer each question below:

  1. SoRB, SPTM and SGM were developed for RL and navigation. Metro brings graph retrieval to planning with frozen JEPA world models and differs from them in four ways:

    • Metro is training-free. The works you mention learn edge distances with a value function (SoRB) or a reachability network (SPTM), on which SGM builds, and execute with a trained policy. Metro instead prices edges in demonstrator frames from the corpus's own latent statistics and executes with the world model's zero-shot MPC. It thereby inherits the world model's limitations, such as aliasing (A2). Training a distance as in SPTM or SGM on top of Metro could filter aliased bridges and further improve performance, at the cost of training; we leave this to future work.
    • Unlike SoRB and SGM, Metro needs no rewards or online interaction: SoRB learns its distances and policy with goal-conditioned RL, and SGM removes faulty edges by executing them at test time. Metro's graph is built from offline, reward-free expert or random-action data.
    • Metro's sub-goals are recorded latent states that replace the goal in the world model's planning objective (Eq. 1), rather than goals for a goal-conditioned policy.
    • We evaluate on manipulation (PushT, Cube, Reacher) as well as navigation tasks.

    To our knowledge, Metro is the first to show that, for JEPA world models, a training-free retrieval macro planner matches or outperforms trained hierarchies (Hi-LeWM-C, H-JEPA, HWM).

    On framing, we agree that realisability is necessary but not sufficient. The abstract and the end of Section 4 now say that Metro's sub-goals are realisable and selected by shortest-path search over recorded experience. We keep the title, as realisable sub-goals are what our analysis shows existing hierarchical JEPA approaches lack, rather than the cause of Metro's gains.

  2. We apologise for the confusing wording in A2. We wrote the sentence about unreachable bridges in general, not specifically about aliasing: the ε cap and frame pricing mitigate long bridges, and neither applies to aliased pairs. We have reworded A2 to make this clear. As A2 states, aliasing is a property of the world model's representation, which the micro planner's objective shares, and Metro cannot detect it. Re-entry does not detect it either; it only recovers when a bridge leads to a sub-goal the micro planner cannot reach, by re-planning from wherever the planner landed. If the current state itself is aliased, re-entry can attach it to the wrong vertex; if this happens repeatedly, A2 is violated and Metro fails.

  3. Empirically, A4 holds at 99.6% of the points of the pricing curves across all benchmarks in both our setup and the H-JEPA protocol; the only exception is Λ(2) on TwoRoom (7.83 < Λ(1) = 8.22). The running maximum is a safeguard for such edge cases, so that a larger latent gap is never charged fewer frames.

    You are right that Eq. 5 is undefined when Λ(d+1) = Λ(d). To make Eq. 5 well defined, we change the interval from [Λ(d), Λ(d+1)] to [Λ(d), Λ(d+1)). A tied interval is then empty, so a gap equal to the tied value falls into the next interval, whose denominator is non-zero, or is clipped to H if the tie is at the end of the curve. In practice, no bridge falls in a tied interval, as every TwoRoom bridge is shorter than Λ(1), so this does not change any of our results.

  4. You are right on both points. Routing by latent distance is our only fallback, as latent distance is the only signal left when the graph has no route from the current state to the goal, i.e. when A1 (coverage) is violated. Metro then steps to the entry candidate closest to the goal (we corrected the imprecise wording in Section 5.3) if it is closer than the current state, and otherwise hands the goal to the micro planner. We agree that this choice is uninformative when the goal is far, as the latent distance saturates; the micro planner has the same problem, as its objective uses the same distance. With the full corpus, this fallback is used in at most 0.2% of macro decisions.

    Yes, this accounts for the no-bridge variant falling below flat planning. As noted in R1-e, few single demonstrations connect start and goal, so without bridges Metro mostly falls back, which is uninformative at these distances. We revised the paper to state this in Appendix I (A1) and Appendix E.3.

Thank you again for your rigorous review; your feedback has significantly improved our paper. We hope these answers address your remaining concerns.

Reviewer_RkgXOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes METRO-WM, a hierarchical planning framework for long-horizon tasks using pretrained latent world models. The authors observe that existing hierarchical latent planners may generate physically unrealisable sub-goals, which can negatively affect the performance of low-level planners. To address this issue, they propose retrieving sub-goals from previously observed trajectories instead of generating new latent representations. The experimental results demonstrate that the proposed method achieves improved long-horizon planning performance.

Critical strengths
  1. The authors observe that the generated latent sub-goals may not correspond to physically realisable states, which is a reasonable concern in sub-goal-based hierarchical latent planning.

  2. The authors construct a graph based on offline trajectories, where consecutive states within each trajectory are connected through temporal edges, while similar states from different trajectories are connected through bridge edges.

Critical weaknesses
  1. My major concern is that the proposed method only guarantees that the retrieved sub-goals correspond to physically realisable states, but does not guarantee their reachability from the current state. I hope the authors can at least clarify which components of the proposed method can partially address this issue.

  2. I think the design of the edge weights is one of the key components of this paper. I hope the authors can provide a more detailed explanation of the intuition and motivation behind this design, especially Equation 5.

Questions for authors

N.A.

2 comments
AuthorsOpenReview ↗

Response to Reviewer RkgX

R2-a. Reachability is not guaranteed. This is correct: Metro guarantees realisability, not reachability, and the paper does not claim reachability. We have reworded parts of the paper to improve clarity (Sec. 5). Two components partially address reachability:

  1. sub-goals within micro planner’s range; each edge is priced in frames and each sub-goal is about H demonstrator frames away. Charging every bridge one frame instead lowers PushT success from 62.67 to 28.67 (App. E.3).

  2. re-entry: If the micro planner doesn’t reach the sub-goal, the next macro step re-plans from where it actually is (Table 3).

R2-b. Bridge edge weight intuition. We expanded the explanation (Sec. 5.2).

Dijkstra sums edge costs along a route, so all edges need one unit. A temporal edge is one frame of demonstrator motion, so we price bridges in frames too. Raw latent distances would not work: they saturate (App. D, Fig. 8), so summing them misranks long routes.

Eq. 5 converts a bridge's latent gap into frames. Λ(d) is the latent distance a demonstrator typically covers in d frames, and κ inverts it: a bridge whose gap equals Λ(d) costs d frames. A bridge between near-identical frames thus costs one frame, like a temporal step, while a bridge across a larger gap costs as many frames as a demonstrator needs to cover it, so the search cannot take it as a free shortcut.

We use a low quantile (α = 0.25) so that bridges are over-priced: a higher quantile charges fewer frames and lets the search buy shortcuts the micro planner cannot execute. We clip at H, the range where the micro planner is reliable and the latent metric has not saturated. We show that this matters as charging every bridge at a cost of one frame or charging a bridge at all lowers success rate (App. E.3).

AuthorsOpenReview ↗

Follow-up on our response

Dear Reviewer, thank you again for your review. As the discussion period closes on Nov 18, we wanted to check whether our response has addressed your concerns. We would be happy to clarify anything further or provide additional details. If our responses have resolved your concerns, we would be grateful if you would consider updating your assessment.

Reviewer_9GqVOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

METRO-WM addresses long-horizon planning with frozen latent world models by retrieving intermediate sub-goals from recorded experience. It constructs a graph with directed temporal edges and symmetric cross-episode bridges, calibrates bridge costs in frame units, and uses shortest-path search with dynamic graph re-entry to guide a local planner. The paper also introduces a decoder-free realizability residual to investigate generated latent sub-goals.

Critical strengths
  1. Retrieving recorded states ensures that issued sub-goals correspond to actual observations. The combination of temporal connectivity, cross-episode stitching, and graph re-entry provides a coherent solution without training an additional macro model.
  2. At the longest evaluated distance, (d=75), METRO-LeWM improves success over Hi-LeWM-C by 16.67, 18.00, 37.33, and 4.00 percentage points on PushT, Cube, TwoRoom, and Reacher, respectively, while reporting 2.2–10.9× lower latency (Section 6.2). Table 1 also shows substantially lower offline compute for graph preprocessing than macro-model training.
  3. The re-entry ablation supports an important design choice: removing re-entry reduces METRO-LeWM success from 100% to 76.67% on TwoRoom and from 62.67% to 49.33% on PushT.
Critical weaknesses
  1. CEM provides an upper bound on the minimum residual. Its plateau under a fourfold search-budget increase is suggestive, but cannot establish that no realizable state exists. The strong wording in Section 4 should be qualified.
  2. Symmetric bridges rely on visual similarity, which need not imply mutual reachability or compatible hidden dynamics. The unrestricted nearest-neighbor attachment fallback further weakens any interpretation of the graph as guaranteeing feasible routes.
  3. The paper selects configurations from extensive sweeps, but does not clearly identify a separate validation set. Appendix G.3 also documents different micro-planner search budgets and execution schedules across methods, complicating attribution of the gains specifically to macro planning.
  4. The paper acknowledges established retrieval-based graph planning. Additional comparisons isolating cross-episode bridges and quantile-calibrated edge costs would clarify which adaptations account for the improvement beyond graph retrieval itself.
Questions for authors

The authors are suggested to address my concern listed above.

2 comments
AuthorsOpenReview ↗

Response to Reviewer 9GqV

R1-a. Realisability wording. We agree and have qualified the wording (Sec. 4): the search finds no nearby realisable state. To improve interpretability, we add a reference (App. B): the same search run on Metro’s sub-goals, which are recorded frames and therefore realisable: Median r(z):

CubePushT
Hi-LeWM-C sub-goals (23k → 92k renders)0.92 → 0.902.31 → 2.30
Metro sub-goals (23k)0.0890.206

The smaller search budget already reaches about 0.09 on Cube and 0.21 on PushT. Note that it is not exactly zero because the simulator’s re-render is not pixel-identical to the recorded frame. This does not prove that no realisable state exists but it shows that none is found anywhere near the scale at which a realisable frame is found.

R1-b. Feasible route guarantees. We agree, and we do not claim that graph guarantees feasible routes. Metro guarantees only that its sub-goals are realisable (each is a recorded frame) and that its route is a shortest path over the graph. A bridge is an estimate that two visually similar frames are a few frames apart, and it is priced by that estimate. When a bridge is not physically reachable, re-entry mitigates it: the next macro step re-plans from where the planner actually is (committing to the first route instead lowers success, e.g. TwoRoom 100 → 76.67, PushT 62.67 → 49.33; Table 3). We revised the paper to make this clearer (Sec. 1,2,5).

R1-c. Validation set. The protocol of the main experiments, from LeWM (Maes et al., 2026) and Hi-LeWM (Caselli et al., 2026), provides no separate validation set: evaluation queries are sampled from the training dataset, and Hi-LeWM sweeps its planning hyperparameters and reports the best result on these queries. We apply the same procedure to every method. The hierarchical baselines use the authors' best reported configurations (PushT, Cube) or the best of a sweep following their paper (TwoRoom, Reacher), and Metro's (h, H) are selected the same way. We now state this explicitly (App. G.4). To address your concern, we evaluate Metro on the new H-JEPA protocol (see general response; Sec. 6.3, App. H), where planner configurations are tuned on tasks disjoint from the evaluation set.

R1-d. Micro-planner differences. Each method runs at its own best configuration, as is standard: the hierarchical baselines use the authors' best reported micro-planner settings, and Metro uses LeWM's released solver. Where the settings differ, they favour the baselines: their micro planner uses 4× the samples (1200 vs 300) and replans after every action block, while Metro executes each plan open-loop.

To check that Metro's gain does not come from its own micro-planner settings, the fallback ablation (App. E.4) removes the whole graph: Metro keeps its micro planner settings and execution schedule, but hands the final goal to the micro planner instead of a sub-goal. Success (%), d = 75, T = 300:

PushTCubeTwoRoomReacher
Metro-LeWM62.67 ± 4.6292.67 ± 3.06100.00 ± 0.00100.00 ± 0.00
Same micro planner without the graph18.67 ± 1.1568.67 ± 1.1516.00 ± 2.0057.33 ± 1.15

The gain therefore comes from Metro's macro planning, not from its micro-planner settings.

R1-e. Isolating bridges and edge costs. We added a bridge ablation (App. E.3, Table 8), where we either remove all cross-episode bridges, keeping only temporal edges, or charge every bridge one frame regardless of its latent gap (unit cost). We rerun the (h,H) sweep for both cases. Success (%), d = 75, T = 300:

PushTCubeTwoRoomReacher
Metro-LeWM62.67 ± 4.6292.67 ± 3.06100.00 ± 0.00100.00 ± 0.00
Unit-cost bridges28.67 ± 3.0688.67 ± 4.16100.00 ± 0.00100.00 ± 0.00
No bridges12.67 ± 1.1568.00 ± 2.0012.67 ± 3.0646.67 ± 3.06

Removing cross-episode bridges costs 25-87 pp on every benchmark as few single demonstrations connect start and goal. Unit-cost bridges lose 34 pp on PushT, where half of the bridges span more than one frame, and 4 pp on Cube. On TwoRoom and Reacher, every bridge spans less than one frame, so both pricings are identical.

AuthorsOpenReview ↗

Follow-up on our response

Dear Reviewer, thank you again for your review. As the discussion period closes on Nov 18, we wanted to check whether our response has addressed your concerns. We would be happy to clarify anything further or provide additional details. If our responses have addressed your concerns, we would be grateful if you could let us know, and consider reflecting this in your assessment.

Comments on the paper

1 comment
AuthorsOpenReview ↗

General Response to all Reviewers

We thank all reviewers for their careful reading and constructive feedback. All changes in the revised paper are shown in blue.

After the submission deadline, the concurrent H-JEPA (Zhang et al., 2026b) released its code, checkpoints and test tasks (https://github.com/kevinghst/H-JEPA), together with HWM (Zhang et al., 2026a) as a baseline. We adopt their protocol (App. H) and compare Metro against H-JEPA and HWM with 2–4 levels (Sec. 6.3, Table 2) for two reasons:

  1. to compare with the latest hierarchical world models,
  2. to address some of the reviewers' concerns, as detailed in the individual responses

Metro uses their flat LeWM checkpoints and planner. We rerun every baseline with their released code, which largely reproduces their reported results. Success (%), mean ± SD over 3 seeds:

CubePushTAntMazeFourRoom
Flat LeWM25.33 ± 4.1638.00 ± 5.2916.00 ± 5.2946.00 ± 15.62
Best H-JEPA (levels)56.67 ± 6.43 (4)41.33 ± 6.43 (2)67.33 ± 6.43 (3)96.00 ± 2.00 (3)
Best HWM (levels)57.33 ± 4.16 (3)49.33 ± 4.16 (2)33.33 ± 4.62 (2)98.00 ± 2.00 (3)
Metro-LeWM64.00 ± 7.2152.67 ± 4.1681.33 ± 11.7298.67 ± 1.15

Metro matches or exceeds the best hierarchy on every benchmark, without choosing a depth or training a macro level.

The released hierarchies also let us repeat our realisability analysis on H-JEPA and HWM (App. B), showing that unrealisable sub-goals are not specific to Hi-LeWM-C.

Additions in the revision:

  • Comparison with H-JEPA and HWM in the H-JEPA protocol (Sec. 6.3, Table 2, App. H) (9GqV).
  • Realisability of H-JEPA and HWM sub-goals (App. B).
  • A realisable reference for the realisability residual: r(z) of Metro's sub-goals (Sec. 4, App. B) (9GqV).
  • A fallback ablation: Metro with every episode that passes near the goal removed from the graph, up to the whole graph, where Metro always hands the goal to its micro planner (App. E.4) (9GqV, 6CTP).
  • An ablation of cross-episode bridges and their pricing (App. E.3) (9GqV, RkgX).
  • An expanded explanation of the bridge edge weights, Eq. 5 (Sec. 5.2) (RkgX).
  • Assumptions and limitations (App. I) (6CTP).
  • Related work on latent sub-goals outside JEPA (Sec. 2, App. F) (6CTP).
  • Wording changes throughout the paper to make the claims clearer and more precise, e.g. Metro guarantees realisable sub-goals, not reachability (9GqV, RkgX, 6CTP).

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.