acceptodds
Under review as a conference paper at ICLR 2027

Anchors Guided Latent World Model for Long-Horizon Vision-and-Language Navigation

Abstract

Vision-and-Language Navigation (VLN) remains challenging, requiring agents to ground language instructions into actions under partial observability. In long-horizon VLN tasks, accumulated trajectory errors during planning lead to progressive deviation from the goal, degrading navigation efficiency. Recent methods introduce world models to anticipate future navigation states and improve candidate action evaluation, which can be broadly categorized into two categories. One line predicts the next state for local action refinement, while the other performs multi-step prediction for long-horizon planning. However, both classes of methods lack an explicit representation of global structure, making them susceptible to inefficient exploration in semantically ambiguous environments. To address this limitation, we propose AnchorWorld, which introduces a set of sparse prior anchors, explicitly supervised by future navigation states, to guide latent-space action generation within the world model, thereby enhancing global structural awareness and improving navigation efficiency. Extensive experiments on the R2R and R4R datasets demonstrate the effectiveness of the proposed method. The advantages are most pronounced in long-horizon navigation, demonstrating improved efficiency and robustness in long-term decision-making.

Primary area:
reinforcement learning
Keywords:
Vision-and-Language Navigation, World Model, Long-Horizon Planning, Sparse Anchors, Latent-Space Action Generation

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_rxMsOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes AnchorWorld, an anchor-guided latent world model for long-horizon Vision-and-Language Navigation. The method introduces a Prior Anchor Extractor (PAE), which predicts sparse future anchors from navigation history and language instructions under shortest-path supervision, and an Anchor-based Latent Planner (ALP), which generates multi-step latent predictions for candidate navigation actions conditioned on these anchors. The predicted future representations are incorporated into a DUET-based navigation policy through attention and gated feature fusion. Experiments on R2R and R4R demonstrate competitive navigation performance. Additional experiments on REVERIE, an alternative backbone, and computational cost are provided in the appendix.

Critical strengths
  1. The proposed framework addresses a relevant challenge in long-horizon VLN by combining sparse global trajectory priors with candidate-conditioned latent rollouts. This design provides a potentially efficient alternative to computationally expensive explicit trajectory search.

  2. AnchorWorld achieves competitive performance on R2R and R4R, with improvements particularly evident in longer navigation trajectories. The component ablations indicate that PAE and ALP provide complementary benefits.

Critical weaknesses
  1. Insufficiently specified candidate-conditioned future supervision and probabilistic world-model objective. A central claim is that ALP generates distinct future latent trajectories for different candidate actions. However, Appendix A.3 defines the future-state supervision using shortest-path states without clearly specifying candidate-dependent targets. It is therefore unclear how counterfactual candidate actions are supervised, particularly when their consequences are not observed in the expert trajectory. This ambiguity raises concerns about whether the learned model truly captures action-dependent future dynamics. Moreover, although ALP is described as a CVAE, the reported objective contains only an MSE prediction loss, without an explicit variational posterior or KL regularization. The probabilistic formulation therefore requires further clarification.

  2. Potential mismatch between shortest-path supervision and long-horizon instruction following. PAE is supervised using future anchors uniformly sampled from the shortest path between the current position and the goal. However, long-horizon VLN instructions, particularly in R4R, may specify routes that differ from the shortest path. Consequently, the proposed anchors may favor goal-directed shortcuts rather than faithful execution of intermediate language instructions. The current R4R evaluation emphasizes SR, SPL, and NE but does not directly measure whether the agent follows the instructed route. This limits the evidence supporting the claimed improvements in long-horizon instruction-grounded planning.

Questions for authors
  1. How are the ground-truth future states constructed for different candidate actions in ALP? Does each candidate receive a distinct target trajectory? If not, how does the model avoid predicting similar futures for different actions? Quantitative analysis of action-conditioned prediction accuracy would materially strengthen this claim.

  2. Could the authors clarify the conditional prior, posterior distribution, and complete optimization objective of ALP? Why is no KL divergence term included in the reported loss? How is meaningful stochastic variation maintained in the learned latent predictions?

  3. Given that R4R reference trajectories are not necessarily shortest paths, could the authors compare shortest-path anchor supervision with instruction-trajectory-based supervision? Could they also report route-fidelity metrics such as nDTW, SDTW, or CLS to determine whether improvements in goal-reaching efficiency preserve instruction adherence?

7 comments
AuthorsOpenReview ↗

Response to Reviewer rxMs (Part 1/2)

Response to Weakness1 and Question2

1. Clarifying Candidate-Specific Future Supervision

Thank you for the thoughtful comment. During latent rollout, our world model uses the state at each step and key anchors along the shortest path from that state to the navigation goal as conditions, rather than rolling out a sequence of future candidate actions. This design does not require multiple counterfactual expert trajectories; the dataset provides only one expert trajectory per instruction. Instead, supervision requires only shortest paths from navigation states to the goal. Specifically, PAE is supervised by key points sampled approximately uniformly along a shortest path. Its predicted anchors then guide ALP’s rollout, whose outputs are aligned with shortest-path states.

We have elaborated on the candidate-specific supervision in Appendix A.3.1.

2. Clarifying the loss of ALP

Thank you for your careful check. In describing ALP, we focused on the future-state prediction objective without fully presenting the associated CVAE training formulation. Consequently, the KL regularization term was not explicitly shown in the reported loss. We have expanded Appendix A.3.1 to describe ALP’s training procedure in detail.

For each candidate jj, we construct a shortest-path continuation from its resulting viewpoint to the goal and encode the consecutive viewpoint states as targets yt,jk⋆y_{t,j}^{k\star}. Targets for unselected candidates are derived from the training graph; they provide counterfactual supervision rather than representing observed executions. The candidate embedding ct,jc_{t,j}, previous predicted state, and shared PAE anchor define the rollout context ut,jku_{t,j}^k. ALP specifies a conditional prior pψ(zt,jk∣ut,jk)p_\psi(z_{t,j}^k\mid u_{t,j}^k) and a training-time recognition distribution qφ(zt,jk∣ut,jk,yt,jk⋆)q_\varphi(z_{t,j}^k\mid u_{t,j}^k,y_{t,j}^{k\star}). Its complete objective combines state reconstruction with a posterior-to-prior KL term, LALP=Lrec+βLKL\mathcal{L}_{\mathrm{ALP}}=\mathcal{L}_{\mathrm{rec}}+\beta\mathcal{L}_{\mathrm{KL}}; the original MSE-only description omitted the latter. Training uses reparameterized samples from the recognition distribution for reconstruction, whereas inference and policy scoring use separate prior-sampled rollouts without access to target states. The learned candidate-conditioned Gaussian permits stochastic predictions, and the KL term aligns the training-time distribution with the inference-time prior.

Response to Weakness2 and Question3

1. Clarifying our objective and evaluation focus.

Our work does not claim superior instruction-following capability; it targets improved goal reaching and navigation efficiency in long-horizon navigation, as stated in Lines 107 and 199.

Our reported metrics emphasize goal-reaching performance (NE/SR) and navigation efficiency (SPL), whereas nDTW/SDTW/CLS assess instruction fidelity through adherence to reference routes. These metrics evaluate different aspects of navigation that can involve a trade-off. In R4R, reference routes are constructed by concatenating adjacent trajectories and are not always the shortest paths. Consequently, closer adherence to the reference route does not necessarily yield the most efficient path to the destination.

2. Additional evaluation on R4R.

As suggested, we additionally report CLS, nDTW, and SDTW to assess instruction fidelity alongside goal-reaching performance and navigation efficiency on R4R Validation Unseen. Here, we use α\alpha to tradeoff of instruction fidelity and navigation efficiency: α=0\alpha=0 disables the shortest-path enhancement in our approach; α=0.5\alpha=0.5 partially incorporates our method, and α=1\alpha=1 applies it fully.

α\alphaNE ↓SR ↑SPL ↑CLS ↑nDTW ↑SDTW ↑
05.9148.7745.7046.2340.1025.98
0.55.6450.5547.0146.8440.9027.16
15.5351.3547.6246.4340.6227.43

Overall, our method substantially improves navigation efficiency, while gains in route fidelity are modest. One possible explanation is that R4R’s longer instructions may describe detours rather than the most efficient shortest paths.

AuthorsOpenReview ↗

Response to Reviewer rxMs (Part 2/2)

3. Shortest-path supervision and instruction adherence

PAE learns anchor targets aligned with the supervision it receives. We compare the two supervision strategies below:

(i) With supervision from uniformly sampled points along the shortest path, our method achieves better goal-reaching performance and navigation efficiency (NE, SR, and SPL in B1). These results suggest that PAE can filter out textual information that is redundant for goal reaching when fusing instruction and visual features (Fig. 2).

(ii) With instruction-aligned trajectory supervision, our method achieves higher instruction fidelity, as measured by CLS and nDTW (B2). This supervision explicitly encourages PAE to generate anchors that follow the instructed route.

IdxSettingNE ↓SR ↑SPL ↑CLS ↑nDTW ↑SDTW ↑
B1Ours (shortest-path supervision)5.5351.3547.6246.4340.6227.43
B2Ours (Instruction-aligned trajectory supervision)5.6546.9540.3747.9043.7426.86

Together, these results suggest that PAE can prioritize features relevant to the chosen learning objective, allowing the same framework to accommodate different route requirements.

Response to Question1

Thanks for your thoughtful comment. our responses are mainly three-fold:

(i) During latent rollout, our future world model predicts based on the position at each step and key anchor conditions along the shortest path from that position to the navigation goal, rather than rolling out based on candidates’ future actions. The advantage is that our learning objective does not require multiple counterfactual expert trajectories (the dataset contains only one expert trajectory). It requires only the shortest path from each navigation step to the goal. Accordingly, PAE is first supervised by key points uniformly sampled along the shortest path. The anchors predicted by PAE are then provided to ALP for prediction, and ALP's outputs are aligned with the shortest path.

(ii) Regarding the discriminability of counterfactual predictions raised by the reviewer, predictions of different futures in our method are conditioned on the same anchors along the future shortest path. Our predictions are therefore more deterministic, focusing on shortest-path navigation. We introduce Gaussian sampling [1, 2] to allow the predictions to reflect real-world uncertainty. Nevertheless, discriminability is a valuable research question that we plan to investigate further. Thank you again for this insightful question.

[1] DESIRE: Distant Future Prediction in Dynamic Scenes with Interacting Agents. CVPR, 2017.
[2] Hierarchical Latent Structure for Multi-modal Vehicle Trajectory Forecasting. ECCV, 2022.

(iii) There is currently no unified standard for evaluating counterfactual reasoning in world-model research. This is mainly because expert ground-truth trajectories for different future actions from the same initial state are unknown in the test set and difficult to collect. We therefore use an alternative evaluation to quantify our method. Specifically, in our setting, “future-state prediction” could refer either to PAE’s anchor predictions or to ALP’s predicted latent states. To address both interpretations, we report two prediction-accuracy analyses on R2R Val Unseen. We measure the cosine similarity and mean squared error (MSE) between the states predicted by PAE or ALP and their ground-truth targets. The table reports the mean ± standard deviation for each step and across all three steps.

ModuleStepCosine Similarity ↑MSE ↓
PAE10.805 ± 0.0610.182 ± 0.054
PAE20.761 ± 0.0760.221 ± 0.069
PAE30.730 ± 0.0890.247 ± 0.081
PAEMean0.765 ± 0.0830.216 ± 0.074
ALP10.742 ± 0.0830.241 ± 0.072
ALP20.697 ± 0.0900.281 ± 0.081
ALP30.673 ± 0.0930.303 ± 0.085
ALPMean0.710 ± 0.0930.270 ± 0.083
Area_Chair_7GEvOpenReview ↗

Response Reminder

Dear Reviewer,

As we approach the end of the discussion period, we invite you to review the authors’ response to your comments. Please update us on whether your concerns have been sufficiently addressed and whether you wish to pose further questions to the authors.

We greatly appreciate your contribution.

Reviewer_1CoZOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

AnchorWorld addresses long-horizon vision-and-language navigation (VLN), focusing on the challenge of balancing candidate action coverage and future rollout depth under a limited computational budget. It proposes a latent world model guided by sparse prior anchors. The prior anchor extractor (PAE) infers anchors from navigation history and instructions, using future states along the shortest path as supervision. Conditioned on local observations and global anchors, the anchor-based latent planner (ALP) generates a compact, fixed-length latent future rollout for each candidate action. These route-level, candidate-conditioned predictive representations support policy learning without explicit tree search.

Critical strengths
  1. AnchorWorld uses sparse, instruction-conditioned route anchors to guide candidate-specific latent predictions. This provides a practical way to introduce longer-horizon guidance without dense visual reconstruction or explicit trajectory tree expansion.

  2. AnchorWorld achieves 65.0 SPL on R2R Test Unseen, compared with 59.0 for the reported DUET baseline and 62.8 for the reproduced DreamWalker. Further gains on R4R and REVERIE provide encouraging evidence that the method benefits navigation across different instruction settings.

  3. The experiments evaluate PAE and ALP separately and jointly, compare anchor sampling strategies, and examine the number of anchors. Table 2 shows that combining both modules yields higher SPL than either module alone. Additional experiments on REVERIE and with HAMT broaden the empirical evaluation, although the gains with HAMT on unseen splits are modest.

Critical weaknesses
  1. The claimed action-conditioned world model is insufficiently described and validated. In Eqs. (4)–(5), each recursive update receives the same initial candidate-action embedding, but subsequent actions or a continuation policy are not defined. Moreover, K indexes uniformly sampled positions along the remaining route while also being described as the rollout horizon, leaving the temporal meaning of each transition unclear.

  2. The contributions should be better summarized. The statement "Extensive experiments on R2R and R4R demonstrate state-of-the-art performance, with notable gains in long-horizon navigation efficiency." cannot be considered as a contribution. Conducting experiments is a means to verify the effectiveness of your technical innovations, not an end in itself.

  3. PAE is supervised along the shortest path to the final goal, whereas R4R instructions may describe longer, non-shortest routes. The evaluation also prioritizes SR, SPL, and NE over instruction-following metrics. However, R4R was specifically introduced to expose the limitations of goal-completion metrics in evaluating adherence to instructed routes. Without CLS, nDTW, or SDTW, the central claim of improved long-horizon instruction following remains insufficiently supported.

  4. The ablations do not isolate the benefits of learned future dynamics. Component ablations show that PAE and ALP improve navigation scores, but do not explain why. There is no capacity-matched control that directly incorporates predicted anchors without recursive rollout, nor a diagnostic experiment that removes or shuffles candidate-action conditioning. Direct evaluation of future-state prediction accuracy is also missing. The gains may therefore arise from additional route supervision or feature-processing capacity rather than the claimed candidate-conditioned imagination mechanism.

  5. The comparison protocol remains unclear. DreamWalker was originally designed for continuous VLN, but its adaptation to the discrete graph setting is insufficiently documented. The paper should also clarify whether its adapted action space, planning procedure, and computational cost are comparable to those of AnchorWorld.

  6. Table 10 reports an inference latency of 0.23 ms for AnchorWorld, but does not adequately specify the timing scope, batch size, numerical precision, cached computations, GPU synchronization, or number of candidates. The memory measurement is also undefined. Although the trade-off between candidate coverage and rollout horizon is central to the paper’s motivation, runtime scaling with these two factors is not reported. The table therefore does not establish the claimed computational advantage under comparable operating conditions.

Questions for authors
  1. Could you clarify what each transition and (K) represent in Eqs. (4)–(5), and what determines the continuation after the first action when the initial candidate-action embedding is reused? What evidence isolates the benefit of candidate-conditioned dynamics from additional anchor supervision or model capacity?

  2. How are shortest-path anchor targets reconciled with instructions that describe non-shortest routes? Are results available on route-adherence metrics such as CLS, nDTW, or SDTW?

  3. How was DreamWalker adapted to the discrete graph setting, and were its action space and planning budget comparable to AnchorWorld's?

4 comments
AuthorsOpenReview ↗

Response to Reviewer 1CoZ (Part 1/3)

Response to Weakness1 and Sub-question1 of Question1

1. Clarifying action conditioning and latent continuation.

In the ALP model, each future-transition prediction is conditioned on the current candidate action (ct,jc_{t,j}), which serves as a clean prior that provides an initial directional cue. Subsequent transition predictions are primarily guided by the anchors generated by PAE. Thus, our model requires neither explicit subsequent actions nor a continuation policy. The core idea is an anchor-guided latent world model. We have highlighted this point in the main text (Line 201-203).

2. Clarifying the meaning of KK.

Thanks for your thoughtful comment. In our formulation, our ALP adopts one-to-one temporal conditioning: the update at rollout step kk is conditioned on the kk-th anchor generated by PAE. Accordingly, the number of PAE anchors matches ALP’s rollout horizon, with each anchor conditioning its corresponding rollout step in sequence.

In the table below, we compare four variants: (A1) our proposed one-to-one temporal conditioning; (A2) fusing all ordered anchors into a single anchor representation and using this same representation at every ALP rollout step; (A3) randomly shuffling the anchors before feeding them to ALP; and (A4) removing temporal conditioning. The results show that our proposed temporal conditioning achieves the best performance, with the ranking A1 > A2 > A3 > A4.

IdxMethodNE ↓SR ↑SPL ↑
A1Ours (one-to-one temporal conditioning)2.9072.763.3
A2Ours (fusion-condition)3.0272.462.62
A3Ours (shuffle-condition)3.1772.262.0
A4Ours (remove-condition)3.1172.061.6

Response to Weakness2

Our initial intention was to summarize empirical evidence supporting the technical contributions. Specifically, Fig. 1(d) motivates the need to maintain navigation efficiency on longer routes, while Tables 1 and 4 demonstrate the benefits of our design. Together, these results support the insight that sparse route-level anchors can guide compact, goal-directed latent imagination, achieving the highest SPL among the compared methods on R2R Test Unseen and R4R Validation Unseen.

We have highlighted this part in the revised contribution (Line 106-107).

Response to Weakness3 and Question2

1. Clarifying our objective and evaluation focus.

Our work does not claim superior instruction-following capability; it targets improved goal reaching and navigation efficiency in long-horizon navigation, as stated in Lines 107 and 200.

Our reported metrics emphasize goal-reaching performance (NE/SR) and navigation efficiency (SPL), whereas nDTW/SDTW/CLS assess instruction fidelity through adherence to reference routes. These metrics evaluate different aspects of navigation that can involve a trade-off. In R4R, reference routes are constructed by concatenating adjacent trajectories and are not always the shortest paths. Consequently, closer adherence to the reference route does not necessarily yield the most efficient path to the destination.

2. Additional evaluation on R4R.

As suggested, we additionally report CLS, nDTW, and SDTW to assess instruction fidelity alongside goal-reaching performance and navigation efficiency on R4R Validation Unseen. Here, we use α\alpha to tradeoff of instruction fidelity and navigation efficiency: α=0\alpha=0 disables the shortest-path enhancement in our approach; α=0.5\alpha=0.5 partially incorporates our method, and α=1\alpha=1 applies it fully.

α\alphaNE ↓SR ↑SPL ↑CLS ↑nDTW ↑SDTW ↑
05.9148.7745.7046.2340.1025.98
0.55.6450.5547.0146.8440.9027.16
15.5351.3547.6246.4340.6227.43

Compared with α=0\alpha=0, α=1\alpha=1 improves all six metrics, including SPL by 1.92 points and SDTW by 1.45 points. Thus, future-feature feedback improves navigation efficiency alongside aggregate route fidelity. Among these settings, α=0.5\alpha=0.5 achieves the highest CLS and nDTW, while α=1\alpha=1 achieves the best NE, SR, SPL, and SDTW.

AuthorsOpenReview ↗

Response to Reviewer 1CoZ (Part 2/3)

3. Shortest-path supervision and instruction adherence

PAE learns anchor targets aligned with the supervision it receives. We compare the two supervision strategies below:

(i) With supervision from uniformly sampled points along the shortest path, our method achieves better goal-reaching performance and navigation efficiency (NE, SR, and SPL in B1). These results suggest that PAE can filter out textual information that is redundant for goal reaching when fusing instruction and visual features (Fig. 2).

(ii) With instruction-aligned trajectory supervision, our method achieves higher instruction fidelity, as measured by CLS and nDTW (B2). This supervision explicitly encourages PAE to generate anchors that follow the instructed route.

IdxSettingNE ↓SR ↑SPL ↑CLS ↑nDTW ↑SDTW ↑
B1Ours (shortest-path supervision)5.5351.3547.6246.4340.6227.43
B2Ours (Instruction-aligned trajectory supervision)5.6546.9540.3747.9043.7426.86

Together, these results suggest that PAE can prioritize features relevant to the chosen learning objective, allowing the same framework to accommodate different route requirements.

Response to Weakness4 and Sub-question2 of Question1

Thank you for the constructive comments.

1. Ablating learned future dynamics and explaining the contributions of PAE and ALP

The comparison between C2 and C4 in Table 2 demonstrates the performance gain from ALP’s future-dynamics prediction. We also explain why PAE and ALP together (C4) improve navigation metrics—including NE, OSR, SR, and SPL—in Section 4.3, Lines 431–472.

2. Comparing prediction paradigms for ALP

As suggested, we compare ALP’s original autoregressive rollout (D2) with a one-shot prediction of all future states (D1). In the one-shot variant, all anchors predicted by PAE are concatenated along the channel dimension and provided as input. We use a network matched in parameter count to the autoregressive model to ensure a fair comparison.

The results show that recursive rollout achieves better navigation performance than one-shot prediction, suggesting that autoregression makes more effective use of PAE’s anchor conditions. We further examine how the anchors are used in the next analysis.

IndexDescriptionNE ↓OSR ↑SR ↑SPL ↑
D1One-shot prediction3.0480.472.262.9
D2Autoregressive rollout2.9081.072.763.3

3. Assessing conditioning in ALP

We add two diagnostic controls that remove (E1) or shuffle (E2) the conditioning information while retaining recursive rollout. On R2R Val Unseen, the original ordered-anchor conditioning (E3) achieves the best navigation performance. This suggests that ALP benefits from temporally aligned anchors: each predicted future state is conditioned on the anchor corresponding to its rollout step.

IndexDescriptionNE ↓OSR ↑SR ↑SPL ↑
E1Ours (removed condition)3.1180.072.061.6
E2Ours (shuffled condition)3.1781.072.262.0
E3Ours (ordered condition)2.9081.072.763.3

4. Quantifying future-state prediction accuracy

''Future-state prediction'' could refer either to PAE's anchor predictions or to ALP’s predicted latent states. To address both interpretations, we report two prediction-accuracy analyses on R2R Val Unseen. We measure the cosine similarity and mean squared error (MSE) between the states predicted by PAE or ALP and their ground-truth targets. The table reports the mean ± standard deviation for each step and across all three steps.

ModuleStepCosine Similarity ↑MSE ↓
PAE10.805 ± 0.0610.182 ± 0.054
PAE20.761 ± 0.0760.221 ± 0.069
PAE30.730 ± 0.0890.247 ± 0.081
PAEMean0.765 ± 0.0830.216 ± 0.074
ALP10.742 ± 0.0830.241 ± 0.072
ALP20.697 ± 0.0900.281 ± 0.081
ALP30.673 ± 0.0930.303 ± 0.085
ALPMean0.710 ± 0.0930.270 ± 0.083
AuthorsOpenReview ↗

Response to Reviewer 1CoZ (Part 3/3)

5. Clarifying the sources of navigation gains

Our contribution lies in turning route supervision into candidate-specific future predictions that directly inform action selection. The additional supervision and model components implement this mechanism: PAE learns sparse route-level priors, while ALP uses them to recursively predict a separate latent future for each candidate action. These predictions then refine the candidate representations used by the navigation policy.

The key design is how route-level guidance, candidate conditioning, and recursive prediction are integrated to support action evaluation through fixed-length latent rollouts, without explicit tree search. The component ablation provides supporting evidence: with PAE retained, adding ALP improves SPL from 61.7 to 63.3 on R2R Val Unseen (Table 2). Thus, the evaluated anchor-only variant does not recover the performance of the full framework.

Response to Weakness5 and Question3

Thank you for highlighting this point. To ensure a fair comparison, DreamWalker* and our method differ only in their planning procedures. Both use a prediction depth and breadth of three, and both operate in the same VLN action space.

Specifically, we adapt DreamWalker* to our discrete-graph navigation setting as follows:

  • We select the top three real candidate viewpoints as starting points for prediction. Subsequent search proceeds over the top three imagined nodes predicted by the waypoint predictor.
  • We train DreamWalker*’s goal-distance function as a distance evaluator that scores imagined discrete nodes according to their distance from the navigation goal.
  • The agent ultimately executes the highest-scoring real candidate viewpoint.

Response to Weakness6

We thank the reviewer for raising these concerns. First, we correct a unit typo in the original Table 10. The corrected values appear in the updated Table 11:

MethodParams (M)MemoryTime per inference (ms)SPL ↑
DUET181690 MB1859.0
UNeMo216826 MB2260.1
DreamWalker3251.12 GB8962.8
Ours211807 MB2365.0

Inference latency is the average time per navigation step, measured with a batch size of 8, three predicted anchors, and a three-step latent rollout, without cached computations. Inference was performed on a single GPU, so no inter-GPU synchronization was involved. We have clarified these details in Appendix A.3.2.

To examine the trade-off between computational cost and navigation performance, we provide the following sensitivity analysis. KK denotes the number of anchors, and HH denotes the latent-rollout horizon.

KKH=1H=1 Inference time (ms)H=1H=1 SPLH=3H=3 Inference time (ms)H=3H=3 SPLH=5H=5 Inference time (ms)H=5H=5 SPL
12060.62260.92360.1
32262.12363.32762.5
52560.02863.92963.1

For each fixed KK, H=3H=3 achieves the highest SPL, while inference time generally increases with the rollout horizon. We therefore select K=3K=3 and H=3H=3 as our operating point, balancing navigation performance and computational cost.

Area_Chair_7GEvOpenReview ↗

Response Reminder

Dear Reviewer,

As we approach the end of the discussion period, we invite you to review the authors’ response to your comments. Please update us on whether your concerns have been sufficiently addressed and whether you wish to pose further questions to the authors.

We greatly appreciate your contribution.

Reviewer_4Y2QOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper proposes AnchorWorld, a latent world-model module on top of DUET for long-horizon VLN. A Prior Anchor Extractor (PAE) predicts sparse latent anchors supervised by states sampled along the shortest path to the goal, and an Anchor-based Latent Planner (ALP) rolls out latent futures for each candidate action conditioned on these anchors. The paper claims SOTA SPL, with larger gains on long trajectories.

Critical strengths
  1. The paper is clearly written, and the refinement-based vs. search-based framing in Figure 1 motivates the problem well.
  2. The method is lightweight, with about 30M extra parameters over DUET (Table 10).
  3. SPL improves consistently over DUET on R2R, R4R and REVERIE.
Critical weaknesses
  1. The long-horizon claim is weakly supported. The ">=7 waypoints" R2R subset is only slightly longer than an average R2R path (about six waypoints, A.2), and its size is not reported. On R4R, shortest-path anchors can teach the agent to cut corners, which SPL rewards, but the paper explicitly drops instruction-fidelity metrics. I suggest the authors report CLS/nDTW/SDTW on R4R [*1].

[*1] Vihan Jain et al. Stay on the Path: Instruction Fidelity in Vision-and-Language Navigation. ACL 2019.

Questions for authors

Same as weaknesses.

2 comments
AuthorsOpenReview ↗

Response to Reviewer 4Y2Q

Thank you for your positive comment and insightful experimental suggestion.

1. Clarifying the long-horizon evaluation.

In the Introduction, we follow [1] and use the mean R2R trajectory length of six waypoints to define shorter (≤6) and longer (≥7) subsets, evaluating navigation efficiency in both settings. The subset sizes and results are shown below, with further details provided in Appendix A.1.

[1] UNeMo: Collaborative Visual-Language Reasoning and Navigation via a Multimodal World Model. AAAI, 2026.

Reference waypointsNumber of episodesPercentage of evaluated splitSPL (UNeMo) ↑SPL (DreamWalker) ↑SPL (Ours) ↑
≤651665.90%61.5263.0463.04
≥726734.10%59.5861.0963.69

Furthermore, Table 4 demonstrates our method’s navigation efficiency on R4R, whose validation-unseen reference routes are substantially longer on average than those in R2R (20.2 m vs. 9.5 m). Our method achieves the highest SR and SPL among the compared methods.

2. Adding instruction-fidelity metrics on R4R.

Our reported metrics (SPL/SR/NE) emphasize goal-reaching performance and navigation efficiency, whereas nDTW/SDTW/CLS focus on instruction-following capability. These two groups of metrics assess different aspects of navigation performance that may conflict to some extent. In R4R, reference routes are constructed by concatenating adjacent trajectories and are not always the shortest paths[1]. Consequently, strictly following the reference route may not necessarily yield the most efficient path to the destination.

As suggested, we additionally report CLS, nDTW, and SDTW to assess instruction fidelity alongside goal-reaching performance and navigation efficiency on R4R Validation Unseen. Here, we use α\alpha to tradeoff of instruction fidelity and navigation efficiency: α=0\alpha=0 disables the shortest-path enhancement in our approach; α=0.5\alpha=0.5 partially incorporates our method, and α=1\alpha=1 applies it fully.

α\alphaNE ↓SR ↑SPL ↑CLS ↑nDTW ↑SDTW ↑
05.9148.7745.7046.2340.1025.98
0.55.6450.5547.0146.8440.9027.16
15.5351.3547.6246.4340.6227.43

Overall, our method substantially improves navigation efficiency, while gains in route fidelity are modest. One possible explanation is that R4R’s longer instructions may describe detours rather than the most efficient shortest paths.

[1] Stay on the Path: Instruction Fidelity in Vision-and-Language Navigation. ACL, 2019.

Area_Chair_7GEvOpenReview ↗

Response Reminder

Dear Reviewer,

As we approach the end of the discussion period, we invite you to review the authors’ response to your comments. Please update us on whether your concerns have been sufficiently addressed and whether you wish to pose further questions to the authors.

We greatly appreciate your contribution.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.