acceptodds
Under review as a conference paper at ICLR 2027

GWAM: World Action Models for Efficient and Geometry Grounded Image-Goal Navigation

Abstract

World models capture environment dynamics and support counterfactual reasoning for planning. In image-goal navigation, where the target often lies beyond the current field of view, the agent must not only model environmental dynamics but also maintain a persistent spatial memory of the scene. Existing approaches either introduce spatial context by fine-tuning pretrained models on the target task, which underexploits their spatial priors, or rely on external 3D reconstruction, which is computationally expensive for real-world deployment. In this paper, we propose the Geometry Grounded World Action Model (GWAM) for image-goal navigation. GWAM encodes spatial cues through a learned scene-coordinate localization module that anchors the current, goal, and history frames in a common reference frame, yielding a stable and consistent spatial context that enables efficient navigation without explicit 3D reconstruction. Following the world-action model paradigm, GWAM jointly predicts future states and actions, eliminating the need for costly CEM-style planning. In addition, GWAM operates entirely in latent space, avoiding the computational burden of pixel-level reconstruction and substantially reducing inference cost. Across four trajectory prediction benchmarks, GWAM reduces average ATE and RPE by 40.1% and 35.2%, respectively. In closed-loop navigation experiments, it improves the success rate from 54.0% to 63.0% and the success weighted by path length (SPL) from 36.4% to 59.2%, while maintaining high efficiency.

Primary area:
applications to robotics, autonomy, planning
Keywords:
World Models, World Action Models, Image-Goal Navigation, Spatial Context

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_r28QOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

The paper proposes G²WAM for image-goal navigation, combining the poses of observation and goal images with scene coordinates, and predicting future images and actions in latent space to avoid explicit future-image reconstruction and iterative CEM-based planning. The experiments cover four trajectory prediction datasets, zero-shot transfer to a new dataset, and closed-loop navigation. The results demonstrate competitive performance in average trajectory prediction errors and Gibson navigation.

Critical strengths

1.The method uses a localization module to anchor the current observation, goal image, and historical observations to a shared scene coordinate system. It then extracts goal-relative poses and historical motion information, providing additional geometric context. The ablation results also demonstrate the benefits of incorporating this spatial information.

Critical weaknesses

1.Using latent image representations to provide auxiliary supervision is not a novel idea in itself. However, the ablation studies do not specifically investigate this component, leaving the contribution of latent image representations unclear. In particular, the paper does not clarify how this design compares with approaches that directly use image representations. I therefore consider the methodological novelty of this aspect insufficiently established.

Questions for authors

None.

7 comments
AuthorsOpenReview ↗

Response to Reviewer r28Q (1/2): W1

W1: Novelty and contribution of latent image supervision

Using latent image representations to provide auxiliary supervision is not a novel idea in itself. However, the ablation studies do not specifically investigate this component, leaving the contribution of latent image representations unclear. In particular, the paper does not clarify how this design compares with approaches that directly use image representations. I therefore consider the methodological novelty of this aspect insufficiently established.

Response. We sincerely thank the reviewer for the valuable comment. We agree that using future visual representations as auxiliary supervision is not a new idea in itself, and we do not present it as a standalone contribution of this paper. We have revised the paper to avoid presenting future visual-latent supervision itself as a novel methodological component. Our contribution lies in jointly predicting future latent states and executable actions conditioned on spatial context extracted from images, and in generating action sequences directly through state recursion. Future visual-latent supervision is an auxiliary objective for training this structure.

To address the two questions raised by the reviewer, we added the corresponding controls: first, a comparison with NoMaD, which generates actions directly from image representations; second, an ablation within G²WAM that removes future visual-latent supervision to test its incremental effect. In addition, we added a structural ablation of joint decoding and residual mixing to separate the contribution of the auxiliary supervision from that of the model structure.

1. Comparison with a method that directly uses image representations

In this experiment, NoMaD serves as the control method that generates actions directly from image representations. It encodes the current and past observations and the goal image into visual features and, conditioned on them, generates future waypoints through diffusion denoising, without explicitly predicting or rolling out future visual latents. G²WAM additionally predicts future visual latents jointly with actions and uses the predicted states for the subsequent rollout.

To examine whether the difference arises only from the visual encoder or the geometric information, we set up two NoMaD controls:

  • NoMaD + geometry: keeps NoMaD’s native visual encoder and diffusion-based action generation, and adds goal-relative pose and motion-history conditions.
  • NoMaD + geometry (frozen DINOv3): further replaces the visual encoder with the same frozen DINOv3 used by G²WAM, while keeping the geometric conditions and diffusion-based action generation.

The second setting lets the two methods share the visual encoder and geometric conditions, allowing a more direct comparison between using image representations for action generation and additionally modeling future states jointly with actions.

2. Ablations of visual-latent supervision and the joint structure

Within G²WAM, we set up the following variants:

  • w/o visual-latent supervision: removes only the supervision loss on future visual latents while keeping the latent prediction, rollout, and action-generation structure, to test the incremental effect of this auxiliary supervision.
  • w/o joint decoding & residual mixing: keeps the geometric conditions and the latent transition, removes joint decoding and residual mixing, and still outputs actions through the action-prediction head, to examine the combined contribution of these structural designs.

Four-dataset average trajectory prediction results are as follows (ATE/RPE, lower is better):

AuthorsOpenReview ↗

Response to Reviewer r28Q (2/2): W1

W1: Novelty and contribution of latent image supervision (continued)

MethodGT input ATE / RPE ↓Estimated input ATE / RPE ↓
NoMaD + geometry0.9133 / 0.30001.9527 / 0.4832
NoMaD + geometry (frozen DINOv3)0.9461 / 0.30451.8921 / 0.4700
G²WAM w/o joint decoding & residual mixing0.8065 / 0.28261.8604 / 0.4657
G²WAM w/o visual-latent supervision0.7988 / 0.27631.8800 / 0.4657
Full G²WAM0.7729 / 0.27661.8226 / 0.4575

Compared with NoMaD, which uses image representations directly, the full G²WAM achieves lower ATE/RPE than both NoMaD variants, including when NoMaD uses the same frozen DINOv3 encoder and receives the geometric conditions: relative to this control, the ATE decreases from 0.9461 to 0.7729 under GT inputs and from 1.8921 to 1.8226 under estimated inputs. This supports the overall value of jointly modeling future states and actions.

As for visual-latent supervision itself, removing this loss increases the GT-input ATE from 0.7729 to 0.7988 and the estimated-input ATE/RPE from 1.8226/0.4575 to 1.8800/0.4657, while the GT-input RPE slightly decreases from 0.2766 to 0.2763. The benefit of this supervision therefore lies mainly in ATE and in the estimated-input setting, rather than in a uniform improvement across all metrics.

Keeping the latent transition while removing joint decoding and residual mixing increases both ATE and RPE under both input conditions, supporting the combined contribution of these structural designs.

3. Closed-loop navigation

We further evaluated the same 100 episodes on Mosquito. All methods use a 2 m success threshold, oracle STOP, and at most 100 actions, and execute only the first action before re-observing in each decision cycle:

MethodGT input SR / SPL ↑Estimated input SR / SPL ↑
NoMaD + geometry65% / 0.634957% / 0.5303
NoMaD + geometry (frozen DINOv3)60% / 0.568755% / 0.4896
G²WAM w/o joint decoding & residual mixing65% / 0.649860% / 0.5921
G²WAM w/o visual-latent supervision67% / 0.659959% / 0.5800
Full G²WAM71% / 0.700363% / 0.5923

Compared with NoMaD, which uses the same frozen DINOv3 representations and geometric conditions, the full G²WAM improves SR by 11 and 8 points under GT and estimated inputs and SPL by 0.1316 and 0.1027, respectively; it also outperforms NoMaD with its native encoder by 6 SR points under both inputs, providing a closed-loop comparison between the two modeling approaches.

Compared with the variant without visual-latent supervision, the full model improves SR by 4 points under both input conditions and SPL by 0.0404 and 0.0123, respectively. This supports the incremental value of this auxiliary supervision.

Compared with the variant without joint decoding and residual mixing, the full model improves SR by 6 points and SPL by 0.0505 under GT inputs; under estimated inputs, SR improves by 3 points while SPL is essentially unchanged. The gains of these structural designs in the estimated-input closed-loop evaluation are therefore mainly reflected in the success rate.

In summary, we do not present future visual supervision itself as a novelty; our contribution is joint state–action modeling conditioned on spatial context. The controls show that, even compared with a direct policy that uses the same visual representation and geometric conditions, this design achieves better results in both trajectory prediction and closed-loop navigation, and future visual supervision, as an auxiliary objective, brings further gains on top of it.

We hope that the clarifications above and the additional experiments in the revised manuscript have addressed your concerns. If you find that these additions resolve the issues raised in your review, we would be grateful if you would consider updating your score accordingly. As the rebuttal deadline is approaching, please feel free to let us know if you have any further questions or concerns, so that we can address them in a timely manner during the remaining discussion period.

Reviewer_RHoROpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes G²WAM for image-goal navigation. A scene-specific R-SCoRe localizer first estimates the poses of the current, goal, and historical images in a shared scene coordinate frame, from which goal-relative pose and recent motion statistics are extracted. Using frozen DINOv3 image features, the model then jointly predicts future visual latents and actions through latent-state transitions conditioned on the goal and geometry. A motion adapter converts the predictions into executable waypoints. This design avoids future RGB reconstruction and CEM-based action search.

The authors evaluate eight-step trajectory prediction on four datasets and report transfer results on Go Stanford. On 100 closed-loop navigation episodes in the known Gibson Mosquito scene, the method achieves 63.0% SR, 59.2% SPL, and 3.39 m DTG. Ablations of pose and motion conditioning indicate that geometric inputs substantially improve trajectory prediction. The closed-loop experiment follows a known-scene, held-out-trajectory protocol, using a scene-specific localization database and navigation-model fine-tuning.

Critical strengths

1. The system design has practical value and substantially improves closed-loop path efficiency.

Conditioning joint latent-state and action prediction on low-dimensional goal-relative pose and recent motion information is a reasonable design with potential deployment benefits. The method avoids future RGB reconstruction and CEM search. Under the reported known-scene protocol, it achieves 63.0% SR and 59.2% SPL, improving over DINO-WM by 9.0 and 22.8 percentage points, respectively. The SPL improvement, in particular, suggests that the system may effectively reduce unnecessary movement.

2. The geometric-conditioning ablations are comprehensive and transparently demonstrate the effects of localization error.

Table 4 distinguishes no spatial conditioning, relative-pose-only conditioning, and combined relative-pose and motion conditioning, and further compares ground-truth and estimated inputs. Average ATE/RPE decreases from 2.99/0.71 without conditioning to 1.82/0.46 with estimated pose and motion, and further to 0.78/0.28 with ground-truth inputs. These results support the usefulness of geometric conditioning and demonstrate the importance of localization quality to final performance. The explicit disclosure of the known-scene scope and held-out evaluation trajectories also improves the transparency of the experimental interpretation.

Critical weaknesses

1. The contributions of localization information and the proposed world action model are not sufficiently disentangled.

G²WAM uses a scene-specific localization database to obtain goal-relative pose and motion statistics and is further fine-tuned on the Mosquito scene. Although the paper states that methods are evaluated on the same episodes, it does not sufficiently explain whether the baselines receive equivalent localization information, scene-adaptation data, and training budgets. Table 4 shows that geometric conditioning helps G²WAM, but does not establish that joint future-state and action prediction outperforms a direct navigation policy given the same information. A direct action-prediction baseline using the same DINOv3 features, pose and motion conditions, and training data is particularly important for assessing the core contribution.

2. The closed-loop protocol does not sufficiently establish the necessity of multi-step latent world modeling, and the validation scope is limited.

Appendix B explicitly states that the model predicts eight steps but executes only the first action before observing and replanning. According to Eq. 3, latent states generated by later rollout steps do not feed back to revise the already-generated first action. Multi-step prediction can therefore affect shared parameters through training supervision, but the closed-loop results do not establish an independent decision-making benefit from multi-step future simulation at inference time. Key comparisons are missing, including removing future latent-state supervision, using a direct policy, and performing only one-step inference. In addition, closed-loop evaluation covers only 100 episodes in one known scene, without multi-scene validation or uncertainty analysis, limiting conclusions about general navigation effectiveness.

Questions for authors

The following questions seek clarification and evidence for the core conclusions; the answers could change my rating.

  1. Are the information and training conditions matched across baselines? Please specify, for each baseline in Table 3, whether it uses the same Gibson adaptation data and 200 Mosquito trajectories as G²WAM, and whether it receives the same estimated goal-relative pose and motion information. Could you provide a direct policy baseline with the same visual encoder, geometric conditions, and training budget?

  2. What is the independent contribution of future-state modeling? Under the protocol that executes only the first action at each cycle, could you compare the full model against an action model without future latent-state supervision, and compare one-step versus eight-step inference in closed-loop navigation? Please distinguish the benefits of multi-step training from those of multi-step inference.

  3. Which metric and runtime definitions were actually used? Please reconcile the ATE/RPE definitions in the main text and appendix, and clarify whether the action-selection timings in Table 5 use matched batch sizes, rollout lengths, and timing boundaries. In particular, which settings correspond to EvoWorld’s 150.26 s in the table and 8.7 s in the text? Please also report batch-size-one, single-agent end-to-end latency, including image encoding, online localization, and action prediction.

  4. How stable are the closed-loop results, and how broadly do they apply? For the SR comparison of 63% versus 54%, could you provide paired per-episode outcomes or paired bootstrap confidence intervals? Are closed-loop results available for additional known scenes? Please confirm that all baselines use the same 2 m success threshold, oracle STOP, and action limit, and explain how episodes starting within the success threshold affect the aggregate metrics.

10 comments
AuthorsOpenReview ↗

Response to Reviewer RHoR (1/8): Q1 / W1

Q1 / W1: Matching of information and training conditions across baselines

Are the information and training conditions matched across baselines? Please specify, for each baseline in Table 3, whether it uses the same Gibson adaptation data and 200 Mosquito trajectories as G²WAM, and whether it receives the same estimated goal-relative pose and motion information. Could you provide a direct policy baseline with the same visual encoder, geometric conditions, and training budget?

Response. We sincerely thank the reviewer for this valuable suggestion. Below we specify the data and information conditions of each baseline in Table 3 and add direct-policy controls matched in information and training budget.

Adaptation data and training budgets. All methods in Table 3 use the same Gibson adaptation data and the same 200 Mosquito training trajectories. Their Mosquito adaptation schedules are summarized below:

MethodMosquito adaptation dataTraining schedule
G²WAM, NoMaD, DreamerV3, DINO-WM, GNM, LeWMThe same 200 trajectories500 updates, effective batch size 32
NWMThe same 200 episodes, with a longer training windowScene fine-tuning for 5 epochs (60 updates in total), followed by 1,500 distillation updates

NWM employs a separate deployment-acceleration pipeline: the original NWM paper (Real-World Applicability section and its Table 8) uses distillation to reduce the number of diffusion denoising steps from 250 to 6 to lower online inference cost. We reproduced NWM following this configuration and report its scene-adaptation and distillation budgets separately, rather than placing it in the unified 500-update group.

Information conditions. G²WAM constructs spatial context from images through its localization module and integrates it into the model through an explicit geometric conditioning interface, which is part of its architecture. The original baselines in Table 3 have no such interface and were adapted with their own native inputs; that is, they did not receive the same pose and motion information. We will state this explicitly in Appendix C.3 of the revision. Notably, this way of constructing spatial context and fusing it with the navigation network can be readily transferred to existing baselines, as the results below also confirm.

Matched direct-policy controls. To further address the reviewer’s request for a direct-policy control, we added:

  • NoMaD + geometry: keeps NoMaD’s native visual encoder and diffusion-based action generation, and adds the same goal-relative pose and motion-history conditions as G²WAM.
  • NoMaD + geometry (frozen DINOv3): further adopts the same frozen DINOv3 encoder and image preprocessing as G²WAM, while keeping the geometric conditions and NoMaD’s diffusion-based action generation.

NoMaD generates future waypoints directly from visual and geometric conditions, without explicitly predicting or rolling out future visual latents. The second setting therefore provides a direct action-policy control that shares the visual encoder and geometric conditions with G²WAM.

AuthorsOpenReview ↗

Response to Reviewer RHoR (2/8): Q1 / W1

Q1 / W1: Matching of information and training conditions across baselines (continued)

In the four-dataset training stage, all models use the same training data, 25,800 updates, and a global batch size of 32. Four-dataset average trajectory prediction results are as follows:

MethodGT input ATE / RPE ↓Estimated input ATE / RPE ↓
NoMaD + geometry0.9133 / 0.30001.9527 / 0.4832
NoMaD + geometry (frozen DINOv3)0.9461 / 0.30451.8921 / 0.4700
Full G²WAM0.7729 / 0.27661.8226 / 0.4575

Matching the visual encoder improves NoMaD's trajectory prediction under estimated inputs but not under GT inputs, and a clear performance gap to G²WAM remains in both cases. Relative to NoMaD with the same frozen DINOv3 encoder and geometric conditions, G²WAM reduces GT-input ATE by 18.31%and RPE by9.16%. Under estimated inputs, the corresponding reductions are3.67%and2.66%. The larger relative improvements under GT inputs suggest that when pose and motion estimation errors are removed from the conditioning inputs, the gains from the model design become more pronounced.

We further conducted closed-loop evaluation on Mosquito. All methods use the same 100 test episodes, a 2 m success threshold, oracle STOP, and at most 100 actions, executing the first action and then re-observing in each decision cycle:

MethodGT input SR / SPL ↑Estimated input SR / SPL ↑
NoMaD + geometry65% / 0.634957% / 0.5303
NoMaD + geometry (frozen DINOv3)60% / 0.568755% / 0.4896
Full G²WAM71% / 0.700363% / 0.5923

The closed-loop results provide complementary evidence: compared with the NoMaD control using the same frozen encoder and geometric conditions, G²WAM achieves 11- and 8-percentage-point higher SR under GT and estimated inputs, respectively, with SPL improvements of 0.1316 and 0.1027. G²WAM also outperforms NoMaD with its native encoder by 6 SR points under both inputs.

In summary. In response to the reviewer's questions, the original baselines share the Gibson adaptation data and 200 Mosquito training trajectories, with matched Mosquito update counts and effective batch sizes except for NWM's separately reported schedule. They do not originally receive the same geometric conditions. The added NoMaD controls address this difference, and G²WAM retains better observed trajectory-prediction and closed-loop performance when the visual encoder, geometric inputs, training data, and navigation-training budget are matched. We will report these controls in Appendix C.1 and the complete adaptation conditions in Appendix C.3.

AuthorsOpenReview ↗

Response to Reviewer RHoR (3/8): Q2 / W2

Q2 / W2: Independent contribution of future-state modeling

What is the independent contribution of future-state modeling? Under the protocol that executes only the first action at each cycle, could you compare the full model against an action model without future latent-state supervision, and compare one-step versus eight-step inference in closed-loop navigation? Please distinguish the benefits of multi-step training from those of multi-step inference.

Response. We thank the reviewer for this insightful question. We added an ablation of future visual-latent supervision, a control with a policy that generates actions directly from image representations, and closed-loop evaluations with different training and inference prediction lengths, in order to separate the roles of future-state supervision, training prediction length, inference prediction length, and the number of consecutively executed actions.

The first two sets of closed-loop experiments follow a protocol that executes only the first action in each cycle and then re-observes; they are evaluated on the same 100 Mosquito test episodes with a 2 m success threshold, oracle STOP, and at most 100 actions.

1. Role of future visual-latent supervision and future-state modeling

We use controls at two levels. First, within G²WAM, we remove only the supervision loss on future visual latents while keeping the latent prediction, rollout, and action-generation structure, to test the incremental contribution of this supervision. Second, we use NoMaD + geometry (frozen DINOv3) as a direct-policy control: it uses the same frozen visual encoder and geometric conditions as G²WAM and generates actions through a diffusion policy, without explicitly predicting or rolling out future visual latents.

MethodGT input SR / SPL ↑Estimated input SR / SPL ↑
G²WAM w/o future visual-latent supervision67% / 0.659959% / 0.5800
Full G²WAM71% / 0.700363% / 0.5923

Compared with the variant without future visual-latent supervision, the full model improves SR by 4 points under both input conditions, and SPL by 0.0404 and 0.0123 under GT and estimated inputs, respectively. This supports the incremental value of this supervision.

Compared with NoMaD, which generates actions directly from image representations and geometric conditions (closed-loop results in the table of our response to Q1), the full model improves SR by 11 and 8 points and SPL by 0.1316 and 0.1027 under GT and estimated inputs, respectively. This supports the overall value of joint future-state and action modeling.

2. Separating multi-step training from multi-step inference

We further compare the following settings, all of which execute only the first action:

Model settingGT input SR / SPL ↑Estimated input SR / SPL ↑
Single-step training + single-step inference65% / 0.631060% / 0.4697
Full G²WAM, eight-step training + single-step inference71% / 0.700363% / 0.5923
Full G²WAM, eight-step training + eight-step inference71% / 0.700363% / 0.5923

Regarding the training configuration, with single-step inference, the full model improves SR by 6 and 3 points and SPL by 0.0693 and 0.1226 under GT and estimated inputs, respectively, relative to the single-step-trained model.

Reviewer_LhpkOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes G²WAM, a world-action model for image-goal navigation. A separately trained scene-coordinate localization module (R-SCoRe) anchors the current, goal, and history frames in a shared scene frame; the resulting goal-relative pose and causal motion increments condition a latent world-action model that rolls out autoregressively in frozen DINOv3 patch-latent space (8 steps) and jointly predicts the next latent and canonical waypoint increments (Δx, Δy, Δθ), avoiding both pixel reconstruction and CEM-style planning. Experiments report 40.1%/35.2% ATE/RPE reductions on four trajectory-prediction benchmarks and closed-loop gains on Gibson (SR 54%→63%, SPL 36.4%→59.2%), with much lower latency than world-model baselines.

Critical strengths
  1. Clean efficiency story. Rolling out in latent space without image decoding or CEM gives a real, well-measured speed advantage (Table 5: 4 ms/step vs. 26 ms for DINO-WM and orders of magnitude vs. geometry-aware WMs), and the runtime breakdown is appreciated.
  2. Decoupled, plug-and-play localization interface. Separating pose estimation from the world-action model is a sensible systems decision and makes the approach compatible with future localizers.
  3. Complete empirical package for the chosen setting. Trajectory prediction on four datasets, zero-shot transfer to Go Stanford, closed-loop evaluation in Habitat, conditioning ablations (Table 4), and a GT-pose oracle variant (G²WAM-GT) give a good empirical envelope.
  4. Honest oracle analysis. Including G²WAM-GT and the GT/predicted-pose ablation lets the reader see the headroom, which many papers omit.
Critical weaknesses
  1. The evidence suggests the gains come from the pose context, not from the proposed architecture. Table 4 is the paper's most informative result: with GT pose+motion, ATE drops to 0.46; with predicted pose+motion, 2.02 — a >4× swing driven entirely by localization quality. The obvious missing control is "GT pose + a simple BC/direct policy" (e.g., NoMaD-style) or "GT pose + the same latent transition without the joint decoder/residual machinery." Without it, the paper does not show that the WAM components (transition module, joint latent-action decoder, residual update with α_J/α_B) contribute beyond what pose conditioning alone delivers. As written, the architecture risks being an elaborate wrapper around a localization prior.
  2. The known-scene assumption undercuts the method's relevance and prevents scaling. Each scene requires pre-collected trajectories and a scene-specific localization module. This is a fundamental limitation, not an implementation detail: it rules out exactly the open-world, data-driven regime in which navigation has progressed (GNM/ViNT/NoMaD train one policy across many scenes). The paper should discuss this trade-off explicitly and quantify the per-scene cost (data collection + localizer training) rather than motivating the assumption mainly by convenience.
  3. No scaling or complexity analysis. There is no data-scaling curve, no model-size ablation, and no comparison in which baselines receive comparable adaptation budgets. The model is quite small (frozen DINOv3-384 features, ~25.8k updates, batch 32); some of the efficiency advantage may simply reflect model/task scale rather than the paradigm.
  4. The objective has many unexamined degrees of freedom. Four loss terms (λz=0.3, λa=3.0, λtraj=2.0, λyaw=0.5), cosine+0.3·MSE mixing, turn-step upweighting, residual mixing coefficients, context length H=4, rollout horizon 8 — none are ablated or sensitivity-tested. The turn-upweighting in particular admits a failure mode (over-turning) that is never analyzed.
  5. Missing the closest contemporaneous baselines. Spatial-perceiving WAM, NavWAM, and WAM-Nav are acknowledged but excluded for lack of public code; understandable, but the paper's claims of parity/superiority within the "WAM" class rest on comparisons mostly against world models (which must run CEM) rather than other WAMs. At minimum, the text should soften the within-class claims.
  6. Presentation. "Proposition 1" is an empirical claim with a pointer to experiments, not a proposition; the four-part loss and coordinate conventions would benefit from a single consolidated notation table.
Questions for authors

SEE WEAKNESS

10 comments
AuthorsOpenReview ↗

Response to Reviewer Lhpk (1/8): W1

W1: Gains may come from pose context rather than the architecture

The evidence suggests the gains come from the pose context, not from the proposed architecture. Table 4 is the paper's most informative result: with GT pose+motion, ATE drops to 0.46; with predicted pose+motion, 2.02 — a >4× swing driven entirely by localization quality. The obvious missing control is "GT pose + a simple BC/direct policy" (e.g., NoMaD-style) or "GT pose + the same latent transition without the joint decoder/residual machinery." Without it, the paper does not show that the WAM components (transition module, joint latent-action decoder, residual update with α_J/α_B) contribute beyond what pose conditioning alone delivers. As written, the architecture risks being an elaborate wrapper around a localization prior.

Response. We thank the reviewer for the careful analysis and the valuable suggestions. We agree that localization quality is an important factor and that the original conditioning ablation alone does not isolate the contribution of the model architecture. We first clarify that, in Table 4, the RECON ATE of 2.02 corresponds to no spatial conditioning, while the deployed Pred.+Pred. configuration achieves 1.38; GT+GT achieves 0.46.

To address the architectural question directly, we added controls with matched geometric inputs. First, NoMaD + geometry adds the same goal-relative pose and motion-history conditions to NoMaD. Second, NoMaD + geometry (frozen DINOv3 encoder) additionally replaces NoMaD's visual encoder with the same frozen DINOv3 encoder and image preprocessing used by G²WAM. This reduces the influence of differences in visual representation. Third, G²WAM without joint decoding and residual mixing keeps the same geometric conditions and latent transition, but removes the joint decoding and residual-mixing mechanisms. This last ablation evaluates their combined contribution. For the four-domain trajectory-prediction experiments, all methods use the same training data, 25,800 updates, and a global batch size of 32. For each input condition, the methods receive the same geometric inputs. The results are as follows, with lower values being better:

MethodGT input ATE / RPE ↓Estimated input ATE / RPE ↓
NoMaD + geometry0.9133 / 0.30001.9527 / 0.4832
NoMaD + geometry (frozen DINOv3 encoder)0.9461 / 0.30451.8921 / 0.4700
G²WAM w/o joint decoding & residual mixing0.8065 / 0.28261.8604 / 0.4657
Full G²WAM0.7729 / 0.27661.8226 / 0.4575

Replacing NoMaD's visual encoder with the frozen DINOv3 encoder lowers its ATE and RPE under estimated inputs (from 1.9527/0.4832 to 1.8921/0.4700) but slightly increases them under GT inputs (from 0.9133/0.3000 to 0.9461/0.3045), showing that visual representation is one factor affecting the comparison but does not account for the gap. Regardless of the visual encoder, the full G²WAM achieves lower ATE and RPE than both NoMaD variants under the same geometric inputs. Removing joint decoding and residual mixing increases ATE and RPE under both GT and estimated inputs. Thus, the combined structural design improves trajectory prediction on top of the latent transition and geometric conditions.

The GT-input comparison is particularly informative for the reviewer's question: by supplying ground-truth pose and motion, it removes estimation errors from the spatial-conditioning inputs and allows us to assess the model designs under ideal geometric conditions.

AuthorsOpenReview ↗

Response to Reviewer Lhpk (2/8): W1

W1: Gains may come from pose context rather than the architecture (continued)

First, comparing NoMaD + geometry using the same frozen DINOv3 encoder with the G²WAM variant without joint decoding and residual mixing shows that the transition-based variant reduces ATE from 0.9461 to 0.8065 (14.76%) and RPE from 0.3045 to 0.2826 (7.19%). This variant retains the latent transition and future-state rollout, whereas NoMaD generates actions directly without explicitly predicting future visual states. This comparison therefore supports the value of the transition-based state–action modeling design beyond direct action generation with the same visual encoder and geometric conditions.

Second, adding the joint latent–action decoder and residual updates with αJ\alpha_J and αB\alpha_B further reduces GT-input ATE from 0.8065 to 0.7729 (4.17%) and RPE from 0.2826 to 0.2766 (2.12%). Because this comparison keeps the latent transition and geometric conditions, it more directly measures the combined contribution of joint decoding and residual mixing on top of the transition-based model.

These results distinguish the contribution of geometric information from that of the model design. Moreover, the gains from the model design are larger when the spatial inputs are more accurate: relative to the frozen-DINOv3 NoMaD control, the full G²WAM reduces ATE by 18.31% under GT inputs and by 3.67% under estimated inputs. Crucially, when pose and motion estimation errors are removed from the conditioning inputs, the performance differences persist and the relative improvements are even larger. Thus, the gains cannot be attributed to the availability of geometric inputs alone. As a complementary observation, with all pose and motion conditions removed, G²WAM attains a four-dataset average ATE/RPE of 2.99/0.71, close to NWM (3.04/0.71) and Geometry Forcing (3.13/0.71) in Table 1. This indicates that the model retains a certain trajectory-prediction capability without explicit geometric conditions, and that its performance does not rely entirely on the localization prior.

We also evaluated the same controls in closed-loop navigation on Mosquito. All methods use the same 100 test episodes, a 2 m success threshold, oracle STOP, and a limit of 100 actions. At each decision cycle, the agent executes the first predicted action and then re-observes.

MethodGT input SR / SPL ↑Estimated input SR / SPL ↑
NoMaD + geometry65% / 0.634957% / 0.5303
NoMaD + geometry (frozen DINOv3 encoder)60% / 0.568755% / 0.4896
G²WAM w/o joint decoding & residual mixing65% / 0.649860% / 0.5921
Full G²WAM71% / 0.700363% / 0.5923

As shown in the table, the same conclusions are consistently corroborated by the closed-loop results: the full G²WAM outperforms both NoMaD controls under GT and estimated inputs (by 11 and 8 SR points over the frozen-DINOv3 variant, respectively), while removing the joint latent–action decoder and residual updates reduces SR under both inputs. The gains are clear under both GT and estimated spatial inputs.

AuthorsOpenReview ↗

Response to Reviewer Lhpk (3/8): W2

W2: Known-scene assumption and per-scene cost

The known-scene assumption undercuts the method's relevance and prevents scaling. Each scene requires pre-collected trajectories and a scene-specific localization module. This is a fundamental limitation, not an implementation detail: it rules out exactly the open-world, data-driven regime in which navigation has progressed (GNM/ViNT/NoMaD train one policy across many scenes). The paper should discuss this trade-off explicitly and quantify the per-scene cost (data collection + localizer training) rather than motivating the assumption mainly by convenience.

Response. We thank the reviewer for highlighting the scope of the known-scene setting. We agree that the spatial-context construction in G²WAM requires a scene to be known in advance and therefore introduces a scene-specific preparation cost. However, the purpose of our study is precisely to investigate how a navigation agent can make better use of pre-collected data in a known environment. To clarify, in the 8-step trajectory prediction experiments (Table 1), the results for G²WAM and the baselines follow the commonly used target-scene adaptation protocol: all methods receive the same small amount of data from the test scene for training. Our observation is that fine-tuning a policy on these trajectories alone does not explicitly construct or exploit a metric spatial context for the scene. This motivated us to study a complementary, lightweight, and low-cost paradigm that uses the same limited pre-collected data to construct spatial context and then integrates that context with visual information for navigation beyond the current field of view. G²WAM does not use any additional trajectories, images, or supervision beyond the data available to the baselines; the data and adaptation budget are kept identical, while the data are used differently to provide explicit spatial context.

The spatial-context component is applicable when the environment can be surveyed in advance, but this does not mean that G²WAM only functions in known spaces. When spatial context is unavailable, the model can operate without this input. The zero-shot results in Table 2 evaluate exactly this setting and show that G²WAM remains competitive with the strongest evaluated baselines without scene-specific spatial context. Thus, our claim is not that the method removes the cost of known-scene preparation, but that it provides an effective way to exploit such preparation when it is available, while retaining generalizable navigation capability when it is not.

We also thank the reviewer for raising the question of per-scene cost, which is a critical issue. In fact, when designing the current framework, we paid particular attention to the cost of collecting and training the spatial context for each scene and kept it within a reasonable range. We measured the computational cost of preparing the localization module and fine-tuning the navigation model separately on GoStanford. This experiment represents adaptation to a new target scene and is distinct from the zero-shot evaluation in Table 2. Localization preparation uses 10,000 training images, with retrieval features also extracted for 497 query images. Node2Vec and R-SCoRe are trained from scratch for 2,000 and 10,000 updates, respectively. The same finetuned R-SCoRe is used in Table 2 for G²WAM-PoseFT. All stages run sequentially on a single H800:

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.