- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
The paper proposes G²WAM for image-goal navigation, combining the poses of observation and goal images with scene coordinates, and predicting future images and actions in latent space to avoid explicit future-image reconstruction and iterative CEM-based planning. The experiments cover four trajectory prediction datasets, zero-shot transfer to a new dataset, and closed-loop navigation. The results demonstrate competitive performance in average trajectory prediction errors and Gibson navigation.
Critical strengths
1.The method uses a localization module to anchor the current observation, goal image, and historical observations to a shared scene coordinate system. It then extracts goal-relative poses and historical motion information, providing additional geometric context. The ablation results also demonstrate the benefits of incorporating this spatial information.
Critical weaknesses
1.Using latent image representations to provide auxiliary supervision is not a novel idea in itself. However, the ablation studies do not specifically investigate this component, leaving the contribution of latent image representations unclear. In particular, the paper does not clarify how this design compares with approaches that directly use image representations. I therefore consider the methodological novelty of this aspect insufficiently established.
Questions for authors
None.
7 comments
Response to Reviewer r28Q (2/2): W1
W1: Novelty and contribution of latent image supervision (continued)
| Method | GT input ATE / RPE ↓ | Estimated input ATE / RPE ↓ |
|---|---|---|
| NoMaD + geometry | 0.9133 / 0.3000 | 1.9527 / 0.4832 |
| NoMaD + geometry (frozen DINOv3) | 0.9461 / 0.3045 | 1.8921 / 0.4700 |
| G²WAM w/o joint decoding & residual mixing | 0.8065 / 0.2826 | 1.8604 / 0.4657 |
| G²WAM w/o visual-latent supervision | 0.7988 / 0.2763 | 1.8800 / 0.4657 |
| Full G²WAM | 0.7729 / 0.2766 | 1.8226 / 0.4575 |
Compared with NoMaD, which uses image representations directly, the full G²WAM achieves lower ATE/RPE than both NoMaD variants, including when NoMaD uses the same frozen DINOv3 encoder and receives the geometric conditions: relative to this control, the ATE decreases from 0.9461 to 0.7729 under GT inputs and from 1.8921 to 1.8226 under estimated inputs. This supports the overall value of jointly modeling future states and actions.
As for visual-latent supervision itself, removing this loss increases the GT-input ATE from 0.7729 to 0.7988 and the estimated-input ATE/RPE from 1.8226/0.4575 to 1.8800/0.4657, while the GT-input RPE slightly decreases from 0.2766 to 0.2763. The benefit of this supervision therefore lies mainly in ATE and in the estimated-input setting, rather than in a uniform improvement across all metrics.
Keeping the latent transition while removing joint decoding and residual mixing increases both ATE and RPE under both input conditions, supporting the combined contribution of these structural designs.
3. Closed-loop navigation
We further evaluated the same 100 episodes on Mosquito. All methods use a 2 m success threshold, oracle STOP, and at most 100 actions, and execute only the first action before re-observing in each decision cycle:
| Method | GT input SR / SPL ↑ | Estimated input SR / SPL ↑ |
|---|---|---|
| NoMaD + geometry | 65% / 0.6349 | 57% / 0.5303 |
| NoMaD + geometry (frozen DINOv3) | 60% / 0.5687 | 55% / 0.4896 |
| G²WAM w/o joint decoding & residual mixing | 65% / 0.6498 | 60% / 0.5921 |
| G²WAM w/o visual-latent supervision | 67% / 0.6599 | 59% / 0.5800 |
| Full G²WAM | 71% / 0.7003 | 63% / 0.5923 |
Compared with NoMaD, which uses the same frozen DINOv3 representations and geometric conditions, the full G²WAM improves SR by 11 and 8 points under GT and estimated inputs and SPL by 0.1316 and 0.1027, respectively; it also outperforms NoMaD with its native encoder by 6 SR points under both inputs, providing a closed-loop comparison between the two modeling approaches.
Compared with the variant without visual-latent supervision, the full model improves SR by 4 points under both input conditions and SPL by 0.0404 and 0.0123, respectively. This supports the incremental value of this auxiliary supervision.
Compared with the variant without joint decoding and residual mixing, the full model improves SR by 6 points and SPL by 0.0505 under GT inputs; under estimated inputs, SR improves by 3 points while SPL is essentially unchanged. The gains of these structural designs in the estimated-input closed-loop evaluation are therefore mainly reflected in the success rate.
In summary, we do not present future visual supervision itself as a novelty; our contribution is joint state–action modeling conditioned on spatial context. The controls show that, even compared with a direct policy that uses the same visual representation and geometric conditions, this design achieves better results in both trajectory prediction and closed-loop navigation, and future visual supervision, as an auxiliary objective, brings further gains on top of it.
We hope that the clarifications above and the additional experiments in the revised manuscript have addressed your concerns. If you find that these additions resolve the issues raised in your review, we would be grateful if you would consider updating your score accordingly. As the rebuttal deadline is approaching, please feel free to let us know if you have any further questions or concerns, so that we can address them in a timely manner during the remaining discussion period.
Response to Reviewer r28Q (1/2): W1
W1: Novelty and contribution of latent image supervision
Response. We sincerely thank the reviewer for the valuable comment. We agree that using future visual representations as auxiliary supervision is not a new idea in itself, and we do not present it as a standalone contribution of this paper. We have revised the paper to avoid presenting future visual-latent supervision itself as a novel methodological component. Our contribution lies in jointly predicting future latent states and executable actions conditioned on spatial context extracted from images, and in generating action sequences directly through state recursion. Future visual-latent supervision is an auxiliary objective for training this structure.
To address the two questions raised by the reviewer, we added the corresponding controls: first, a comparison with NoMaD, which generates actions directly from image representations; second, an ablation within G²WAM that removes future visual-latent supervision to test its incremental effect. In addition, we added a structural ablation of joint decoding and residual mixing to separate the contribution of the auxiliary supervision from that of the model structure.
1. Comparison with a method that directly uses image representations
In this experiment, NoMaD serves as the control method that generates actions directly from image representations. It encodes the current and past observations and the goal image into visual features and, conditioned on them, generates future waypoints through diffusion denoising, without explicitly predicting or rolling out future visual latents. G²WAM additionally predicts future visual latents jointly with actions and uses the predicted states for the subsequent rollout.
To examine whether the difference arises only from the visual encoder or the geometric information, we set up two NoMaD controls:
The second setting lets the two methods share the visual encoder and geometric conditions, allowing a more direct comparison between using image representations for action generation and additionally modeling future states jointly with actions.
2. Ablations of visual-latent supervision and the joint structure
Within G²WAM, we set up the following variants:
Four-dataset average trajectory prediction results are as follows (ATE/RPE, lower is better):