- Rating
- 2
- Confidence
- 2
- Informativeness
- 1
- Presentation
- 3
- Soundness
- 1
Read review
Paper summary
This paper examines goal grounding as a core bottleneck in object goal navigation using vision-language models. It focuses on the difficulty of identifying the target object and deciding when to stop navigating. To address this, the authors propose the StopWatch framework. The framework consists of two models that separate object perception from navigation policy learning. A dedicated perception model predicts structured grounding information, including visible object categories and their distances to the goal, and passes it to a separately trained VLM policy model for action prediction. Experiments on six navigation benchmarks show that the method outperforms existing approaches. Ablation studies and failure case analyses further highlight the importance of explicit grounding supervision and accurate object perception in improving navigation success rates.
Critical strengths
None
Critical weaknesses
The methodological novelty is limited relative to existing modular ObjectNav approaches. Systems such as PONI and VLFM have already explored explicit goal recognition, spatial localization, and stopping decisions in depth. The proposed method largely reorganizes these established components by expressing perception outputs as structured text and passing them to a separately trained VLM policy. The combination is effective, but the paper does not convincingly establish a fundamentally new mechanism for grounding or navigation learning.
Questions for authors
The methodological novelty is limited relative to existing modular ObjectNav approaches. Systems such as PONI and VLFM have already explored explicit goal recognition, spatial localization, and stopping decisions in depth. The proposed method largely reorganizes these established components by expressing perception outputs as structured text and passing them to a separately trained VLM policy. The combination is effective, but the paper does not convincingly establish a fundamentally new mechanism for grounding or navigation learning.
The diagnosis is overgeneralized. The paper presents goal grounding as the central bottleneck in object goal navigation, yet recognizing target objects and determining their locations are already core components of the ObjectNav problem, and prior work has studied related difficulties. Moreover, the experiments focus on VLM policies trained with behavior cloning. They mainly expose the limitations of these particular policies. They do not show that goal grounding is the dominant challenge, nor that it applies to ObjectNav methods more broadly.
The performance gains over existing modular systems are not sufficiently attributed. StopWatch substantially outperforms VLFM, but VLFM already includes explicit target detection, spatial localization, and a stopping mechanism. The improvement may therefore come from stronger perception models, additional supervised training, the learned navigation policy, or some combination of these, rather than from the proposed grounding formulation itself.
The need for the complete grounding representation is not well justified. In Table 2, full grounding improves success by 6.1 percentage points over the baseline without grounding, but by only 0.7 points over the best single-field variant. This points to considerable overlap between semantic and distance grounding, particularly in signaling whether the goal is visible and whether the agent has arrived. A simpler predictor of goal visibility or an auxiliary STOP classifier might recover much of the gain.
The paper provides no controlled comparisons supporting the text interface. The paper does not adequately explain why converting explicit perception information into structured text is preferable to conventional representations such as semantic maps, masks, or spatial features. In addition, the advantage of the two-model architecture does not,, by itself,, show that the text interface drives the improvement.
5 comments
Q4. Full grounding improves little over the best single field, so a simpler visibility predictor or an auxiliary STOP classifier might suffice.
- We corrected the ablation in the revision (Table 2). The semantic field alone raises success from 63.6% to 68.7%, and full grounding reaches 69.6%.
- The small margin of the distance field reflects prediction accuracy, not redundancy. Substituting the ground-truth distance whenever both the predicted and the annotated inventory contain the goal raises success from 69.6% to 78.6% (Fig. 7a). Wrong distance also causes 26% of the remaining failures, against 43% for wrong visibility (Section 4.4).
| Clean HM3D-OVON val_unseen | Success (%) |
|---|---|
| Semantics only | 68.7 |
| Semantics and distance (ours) | 69.6 |
| Ours with ground-truth distance | 78.6 |
- A perfect goal-visibility predictor falls short of the object list. Due to the time limit, we ran a smaller comparison than Table 2. On 256 randomly sampled val_unseen episodes, a policy that reads a ground-truth goal-visibility flag reaches 48.8%, and one that reads the predicted list of objects in view reaches 65.2%. Even ground-truth visibility thus helps the policy less than the object list, so a simpler visibility predictor would not suffice. We will add the complete ablation to Table 2 in the camera-ready version.
Q5. No controlled comparison supports the text interface over semantic maps, masks, or spatial features.
- Semantic patch tokens perform about as well as text, so we chose text as the simpler interface. We passed the semantics in view to the policy as semantic patch tokens, pooled from the semantic masks over image patches, which keeps their spatial layout. Success was similar to that of the category list, but each frame then needs one token per patch instead of a few tokens per category (Appendix F).
- Text follows how VLMs and VLAs already reason. They generate reasoning text before acting (Wei et al., 2022; Zawalski et al., 2025), and the text interface gives the policy grounding in the form it already reads, without a new encoder (Section 3.3).
- We do not claim that text is the best interface, and it is not among our main claims. Our claims concern goal grounding, as stated at the top. Text is a simple interface that carries the semantics in view and the distance to the goal, and it suffices to test these claims.
Kind Reminder on the Author-Reviewer Discussion
As the author-reviewer discussion period ends soon, we would be grateful if you could look at our response and the revised PDF, and let us know whether they address your concerns. If you have any follow-up questions, please post them soon so that we can reply before the discussion closes.
Thank you again for your time and your review.
We thank the reviewer for the detailed questions. We would like to first point out a misunderstanding of our core contribution. The review assesses the paper as a new mechanism for grounding or navigation, but the paper does not claim one. Our contribution is a finding about why current VLM navigation policies fail, together with a solution that resolves most of this failure. Its three claims are below.
W1 / Q1. The methodological novelty is limited relative to modular methods such as PONI and VLFM.
Q2. The diagnosis is overgeneralized, since goal recognition is a known part of ObjectNav and the experiments cover only VLM policies trained by behavior cloning.
Q3. The gain over VLFM is not attributed to the grounding formulation rather than stronger perception, more supervision, or the learned policy.