- Rating
- 3
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper argues that zero-shot ObjectNav agents often commit to a goal without checking whether their evidence is sufficient, and rarely re-evaluate places they have already visited. It proposes RefBeliefMap, a 2D belief map that tracks target relevance together with spatial uncertainty (how well a region has been seen) and semantic uncertainty (how ambiguous the visual-language evidence is). When nearby candidates are ambiguous, the agent either moves to a better viewpoint or asks an MLLM to reassess the evidence. RefBeliefNav then ranks frontiers and historical waypoints under one objective of target relevance, information gain and path cost. The method reports 69.2/35.3 SR/SPL on HM3D and 49.1/18.5 on MP3D, with ablations and a real-robot demo.
Critical strengths
- The motivation is clear. Checking whether the evidence can separate competing candidates before committing to one is a sensible idea, and splitting "not seen well" from "seen but ambiguous" gives each case a natural remedy.
- Treating revisitation as a regular decision, rather than a separate stage after exploration, is a nice design choice. The frontier-only ablation (64.6 vs. 69.5 SR) suggests it helps.
- The adaptive trigger is efficient. With 8.2 LLM calls per episode it outperforms fixed-interval triggering with 37.2 calls, which supports the claim that when to reason matters more than how often.
- The main results and ablations are consistent. The method improves over the strongest compared baseline on both datasets, each module contributes in the ablations, and the sensitivity analyses are extensive.
Critical weaknesses
- The paper's central claim is that spatial and semantic uncertainty identify evidence that is insufficient or likely wrong, but this is never checked directly. In particular, the prompt set is designed to vary context cues, so part of the prompt variance may not reflect real ambiguity about the target. I think a simple analysis using ground-truth object locations, e.g., whether high-uncertainty regions are more often mis-ranked before refinement, would make the "uncertainty-guided" claim much more convincing.
- It is not clear whether the semantic-reasoning gain comes from the MLLM judging which evidence is unreliable, or simply from lowering confidence whenever the trigger fires. The fact that GPT-4o, Gemini and Qwen give nearly identical results is compatible with either explanation. An ablation that replaces the MLLM with a constant or rule-based reliability weight, while keeping the same trigger points, would separate the two.
- Several closely related methods cited in the paper, such as CogNav, WMNav, ApexNav and UniGoal, are not included in Table 1, and some report comparable or higher SR (e.g., CogNav reports 72.5 SR on HM3D). In addition, the detection and stopping logic is not described, although it strongly affects SR on HM3D. I think either adding these comparisons or re-running key baselines with identical detection settings is needed.
- The ablations show that the modules help end-to-end, but not how they change behavior. The paper's story is "detect insufficient evidence, gather more, revise the decision, recover the target," yet there is no statistic on how often refinement changes the chosen waypoint or how often a revisit leads to finding the target. One qualitative example showing this full chain would also help a lot.
- Many individual components have been explored in prior work: belief maps with travel cost (BeliefMapNav), active evidence acquisition (AECNav), and revisiting mapped locations (SSMG-Nav). The main new element is routing spatial and semantic uncertainty to different remedies within one objective, but no ablation tests this routing itself, e.g., running both remedies whenever either uncertainty is high.
Questions for authors
- Spatial uncertainty is computed over a short window of recent observations, so a region not seen recently becomes as spatially uncertain as an unseen one. How much of the revisitation benefit comes from re-evaluation based on new evidence, rather than from old regions quickly becoming "uncertain" again?
- What exactly is an "observation region" for the BLIP-2 score: the whole frame or segmented parts? How is the single spatial support center chosen from depth and camera pose?
- I noticed that five settings affecting different components (σ_a = 1.5, δ = 0.05, τ_rel = 0.20, d_max = 4.0, τ_sp = 0.40) all report exactly 69.3 SR / 35.0 SPL, and that the default value is best in every sweep for both metrics, often by only 0.1–0.2 points on 1,000 episodes. Since each of these settings changes the agent's behavior, an exact match in both metrics across five runs is surprising, and margins this small are within sampling noise. Could you provide the configuration files and per-episode results for these runs and the default setting?
- Removing the information-gain term drops SR by 11.3 points but SPL by only 1.7. Could you explain this, for example with a breakdown of failure modes?
- What fraction of selected waypoints are historical, and in what fraction of successful episodes was the target first found after a revisit?
9 comments
Response to Weakness 3. Thank you for highlighting these closely related methods and for asking us to clarify the evaluation protocol, target detection, and stopping policy. We have expanded the comparison to include CogNav, WMNav, ApexNav, and UniGoal, and provide the relevant implementation and evaluation details below.
Regarding CogNav's higher reported SR on HM3D, we inspected its released evaluation implementation and identified a discrepancy relevant to this comparison. Although its task configuration specifies SUCCESS_DISTANCE: 0.1, the custom get_metrics() function in the environment wrapper assigns success when distance_to_goal is below . This differs from the success criterion used in our experiments. Such a difference can affect both SR and SPL, motivating a comparison under a shared evaluation protocol rather than a direct comparison of numbers obtained under different settings.
Our evaluation follows the Habitat Challenge 2022 ObjectNav configuration and uses the official Habitat evaluator. An episode is successful only if the agent explicitly issues STOP within the -step interaction budget and the minimum geodesic distance from its stopping position to the episode's annotated goal-viewpoint set is less than . Here, the distance threshold applies to the annotated goal viewpoints, rather than to the physical center of the target object.
For visual perception, we use Grounding-DINO for target detection. Our policy issues STOP when the Euclidean distance between the agent's current position and the estimated center of the detected target is below .
Table R6. Comparison with closely related methods on HM3D and MP3D. All baselines are rerun under the same evaluation configuration as our method.
| Method | HM3D SR (%) ↑ | HM3D SPL (%) ↑ | MP3D SR (%) ↑ | MP3D SPL (%) ↑ |
|---|---|---|---|---|
| UniGoal [4] | 53.8 | 24.7 | 42.3 | 16.5 |
| ApexNav [5] | 62.6 | 32.8 | 39.6 | 18.1 |
| WMNav [6] | 59.4 | 29.2 | 44.1 | 16.2 |
| CogNav [7] | 61.5 | 25.4 | 45.6 | 15.9 |
| RefBeliefNav (Ours) | 69.2 | 35.3 | 49.1 | 18.5 |
As shown in Table R6, RefBeliefNav achieves the highest SR and SPL on both HM3D and MP3D under the same evaluation configuration. On HM3D, our method achieves an SR of 69.2% and an SPL of 35.3%, outperforming ApexNav, the strongest baseline on both metrics, by 6.6 and 2.5 percentage points, respectively. In particular, compared with CogNav under the aligned evaluation settings, RefBeliefNav improves SR by 7.7 percentage points and SPL by 9.9 percentage points. On MP3D, our method achieves an SR of 49.1%, exceeding the best baseline, CogNav, by 3.5 percentage points. It also achieves the highest SPL of 18.5%, surpassing ApexNav by 0.4 percentage points. Overall, these results demonstrate consistent improvements across both datasets under a common evaluation configuration, with larger gains on HM3D and a more modest SPL improvement on MP3D.
Response to Weakness 4. Thank you for asking for a clearer account of how evidence refinement and revisitation affect the agent’s behavior. Figure 4 in Appendix A.7 of our manuscript illustrates how evidence reassessment changes waypoint selection and leads the agent back to a previously overlooked area. In the maps of Figure 4, orange, red, and dark blue hollow circles denote historical waypoints, frontiers, and the selected waypoint, respectively. The light blue circle indicates the radius- neighborhood centered on the robot, within which refinement triggers are evaluated. The target object in this episode is a toilet.
-
(a) After initialization, the agent identifies a doorway and selects a frontier near it.
-
(b) The agent considers the area around this frontier to have been sufficiently observed and selects a more distant waypoint, thereby missing the opportunity to enter the bathroom.
-
(c) Upon reaching the distant frontier, the agent discovers a kitchen area.
-
(d) The surrounding region exhibits both high semantic belief values and high uncertainty, triggering semantic refinement. After reassessing the available evidence, the MLLM reduces the semantic belief values in this region.
-
(e) The agent then selects a historical waypoint to revisit an area where the target may have been overlooked.
-
(f) After turning around, the agent identifies and selects a frontier near the bathroom entrance.
-
(g) The agent subsequently approaches the toilet and successfully locates the target.
Although the agent initially misses the bathroom entrance and proceeds toward the kitchen, where BLIP-2 produces overly high semantic scores, semantic refinement corrects the resulting misleading beliefs and enables the agent to reconsider its exploration direction. During subsequent waypoint ranking, a historical waypoint is selected, guiding the agent back to a location from which it can rediscover the bathroom entrance and ultimately find the toilet.
Response to Reviewer 3
Response to Weakness 1. Thank you for asking for direct validation of the connection between uncertainty and incorrect or insufficient evidence. We agree that this connection requires empirical assessment. We also acknowledge that disagreement across target-conditioned prompts can reflect sensitivity to the prompt set, so its interpretation as evidence uncertainty requires care.
We conducted a diagnostic analysis of 3,960 samples collected at refinement-triggering instants from 300 navigation episodes. Before refinement, we compared spatial belief scores with viewing-quality assessments and semantic belief scores with ground-truth target/non-target labels. Under the consistency criteria specified in Tables R3 and R4, spatial and semantic inconsistencies account for 35.0% and 42.5% of the respective evaluated samples.
These results document evidence problems at the selected triggering instants. Because this analysis includes only triggered samples, it does not by itself establish that higher uncertainty predicts a higher error rate. The sampling procedure, annotation criteria, and detailed distributions are provided in our response to Reviewer 2, Weakness 1 and Question 1, together with Tables R3 and R4 in that response.
Response to Weakness 2. Thank you for suggesting a controlled comparison to distinguish the contribution of MLLM-based reliability assessment from the effect of uniform downweighting. Table R5 compares evidence-dependent reliability weights with fixed weights while retaining the same triggering rule and thresholds. This comparison examines whether assigning weights according to the observed evidence provides an additional benefit.
Specifically, we introduce three additional variants with fixed reliability weights γ=0.3, 0.5, and 0.7, respectively. Whenever semantic refinement is triggered, each variant assigns its fixed weight to all regions selected for semantic reassessment, instead of using the evidence-dependent weights predicted by the MLLM.
Table R5. Ablation of semantic reliability weighting on the HM3D ablation subset with 1,000 episodes. All variants retain the same triggering rule and thresholds. Constant-weight variants replace the MLLM-predicted reliability weights with fixed values.
As shown in Table R5, replacing the MLLM-based assessment with constant reliability weights still provides benefits over disabling semantic refinement. All three constant-weight variants improve SR, while γ=0.3 and γ=0.5 also improve SPL. In particular, γ=0.3 increases SR from 65.2% to 67.8% and SPL from 33.9% to 34.8%, corresponding to gains of 2.6 and 0.9 percentage points, respectively. These results are consistent with our explanation in the response to Weakness 1 above: spatial and semantic uncertainty provide useful signals for identifying situations where the available evidence may be insufficient or unreliable, and reducing the influence of such evidence can benefit belief fusion even without MLLM-based reliability assessment. However, uniform downweighting is not consistently beneficial across both metrics, as γ=0.7 slightly reduces SPL to 33.6%.
Nevertheless, all three constant-weight variants underperform MLLM-based weighting in both SR and SPL. The full method achieves 69.5% SR and 35.5% SPL, exceeding the strongest constant-weight baseline (γ=0.3) by 1.7 and 0.7 percentage points, respectively. Thus, none of the tested fixed weights fully recovers the performance of MLLM-based assessment. These results support the additional benefit of assigning reliability weights according to the specific evidence, beyond the improvement provided by uniform confidence reduction after triggering.