acceptodds
Under review as a conference paper at ICLR 2027

Have You Seen It Clearly? Uncertainty-Guided Belief Refinement for Zero-Shot Object Navigation

Abstract

Zero-Shot Object Navigation (ZSON) requires an agent to locate target objects in unfamiliar environments without additional training for the test categories or environments. Recent approaches leverage large language models and vision-language models to construct spatial representations and guide exploration. However, whether the available evidence sufficiently supports a navigation choice, and how new evidence changes the search value of previously visited locations, remain insufficiently addressed. We propose RefBeliefMap, an uncertainty-guided mapping framework that distinguishes target-location belief from spatial and semantic evidence uncertainty. By jointly modeling intra-observation uncertainty and inter-observation disagreement over a sliding memory window, RefBeliefMap assesses the reliability of both evidence channels. When competing frontiers remain insufficiently resolved, these uncertainty estimates guide active observation and semantic reasoning to refine the evidence supporting target-location beliefs before navigation commitment. Building on the refined map, we introduce RefBeliefNav, which jointly evaluates unexplored frontiers and historical waypoints according to their target-finding potential, expected information gain, and navigation cost. This unified objective allows historical locations to be reconsidered as new evidence changes their search value, coordinating exploration and revisitation. Experiments on HM3D and MP3D demonstrate state-of-the-art performance among the evaluated zero-shot baselines, with ablation studies examining the contributions of modules. Real-world experiments on a Unitree Go2 quadruped robot further demonstrate the sim-to-real transferability of our method.

Primary area:
applications to robotics, autonomy, planning
Keywords:
Zero-Shot Object Navigation, Uncertainty-Aware Navigation, Belief Evolution, Revisit-Aware Planning

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_qCSdOpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper argues that zero-shot ObjectNav agents often commit to a goal without checking whether their evidence is sufficient, and rarely re-evaluate places they have already visited. It proposes RefBeliefMap, a 2D belief map that tracks target relevance together with spatial uncertainty (how well a region has been seen) and semantic uncertainty (how ambiguous the visual-language evidence is). When nearby candidates are ambiguous, the agent either moves to a better viewpoint or asks an MLLM to reassess the evidence. RefBeliefNav then ranks frontiers and historical waypoints under one objective of target relevance, information gain and path cost. The method reports 69.2/35.3 SR/SPL on HM3D and 49.1/18.5 on MP3D, with ablations and a real-robot demo.

Critical strengths
  • The motivation is clear. Checking whether the evidence can separate competing candidates before committing to one is a sensible idea, and splitting "not seen well" from "seen but ambiguous" gives each case a natural remedy.
  • Treating revisitation as a regular decision, rather than a separate stage after exploration, is a nice design choice. The frontier-only ablation (64.6 vs. 69.5 SR) suggests it helps.
  • The adaptive trigger is efficient. With 8.2 LLM calls per episode it outperforms fixed-interval triggering with 37.2 calls, which supports the claim that when to reason matters more than how often.
  • The main results and ablations are consistent. The method improves over the strongest compared baseline on both datasets, each module contributes in the ablations, and the sensitivity analyses are extensive.
Critical weaknesses
  • The paper's central claim is that spatial and semantic uncertainty identify evidence that is insufficient or likely wrong, but this is never checked directly. In particular, the prompt set is designed to vary context cues, so part of the prompt variance may not reflect real ambiguity about the target. I think a simple analysis using ground-truth object locations, e.g., whether high-uncertainty regions are more often mis-ranked before refinement, would make the "uncertainty-guided" claim much more convincing.
  • It is not clear whether the semantic-reasoning gain comes from the MLLM judging which evidence is unreliable, or simply from lowering confidence whenever the trigger fires. The fact that GPT-4o, Gemini and Qwen give nearly identical results is compatible with either explanation. An ablation that replaces the MLLM with a constant or rule-based reliability weight, while keeping the same trigger points, would separate the two.
  • Several closely related methods cited in the paper, such as CogNav, WMNav, ApexNav and UniGoal, are not included in Table 1, and some report comparable or higher SR (e.g., CogNav reports 72.5 SR on HM3D). In addition, the detection and stopping logic is not described, although it strongly affects SR on HM3D. I think either adding these comparisons or re-running key baselines with identical detection settings is needed.
  • The ablations show that the modules help end-to-end, but not how they change behavior. The paper's story is "detect insufficient evidence, gather more, revise the decision, recover the target," yet there is no statistic on how often refinement changes the chosen waypoint or how often a revisit leads to finding the target. One qualitative example showing this full chain would also help a lot.
  • Many individual components have been explored in prior work: belief maps with travel cost (BeliefMapNav), active evidence acquisition (AECNav), and revisiting mapped locations (SSMG-Nav). The main new element is routing spatial and semantic uncertainty to different remedies within one objective, but no ablation tests this routing itself, e.g., running both remedies whenever either uncertainty is high.
Questions for authors
  • Spatial uncertainty is computed over a short window of recent observations, so a region not seen recently becomes as spatially uncertain as an unseen one. How much of the revisitation benefit comes from re-evaluation based on new evidence, rather than from old regions quickly becoming "uncertain" again?
  • What exactly is an "observation region" for the BLIP-2 score: the whole frame or segmented parts? How is the single spatial support center chosen from depth and camera pose?
  • I noticed that five settings affecting different components (σ_a = 1.5, δ = 0.05, τ_rel = 0.20, d_max = 4.0, τ_sp = 0.40) all report exactly 69.3 SR / 35.0 SPL, and that the default value is best in every sweep for both metrics, often by only 0.1–0.2 points on 1,000 episodes. Since each of these settings changes the agent's behavior, an exact match in both metrics across five runs is surprising, and margins this small are within sampling noise. Could you provide the configuration files and per-episode results for these runs and the default setting?
  • Removing the information-gain term drops SR by 11.3 points but SPL by only 1.7. Could you explain this, for example with a breakdown of failure modes?
  • What fraction of selected waypoints are historical, and in what fraction of successful episodes was the target first found after a revisit?
9 comments

Response to Reviewer 3

Response to Weakness 1. Thank you for asking for direct validation of the connection between uncertainty and incorrect or insufficient evidence. We agree that this connection requires empirical assessment. We also acknowledge that disagreement across target-conditioned prompts can reflect sensitivity to the prompt set, so its interpretation as evidence uncertainty requires care.

We conducted a diagnostic analysis of 3,960 samples collected at refinement-triggering instants from 300 navigation episodes. Before refinement, we compared spatial belief scores with viewing-quality assessments and semantic belief scores with ground-truth target/non-target labels. Under the consistency criteria specified in Tables R3 and R4, spatial and semantic inconsistencies account for 35.0% and 42.5% of the respective evaluated samples.

These results document evidence problems at the selected triggering instants. Because this analysis includes only triggered samples, it does not by itself establish that higher uncertainty predicts a higher error rate. The sampling procedure, annotation criteria, and detailed distributions are provided in our response to Reviewer 2, Weakness 1 and Question 1, together with Tables R3 and R4 in that response.

Response to Weakness 2. Thank you for suggesting a controlled comparison to distinguish the contribution of MLLM-based reliability assessment from the effect of uniform downweighting. Table R5 compares evidence-dependent reliability weights with fixed weights while retaining the same triggering rule and thresholds. This comparison examines whether assigning weights according to the observed evidence provides an additional benefit.

Specifically, we introduce three additional variants with fixed reliability weights γ=0.3\gamma = 0.3, 0.50.5, and 0.70.7, respectively. Whenever semantic refinement is triggered, each variant assigns its fixed weight to all regions selected for semantic reassessment, instead of using the evidence-dependent weights predicted by the MLLM.

Table R5. Ablation of semantic reliability weighting on the HM3D ablation subset with 1,000 episodes. All variants retain the same triggering rule and thresholds. Constant-weight variants replace the MLLM-predicted reliability weights with fixed values.

MethodReliability weight γ\gammaSR (%) ↑SPL (%) ↑
w/o semantic refinement—65.233.9
Constant weight0.30.367.834.8
Constant weight0.50.566.234.1
Constant weight0.70.765.933.6
MLLM-based weightingEvidence-dependent69.535.5

As shown in Table R5, replacing the MLLM-based assessment with constant reliability weights still provides benefits over disabling semantic refinement. All three constant-weight variants improve SR, while γ=0.3\gamma=0.3 and γ=0.5\gamma=0.5 also improve SPL. In particular, γ=0.3\gamma=0.3 increases SR from 65.2% to 67.8% and SPL from 33.9% to 34.8%, corresponding to gains of 2.6 and 0.9 percentage points, respectively. These results are consistent with our explanation in the response to Weakness 1 above: spatial and semantic uncertainty provide useful signals for identifying situations where the available evidence may be insufficient or unreliable, and reducing the influence of such evidence can benefit belief fusion even without MLLM-based reliability assessment. However, uniform downweighting is not consistently beneficial across both metrics, as γ=0.7\gamma=0.7 slightly reduces SPL to 33.6%.

Nevertheless, all three constant-weight variants underperform MLLM-based weighting in both SR and SPL. The full method achieves 69.5% SR and 35.5% SPL, exceeding the strongest constant-weight baseline (γ=0.3\gamma=0.3) by 1.7 and 0.7 percentage points, respectively. Thus, none of the tested fixed weights fully recovers the performance of MLLM-based assessment. These results support the additional benefit of assigning reliability weights according to the specific evidence, beyond the improvement provided by uniform confidence reduction after triggering.

Response to Weakness 3. Thank you for highlighting these closely related methods and for asking us to clarify the evaluation protocol, target detection, and stopping policy. We have expanded the comparison to include CogNav, WMNav, ApexNav, and UniGoal, and provide the relevant implementation and evaluation details below.

Regarding CogNav's higher reported SR on HM3D, we inspected its released evaluation implementation and identified a discrepancy relevant to this comparison. Although its task configuration specifies SUCCESS_DISTANCE: 0.1, the custom get_metrics() function in the environment wrapper assigns success when distance_to_goal is below 1.0,m1.0\\,\mathrm{m}. This differs from the success criterion used in our experiments. Such a difference can affect both SR and SPL, motivating a comparison under a shared evaluation protocol rather than a direct comparison of numbers obtained under different settings.

Our evaluation follows the Habitat Challenge 2022 ObjectNav configuration and uses the official Habitat evaluator. An episode is successful only if the agent explicitly issues STOP within the 500500-step interaction budget and the minimum geodesic distance from its stopping position to the episode's annotated goal-viewpoint set is less than 0.1,m0.1\\,\mathrm{m}. Here, the distance threshold applies to the annotated goal viewpoints, rather than to the physical center of the target object.

For visual perception, we use Grounding-DINO for target detection. Our policy issues STOP when the Euclidean distance between the agent's current position and the estimated center of the detected target is below 0.1,m0.1\\,\mathrm{m}.

Table R6. Comparison with closely related methods on HM3D and MP3D. All baselines are rerun under the same evaluation configuration as our method.

MethodHM3D SR (%) ↑HM3D SPL (%) ↑MP3D SR (%) ↑MP3D SPL (%) ↑
UniGoal [4]53.824.742.316.5
ApexNav [5]62.632.839.618.1
WMNav [6]59.429.244.116.2
CogNav [7]61.525.445.615.9
RefBeliefNav (Ours)69.235.349.118.5

As shown in Table R6, RefBeliefNav achieves the highest SR and SPL on both HM3D and MP3D under the same evaluation configuration. On HM3D, our method achieves an SR of 69.2% and an SPL of 35.3%, outperforming ApexNav, the strongest baseline on both metrics, by 6.6 and 2.5 percentage points, respectively. In particular, compared with CogNav under the aligned evaluation settings, RefBeliefNav improves SR by 7.7 percentage points and SPL by 9.9 percentage points. On MP3D, our method achieves an SR of 49.1%, exceeding the best baseline, CogNav, by 3.5 percentage points. It also achieves the highest SPL of 18.5%, surpassing ApexNav by 0.4 percentage points. Overall, these results demonstrate consistent improvements across both datasets under a common evaluation configuration, with larger gains on HM3D and a more modest SPL improvement on MP3D.

Response to Weakness 4. Thank you for asking for a clearer account of how evidence refinement and revisitation affect the agent’s behavior. Figure 4 in Appendix A.7 of our manuscript illustrates how evidence reassessment changes waypoint selection and leads the agent back to a previously overlooked area. In the maps of Figure 4, orange, red, and dark blue hollow circles denote historical waypoints, frontiers, and the selected waypoint, respectively. The light blue circle indicates the radius-RR neighborhood centered on the robot, within which refinement triggers are evaluated. The target object in this episode is a toilet.

  • (a) After initialization, the agent identifies a doorway and selects a frontier near it.

  • (b) The agent considers the area around this frontier to have been sufficiently observed and selects a more distant waypoint, thereby missing the opportunity to enter the bathroom.

  • (c) Upon reaching the distant frontier, the agent discovers a kitchen area.

  • (d) The surrounding region exhibits both high semantic belief values and high uncertainty, triggering semantic refinement. After reassessing the available evidence, the MLLM reduces the semantic belief values in this region.

  • (e) The agent then selects a historical waypoint to revisit an area where the target may have been overlooked.

  • (f) After turning around, the agent identifies and selects a frontier near the bathroom entrance.

  • (g) The agent subsequently approaches the toilet and successfully locates the target.

Although the agent initially misses the bathroom entrance and proceeds toward the kitchen, where BLIP-2 produces overly high semantic scores, semantic refinement corrects the resulting misleading beliefs and enables the agent to reconsider its exploration direction. During subsequent waypoint ranking, a historical waypoint is selected, guiding the agent back to a location from which it can rediscover the bathroom entrance and ultimately find the toilet.

Reviewer_sk2aOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes RefBeliefNav, which separates target-location belief from spatial and semantic evidence uncertainty. It uses uncertainty to trigger active observation or semantic reassessment, then chooses among unexplored frontiers and historical waypoints. It reports higher SR and SPL than the evaluated baselines on HM3D and MP3D, and includes module ablations and qualitative Unitree Go2 demonstrations.

Critical strengths
  1. The paper does not demonstrate that its uncertainty estimates are calibrated. Specifically, semantic uncertainty is derived from variations in BLIP-2 scores across different prompts, while semantic refinement leverages heuristic reliability scores obtained from a multimodal large language model. The paper fails to establish that these metrics effectively reflect actual evidential errors.
  2. The target belief fusion process assumes that observations are conditionally independent. However, multiple observations of the same region may be correlated; consequently, even if discrepancies among recent observations are tracked, the resulting beliefs may still exhibit overconfidence.
Critical weaknesses
  1. The uncertainty estimates are not shown to be calibrated. In particular, semantic uncertainty is based on variation across prompt-conditioned BLIP-2 scores, while semantic refinement uses heuristic reliability ratings from a multimodal LLM. The paper does not establish that these quantities track actual evidence errors.
  2. Target belief fusion assumes conditionally independent observations. Repeated views of the same region may be correlated, so the belief could become overconfident even when recent-observation disagreement is tracked separately.
Questions for authors
  1. Do you have calibration or reliability results showing that the proposed spatial and semantic uncertainty measures predict when the corresponding evidence is wrong or insufficient?
  2. How sensitive is target-belief performance to the conditional-independence assumption for repeated observations of the same region?
3 comments

Response to Reviewer 2

Response to Weakness 1 and Question 1. Thank you for asking for direct evidence linking uncertainty-guided refinement to the quality of the underlying observations. To investigate this concern, we conducted a diagnostic analysis of spatial and semantic evidence at refinement-triggering instants. The analysis below characterizes the evidence available before refinement and its agreement with independently assessed viewing quality and ground-truth target labels. We randomly sampled 300 navigation episodes and evaluated all refinement-triggering instants within these episodes, yielding a total of 3,960 samples. We evaluated the corresponding region observations before refinement using ground-truth object locations and category labels, together with manual verification. For both spatial and semantic belief scores, we defined low, intermediate, and high ranges as [0,0.4][0,0.4], (0.4,0.6)(0.4,0.6), and [0.6,1.0][0.6,1.0], respectively. The intermediate score range (0.4,0.6)(0.4,0.6) serves as a decision dead zone for consistency assessment: observations in this range are treated as indeterminate and are labeled neither consistent nor inconsistent.

For spatial evidence, we compared spatial belief scores with the actual viewing quality of the corresponding regions. Viewing quality was assessed using the distance between the camera center and the region center, the viewing angle, and two rounds of manual verification. As shown in Table R3, 10.8% of the evaluated spatial observations had high spatial belief despite poor viewing quality, whereas 24.2% had low spatial belief despite good viewing quality. These two types of spatial evidence inconsistency accounted for 10.810.8\\%+24.2\\%=35.0\\% of all evaluated spatial observations.

Table R3. Spatial evidence at refinement-triggering instants. Bold entries denote spatial evidence inconsistency: high belief with poor viewing quality or low belief with good viewing quality.

Spatial belief scoreGood viewing qualityPoor viewing quality
High [0.6,1.0][0.6,1.0]15%10.8%
Intermediate (0.4,0.6)(0.4,0.6)18.4%9.6%
Low [0,0.4][0,0.4]24.2%22.0%

Total inconsistency: 35.0%.

For semantic evidence, we compared semantic belief scores with the ground-truth target/non-target labels. As shown in Table R4, 32.4% of the evaluated semantic observations assigned high scores to non-target regions, whereas 10.1% assigned low scores to target regions. These semantic evidence inconsistencies accounted for 32.432.4\\%+10.1\\%=42.5\\% of all evaluated semantic observations. These results document concrete spatial and semantic evidence inconsistencies at the selected refinement-triggering instants and support the motivation for evidence refinement.

Table R4. Semantic evidence at refinement-triggering instants. Bold entries denote semantic evidence inconsistency: high belief for a non-target region or low belief for a target region.

Semantic belief scoreTarget regionNon-target region
High [0.6,1.0][0.6,1.0]25.4%32.4%
Intermediate (0.4,0.6)(0.4,0.6)12.2%8.0%
Low [0,0.4][0,0.4]10.1%11.9%

Total inconsistency: 42.5%.

Response to Weakness 2 and Question 2. Thank you for raising the concern that correlated observations may lead to overconfident target beliefs. We agree that tracking disagreement across recent observations does not, by itself, remove their statistical dependence. Eq. (3) in our manuscript uses a conditional-independence approximation, and residual correlation can cause posterior uncertainty to be underestimated.

To assess sensitivity to repeated observation contributions, we compare the original fusion with viewpoint sparsification and redundancy downweighting. As reported in Table R2 in [our response to Reviewer 1, Weakness 1][reviewer-1], these variants increase the median posterior variance by 9%–32% relative to the original fusion. In separate online evaluations, they reduce SR by 2.0–4.0 percentage points and SPL by 1.0–3.1 percentage points. These results quantify the sensitivity of the fused beliefs and indicate that retaining observation contributions benefits navigation in the evaluated setting. They do not establish conditional independence or calibrated posterior uncertainty.

The experimental protocol, fusion equations, and complete results are provided in our response to Reviewer 1, Weakness 1, including Eqs. (R1)–(R10) and Table R2.

Thank you again for your careful feedback. We will incorporate the clarifications and supporting analyses presented above into the revised manuscript.

Thank you for your careful review and constructive comments. We would appreciate hearing whether the clarifications and supporting results in our response have addressed the issues you raised. We would be happy to provide any additional information or clarification you may need.

Thank you for the reply. I am willing to keep my score. One suggestion unrelated to the rebuttal is that the images in the paper should use vector graphics as much as possible. At present they look closer to bitmaps in clarity. The text is also not properly embedded. The quality of the method figure in the current submission is not good enough. I hope it can be polished as necessary later. Of course this is not required at this stage. It is only a suggestion.

Reviewer_8mcuOpenReview ↗
Rating
2(+1)
Confidence
2
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

This paper presents RefBeliefNav, which is a zero-shot object navigation framework. It utilizes RefBeliefMap, an uncertainty-guided belief map. Since other VLMs often hallucinate objects when viewing them from far away, RefBeliefNav weights VLM detections by vieweing distance and angles to generate the detection uncertainty. It also inspects ambiguous objects before setting them as the final goal. They evaluate the framework on the HM3D and MP3D benchmarks. They applied this to Unitree Go2 quadruped robot to show sim-to-real transferability of the method.

Critical strengths

Straight forward and intuitive idea: Allow the VLM detection to have uncertainty which it confirms by moving to middle waypoints to confirm before committing to the final goal. This can improve path efficiency and addresses a major failure mode in ZSON

Critical weaknesses

Possibly inaccurate mathematical models

Invalid conditional independence assumption on consecutive video frames (for Eq 3)

  • This formula requires observations to be conditionally independent given the latent state. In reality, as the robot moves 0.25 m per step looking at the exact same object with the exact same frozen BLIP-2 model and fixed 8 prompts, consecutive errors are nearly 100% correlated

What is σsem,t2σ^2_{sem,t} actually measuring? (linguistic variance not same as visual uncertainty)

Questions for authors

In Table 3 why does removing navigation cost make score worse? Including cost should probably constrain the performance?

7 comments

Response to Reviewer 1

Response to Weakness 1. Thank you for raising this concern about the conditional-independence approximation in Eq. (3) in the manuscript. We agree that consecutive observations may share visual content and model errors, and that treating them as independent can overcount evidence and underestimate uncertainty. Conditional independence in Eq. (3) in the manuscript is therefore a modeling approximation. To examine its practical implications, we analyze viewpoint-dependent changes in semantic scores and evaluate the sensitivity of belief fusion and navigation performance to observation sparsification and downweighting. Previous studies have shown that vision-language models (VLMs) can produce semantically inconsistent descriptions of the same object when presented with images captured from different viewpoints [1], [3], [2]. To this end, we have added experiments showing that changes in position and viewpoint lead to changes in semantic scores. The magnitude of these changes is reported in Table R1. Specifically, the results represent the mean percentage change in semantic scores for the same region before and after each action, measured across 1,000 randomly sampled episodes from HM3D with eight fixed prompts. These measurements show that the outputs are not invariant to the tested movements and viewpoint changes.

Table R1. Mean semantic-score change (%) between matched observations of the same region before and after an action, with BLIP-2 and the eight prompts fixed.

ActionP1P2P3P4P5P6P7P8Mean
MOVE_FORWARD32.321.228.522.526.734.633.124.527.9
TURN_LEFT36.529.432.832.929.834.338.127.732.7
TURN_RIGHT32.138.327.529.939.234.327.332.432.6
LOOK_UP27.427.634.731.327.738.229.730.530.9
LOOK_DOWN35.324.831.625.233.830.735.228.830.7

To further assess the conditional-independence assumption for repeated observations of the same region, additional sensitivity analyses of observation sparsification and downweighting are needed.

We keep the visual model, prompt set, and other components unchanged, and vary only each observation's contribution to target-belief fusion. Introduce a weight βt(q)∈[0,1]\beta_t(q)\in[0,1] into Eq. (3) in the manuscript, yielding Eqs. (R1) and (R2):

\tag{R1}$$ $$\mu_{\mathrm{tar},t}(q) = \sigma_{\mathrm{tar},t}^{2}(q) \left[ \frac{\mu_{\mathrm{tar},t-1}(q)} {\sigma_{\mathrm{tar},t-1}^{2}(q)} + \beta_t(q) \frac{ B_{\mathrm{sp},t}(q)\mu_{\mathrm{sem},t} }{ \widetilde{\sigma}_{\mathrm{sem},t}^{2}(q) } \right]. \tag{R2}$$ Here, $\widetilde{\sigma}_{\mathrm{sem},t}^{2}(q)$ is the semantic variance actually used for fusion, including the reliability adjustment in Eq. (11) in the manuscript when applicable. Setting $\beta_t(q)=1$ recovers the original update, $\beta_t(q)=0$ omits this observation's contribution, and $0<\beta_t(q)<1$ reduces its contribution. The same weight is applied to the precision and mean updates. For each cell $q$, compare the current camera pose with a reference pose maintained separately for that cell. Define the translation and rotation differences in Eq. (R3): $$d_t(q) = \left|x_t-x_{\mathrm{ref}}(q)\right|^2, \qquad \theta_t(q) = \operatorname{angle} \left(r_t, r_{\mathrm{ref}}(q)\right), \tag{R3}$$ where $x_t$ and $r_t$ denote camera position and orientation, respectively, and $\operatorname{angle}$ is the smallest relative rotation angle. The viewpoint-novelty indicator is defined in Eq. (R4): $$z_t(q;d_0,\theta_0) = \begin{cases} 1, & d_t(q)\ge d_0 \ \text{or}\ \theta_t(q)\ge\theta_0, \\ 0, & \text{otherwise}. \end{cases} \tag{R4}$$ The first valid observation initializes the reference pose and is assigned $z_t(q)=1$. For each subsequent valid observation, $z_t(q)$ is evaluated against the existing reference pose, which is updated to the current camera pose only if $z_t(q)=1$. Observations with $z_t(q)=0$ do not reset the reference pose, whether they are sparsified or downweighted. For *viewpoint sparsification*, set the weight as in Eq. (R5): $$\beta_t^{\mathrm{sparse}}(q) = z_t(q;d_0,\theta_0). \tag{R5}$$ For *redundancy downweighting*, set the weight as in Eq. (R6): $$\beta_t^{\mathrm{down}}(q) = z_t(q;d_0,\theta_0) + \eta \left[ 1-z_t(q;d_0,\theta_0) \right], \qquad 0<\eta<1. \tag{R6}$$ We test two sparsification thresholds $(d_1,\theta_1)$ and $(d_2,\theta_2)$, with $d_2\ge d_1$ and $\theta_2\ge\theta_1$ and at least one strict inequality. For downweighting, we fix the thresholds to $(d_1,\theta_1)$ and compare $0<\eta_2<\eta_1<1$. For belief analysis, we replay identical observation sequences with the same per-frame spatial and semantic evidence, changing only the fusion weights. Let $\mathcal{O}$ contain the valid cell–observation pairs, and let $\mathcal{E}$ contain matched cell–time pairs at which beliefs are evaluated. For variant $j$, define the observation precision in Eq. (R7), the relative cumulative observation precision in Eq. (R8), the change in belief means in Eq. (R9), and the posterior variance ratio in Eq. (R10): $$\pi_t(q) = \frac{ B_{\mathrm{sp},t}(q) }{ \widetilde{\sigma}_{\mathrm{sem},t}^{2}(q) }, \tag{R7}$$ $$C_j = 100\, \frac{ \displaystyle \sum_{(q,t)\in\mathcal{O}} \beta_t^{(j)}(q)\pi_t(q) }{ \displaystyle \sum_{(q,t)\in\mathcal{O}} \pi_t(q) }, \tag{R8}$$ $$\Delta_{\mu,j} = \frac{1}{|\mathcal{E}|} \sum_{(q,t)\in\mathcal{E}} \left| \mu_{\mathrm{tar},t}^{(j)}(q) - \mu_{\mathrm{tar},t}^{(0)}(q) \right|, \tag{R9}$$ $$R_{\sigma^2,j} = \underset{(q,t)\in\mathcal{E}}{\operatorname{median}} \frac{\bigl(\sigma_{\mathrm{tar},t}^{(j)}(q)\bigr)^2}{\bigl(\sigma_{\mathrm{tar},t}^{(0)}(q)\bigr)^2}. \tag{R10}$$ The superscript $(0)$ denotes original fusion. Thus, $C_j$ measures the relative cumulative observation precision, $\Delta_{\mu,j}$ measures the change in belief means, and $R_{\sigma^2,j}$ measures the change in posterior variances. Separately, we run each variant online on the same 1,000 HM3D navigation trials used for the ablations in Section 4.3 of the manuscript and report SR and SPL in percent.

Table R2. Sensitivity to potentially redundant observations. Belief statistics use matched replayed observations; SR and SPL use separate online runs on the same evaluation episodes. The baseline values 100100, 00, and 11 follow by definition.

Setting(d0,θ0)(d_0,\theta_0)Similar-view weightCC (%)Δμ\Delta_\muRσ2R_{\sigma^2}SR (%) ↑\uparrowSPL (%) ↑\uparrow
Original---11000169.535.5
Sparse S1(d1,θ1)(d_1,\theta_1)038.60.241.2366.332.8
Sparse S2(d2,θ2)(d_2,\theta_2)028.20.271.3265.532.4
Downweight D1(d1,θ1)(d_1,\theta_1)η1\eta_169.30.091.0967.534.5
Downweight D2(d1,θ1)(d_1,\theta_1)η2\eta_254.00.141.1567.234.1

Parameter values: d1=0.35 md_1=0.35\,\mathrm{m}, θ1=30∘\theta_1=30^{\circ}; d2=0.45 md_2=0.45\,\mathrm{m}, θ2=45∘\theta_2=45^{\circ}; η1=0.5\eta_1=0.5, η2=0.25\eta_2=0.25.

For belief analysis, we use recorded trajectory episodes from the existing set of 1,000 HM3D evaluation episodes used for the ablations in Section 4.3 of the manuscript. Exactly the same episodes are replayed for all variants. As shown in Table R2, S1 and S2 retain 38.6% and 28.2% of the original cumulative observation precision, respectively, corresponding to reductions of 61.4% and 71.8%. D1 and D2 retain 69.3% and 54.0%, respectively. Across the four variants, the mean absolute changes in belief means range from 0.09 to 0.27, and the median posterior variance ratios range from 1.09 to 1.32. The latter correspond to median relative variance increases of 9%–32%. At the shared thresholds (d1,θ1)(d_1,\theta_1), downweighting produces smaller changes in belief means and smaller variance increases than sparsification. These results quantify the sensitivity of the fused beliefs to the contributions of potentially redundant observations.

In the separate online evaluation, the original method achieves 69.5% SR and 35.5% SPL. The four variants achieve 65.5%–67.5% SR and 32.4%–34.5% SPL, corresponding to decreases of 2.0–4.0 percentage points in SR and 1.0–3.1 percentage points in SPL. Both downweighting variants achieve higher SR and SPL than S1 at the same viewpoint thresholds, while all four variants perform below the original method. The observed changes indicate that reducing the contributions of potentially redundant observations affects both the fused beliefs and downstream navigation performance.

Taken together, the literature reports and the experimental evidence support our online fusion rule as a reasonable modeling approximation for the evaluated navigation setting. We considered the possibility of correlated consecutive observations before selecting this update, and adopted it after reviewing the relevant literature and examining its empirical behavior.

Response to Weakness 2. Thank you for highlighting the distinction between prompt-induced score variation and uncertainty in visual evidence. We agree that our original wording did not make this distinction sufficiently clear, and that variation across prompts cannot be directly equated with visual uncertainty. As defined in Section 3.3.1 of the manuscript, σsem,t2\sigma_{\mathrm{sem},t}^{2} measures the dispersion of BLIP-2 relevance scores across different target-conditioned descriptions for a fixed image region. It directly reflects the sensitivity of target-relevance scores to the prompt set.

In our framework, we assess uncertainty in visual observation evidence from two complementary perspectives: spatial support and semantic consistency. Specifically, σsem,t2\sigma_{\mathrm{sem},t}^{2} quantifies semantic score disagreement within an observation and, together with semantic disagreement across observations, constitutes semantic evidence uncertainty. Spatial evidence uncertainty separately characterizes insufficient visible support.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.