- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper argues that zero-shot ObjectNav agents often fail even after a plausible target has been detected: they oscillate between exploration and pursuit, abandon the target near success, or stop without verification. It proposes ConsistNav, a training-free control layer between a fixed perception stack and planner, with a persistent candidate memory, a seven-state controller, and stability-aware action control. The paper also introduces an Action Consistency Score (ACS), and reports 84.2/41.2 SR/SPL on HM3Dv2 and 50.6/25.7 on MP3D, plus ablations and real-robot experiments.
Critical strengths
-
The motivation is clear. Most zero-shot ObjectNav work focuses on better semantic evidence, while this paper points out that keeping the target and stopping at the right time is a separate problem.
-
The improvement on MP3D is large. On this noisier benchmark with more categories, SR improves by 11.4 points and timeout failures drop by about 10 points.
Critical weaknesses
- The "Non-executive baseline" seems to be ApexNav itself, but the paper does not state which components are reused or changed. After subtracting the reported gains, the baseline numbers exactly match ApexNav's results on all three benchmarks. So the method is only tested on top of one existing system, and testing it on another backbone such as VLFM would strengthen the claim of a general executive layer.
- The technical novelty is limited. The method is basically a finite-state machine, a hand-designed candidate score, and stall detection, with dozens of manually specified thresholds and no sensitivity analysis. CogNav already models ObjectNav as a state machine with search, target observation, and candidate verification states, and ApexNav already fuses target evidence across frames, but the paper does not discuss how its design differs from them.
- The ablation shows that the components work together, but it does not separate the effect of semantic commitment from generic anti-stalling fixes. FSEC alone slightly lowers performance, and there is no SAAC-only variant, so it is unclear how much of the gain comes from SAAC, which mainly fixes spinning and stalling. The ablation is also only done on HM3Dv2, not on MP3D where the main gain appears.
- Fig. 4 does not show that the targeted failure mode is reduced. Unstable Commitment barely changes on HM3Dv2 (11.80% to 10.90%) and slightly increases on MP3D (19.90% to 20.59%), while most of the MP3D gain comes from fewer timeouts and missed targets. This makes it unclear how much of the improvement comes from better action consistency.
- ACS is not yet convincing as a contribution. The persistence and release events have no operational definition, and the 12-action window for valid stopping is shorter than the 14-action verification turn budget, which may partly explain the low valid-stop rate of the FSEC variants; a sensitivity analysis on the window would help. Also, FSEC-only gets a higher ACS than PCM-only but has the lowest SR, mostly because it almost never stops at invalid opportunities (111/112), so ACS may over-reward not stopping.
- The 150-episode ACS experiment is hard to interpret. SR there is only 9–19% across variants, far below the 76–84% reported on HM3Dv2, and it is not explained how these episodes and the 22 scenes were selected. If they were drawn mainly from baseline failures, the comparison may be biased.
Questions for authors
- Are the exploration-related parameters, such as the room-prior frontier bonus and the frontier shortlist, also used in the Non-executive baseline?
- For each ablation variant, how do the failure categories in Fig. 4 change? For example, does FSEC mainly reduce Unstable Commitment?
- Could you add a per-episode timeline showing detection confidence, PCM evidence, and FSEC state, so that readers can see the consistency problem and how the controller fixes it?
- Why are recent training-free methods such as CogNav, UniGoal, and WMNav not included in Table 1?
- The paper says the Non-executive baseline is run on your own stack (Habitat-Sim v0.3.1, YOLO-World), but its six SR/SPL numbers on the three benchmarks are exactly the same as those reported in the ApexNav paper. Getting identical numbers with a different simulator and detector is very unlikely. Did you re-run the baseline, or take these numbers from the ApexNav paper?
- In Fig. 4, the baseline's Unstable Commitment rate on MP3D is 19.90%, which cannot be obtained from 2,195 episodes (436/2,195 = 19.86% and 437/2,195 = 19.91%). All other numbers in the figure can. How was this number computed?
4 comments
Response to Reviewer tbAp (2/4): Failure Analysis, ACS Interpretation, and Evaluation Scope (W4–W6)
W4. Failure analysis from 500 additional runs
1.We evaluated five configurations on the same 100 prespecified HM3Dv2 tasks across 36 scenes: 500 runs, each with a 500-action budget. The baseline was rerun. Counts are mutually exclusive; percentages use each row's failures.
| Configuration | Failures | Incorrect STOP¹ | No frontier | Stuck, gave up | Step limit |
|---|---|---|---|---|---|
| Baseline | 56 | 20 (35.7%) | 6 (10.7%) | 2 (3.6%) | 28 (50.0%) |
| PCM | 65 | 39 (60.0%) | 8 (12.3%) | 3 (4.6%) | 15 (23.1%) |
| FSEC | 83 | 15 (18.1%) | 1 (1.2%) | 0 (0%) | 67 (80.7%) |
| PCM+FSEC | 77 | 19 (24.7%) | 6 (7.8%) | 1 (1.3%) | 51 (66.2%) |
| ConsistNav | 69 | 24 (34.8%) | 5 (7.2%) | 3 (4.3%) | 37 (53.6%) |
¹ The agent issued STOP to claim success, but the environment recorded failure.
PCM+FSEC reduces incorrect STOPs relative to PCM. The count falls from 39 to 19: 20 fewer per 100 tasks. In paired comparisons, 21 tasks avoid PCM's incorrect STOP and only one changes in the opposite direction. This directly supports fewer false success claims.
Full reduces timeouts and completes more tasks than PCM+FSEC. With SAAC present, step-limit failures fall from 51 to 37 and successes rise from 23 to 31 (+8). Ten PCM+FSEC timeouts become Full successes, while two successes become failures. The paired outcomes confirm more tasks completed.
These results support the practical value of action-consistent execution. PCM+FSEC reduces incorrect STOPs relative to PCM, while Full reduces timeouts and completes more tasks relative to PCM+FSEC. Together, these gains support coordinating verified stopping with execution progress.
W5. Event definitions, timing and ACS interpretation
1. How persistence and release are judged. Both concern an instance the agent has already pursued. Persistence tests whether pursuit of the same supported instance continues within the response window. Changing the approach waypoint can preserve that commitment; a pass must also satisfy the evaluator's progress criterion. Release tests whether pursuit ends after clear counterevidence against that instance. A single missed detection or unfinished verification scan is insufficient counterevidence. Appendix C.1 will specify the actual evidence criteria, cross-frame instance association, response windows and event deduplication. Event denominators reflect the opportunities encountered by each trajectory.
2. Does the 12-action window underestimate valid stopping? The 12-action window starts when a valid-stop opportunity appears; the 14-action maximum verification-turn budget starts upon entering verification. Their onsets can differ. A valid opportunity during scanning followed by a correct STOP on action 13 or 14 can therefore fail the current window. We plan to hold trajectories and event onsets fixed and rescore at 12, 14 and 16 actions. For each variant, we will report passed/eligible valid-stop events and how many previously failed events subsequently ended in a voluntary STOP validated as successful by Habitat. This will quantify the window's effect on FSEC results.
3. How to interpret FSEC-only's higher ACS. It mainly reflects stronger invalid-stop avoidance: 111/112, versus 5/20 timely valid stops. PCM-only achieves 6/85 and 17/18, respectively. The original formula weights the two termination rates equally; 112 opportunities do not receive extra weight merely because they are more numerous.
W6. Purpose and interpretation of the 150-episode comparison
Thank you for asking about the scope and purpose of this experiment.
1. Shared episodes and the SR gap. The four configurations were evaluated on the same 150 episodes across 22 scenes. Baseline completed 20 episodes and Full ConsistNav completed 29. Both SRs are substantially below the main HM3Dv2 evaluation; this comparison alone does not identify the cause of the gap. We plan two additional random draws of 150 episodes using different seeds and a common prespecified sampling procedure. For each draw, we will report ACS, its components, SR and SPL to test whether the observations recur across samples.
2. Examining ACS on shared episodes. The experiment asks whether ACS captures decision changes introduced by the executive and whether those changes accompany better navigation outcomes. From Baseline to Full, ACS rises from 0.5292 to 0.7493. For example, when counterevidence calls for releasing a candidate, the passed/eligible release count rises from 31/45 to 51/51 (Table C.3). On the same episode set, SR rises from 13.33% to 19.33%, and SPL from 6.01% to 9.43% (Table 4). In this comparison, Full completes more tasks and improves performance accounting for path length. ACS adds a behavioral account of these accompanying changes, including more reliable release of disconfirmed candidates.
Response to Reviewer tbAp (3/4): Exploration Settings, Failure Analysis, and Episode Timeline(Q1-Q3)
Thank you for asking how exploration, component effects and recorded behavior support our claims.
Q1. Were the exploration settings shared?
The two systems use different exploration rules. We report ApexNav's published SR and SPL for the baseline. We did not rerun ApexNav with our room-prior bonus or our 12-frontier shortlist.
-
ApexNav. Room context enters image-text matching. With strong semantic cues, the planner selects a variable-sized frontier set by relative semantic score and plans a tour; otherwise it chooses the nearest reachable frontier. This differs from a separate room-prior bonus and a top-12 path-check shortlist. Room prompt · Frontier selection
-
ConsistNav. When the target-room prior is known and no provisional, active or sufficiently supported candidate is present, frontiers receive a prior-dependent bonus of up to 22.0. The bonus is disabled when such a candidate exists. Ranking combines information gain relative to distance, semantic value, this bonus, and revisit/failure penalties. Only the top 12 receive detailed path checks. These settings can change explored locations and encountered targets. Table B.2 specifies our settings; Table 1 therefore compares complete systems. The component comparisons in Q2 use variants within the same experiments.
Q2. How do failures change across ablations?
We ran all five configurations on the same 100 prespecified HM3Dv2 tasks (36 scenes, 500 runs). The baseline here was run on these tasks. The table counts how failed episodes ended. Parentheses give the percentage of failures in that row. With 100 tasks per row, each count also gives the percentage of all tasks.
| Configuration | Failures | Incorrect STOP¹ | No frontier | Stuck, gave up | Step limit |
|---|---|---|---|---|---|
| Baseline | 56 | 20 (35.7%) | 6 (10.7%) | 2 (3.6%) | 28 (50.0%) |
| PCM | 65 | 39 (60.0%) | 8 (12.3%) | 3 (4.6%) | 15 (23.1%) |
| FSEC | 83 | 15 (18.1%) | 1 (1.2%) | 0 (0%) | 67 (80.7%) |
| PCM+FSEC | 77 | 19 (24.7%) | 6 (7.8%) | 1 (1.3%) | 51 (66.2%) |
| ConsistNav | 69 | 24 (34.8%) | 5 (7.2%) | 3 (4.3%) | 37 (53.6%) |
¹ The agent issued STOP to claim success and the environment recorded failure.
FSEC and incorrect STOPs. On 21 of the 100 tasks, PCM issued an incorrect STOP that PCM+FSEC avoided. Six of those runs succeeded with PCM+FSEC and 15 reached the step limit. Only one task changed in the opposite direction, giving 20 fewer incorrect STOPs overall. Total successes fell from 35 to 23 on this subset. FSEC checks target evidence before authorizing STOP, and the agent made fewer false claims of success with this check in place.
PCM and SAAC. Adding PCM to FSEC raised successes from 17 to 23 and reduced step-limit endings from 67 to 51. With SAAC added to PCM+FSEC, 10 tasks that previously reached the limit succeeded. Two previously successful tasks failed, giving eight more successes overall (23 to 31). Incorrect STOPs rose from 19 to 24. With PCM retaining candidate evidence across views, more tasks succeeded within the step budget, and adding SAAC then turned 10 previous timeouts into successes.
Evidence for better commitment retention is less conclusive. We observed release of a supported target without contrary evidence in 0 PCM and 4 PCM+FSEC failures, with 24 and 33 failures unassessable, respectively.
Q3. Can you show an episode timeline?
We provide a successful logged run from HM3Dv2 scene 4ok3usBNeis: sofa, source episode 5 (runtime ID 10), negative-update coefficient 0.12. The table uses controller indices; confidence is PCM's internal score.
| Index | Recorded event for candidate 2 |
|---|---|
| 124 | SEARCH to SUSPECT to APPROACH; candidate selected |
| 125–126 | Confidence increases: 0.60 to 0.80 to 1.00 |
| 156 | APPROACH to VERIFY |
| 157–170 | Same candidate retained through 14 verification actions |
| 170 | Verification confirmed; VERIFY to SUCCESS |
| 171 | STOP issued (action 0) |
The environment log independently records execution of STOP at environment step 135, success=1, distance to goal 0.02958 m, and SPL 0.62716. Controller indices and environment steps are different counters.
The trace connects accumulated candidate evidence, sustained approach, bounded verification and successful stopping. Candidate 2 remains selected throughout approach and verification; the controller confirms verification before issuing STOP, and the environment confirms task success. This provides an observable link from the internal decision sequence to the navigation outcome.
Response to Reviewer tbAp (4/4): Method Comparisons, Baseline Sources, and Percentage Calculation(Q4-Q6)
Q4. Why were CogNav, UniGoal and WMNav missing from Table 1?
CogNav uses LLM-driven cognitive states, UniGoal graph reasoning for multiple goal types, and WMNav a VLM-based world model. Their reasoning, memory and action selection make them relevant comparisons.
The updated table below adds all three and SysNav (2026). It reports published ObjectNav SR/SPL in percent. Bold marks each column's highest value, underlining its second highest, and a dash an unreported result. Y/N mean Yes/No. Each cited row is a literature reference for the complete system.
| Method | Unsupervised | Zero-shot | HM3Dv2 SR | SPL | HM3Dv1 SR | SPL | MP3D SR | SPL |
|---|---|---|---|---|---|---|---|---|
| PONI (2022) | N | N | — | — | — | — | 31.8 | 12.1 |
| OpenFMNav (2024) | Y | Y | — | — | 54.9 | 24.4 | 37.2 | 15.7 |
| InstructNav (2024) | Y | Y | 58.0 | 20.9 | — | — | — | — |
| VLFM (2024) | Y | Y | 63.6 | 32.5 | 52.5 | 30.4 | 36.4 | 17.5 |
| TriHelper (2024) | Y | Y | — | — | 56.5 | 25.3 | — | — |
| SG-Nav (2024) | Y | Y | 49.6 | 25.5 | 54.0 | 24.9 | 40.2 | 16.0 |
| ApexNav (2025) | Y | Y | 76.2 | 59.6 | 39.2 | 17.8 | ||
| CogNav (ICCV 2025) | Y | Y | — | — | 72.5 | 26.2 | 46.6 | 16.1 |
| UniGoal (CVPR 2025) | Y | Y | — | — | 54.5 | 25.1 | 41.0 | 16.4 |
| WMNav (IROS 2025) | Y | Y | 72.2 | 33.3 | 58.1 | 31.2 | 45.4 | 17.2 |
| SysNav (RA-L 2026) | Y | Y | 37.2 | 30.5 | 50.7 | |||
| ConsistNav | Y | Y | 84.2 | 41.2 | 63.2 | 34.8 | 25.7 |
Q5. Were the six baseline numbers rerun or quoted?
The six values were directly quoted from ApexNav. They are 76.2/38.0 on HM3Dv2, 59.6/33.0 on HM3Dv1 and 39.2/17.8 on MP3D. This explains the numerical identity.
Table 1 already cites ApexNav and identifies prior rows as literature-reported results. However, Sec. 4.2 describes the corresponding differences as a controlled same-stack comparison. Table 2 repeats 76.20/38.00 under "Non-executive baseline" with the same-stack caption. Those descriptions are inconsistent with the actual source of the numbers.
Habitat-Sim v0.3.1 and YOLO-World/GroundingDINO describe our implementation. The quoted ApexNav row follows its source evaluation. We will label Table 2's cited row "ApexNav (reported)", specify the scope of our implementation settings, and correct the abstract, Secs. 4.2/4.4 and conclusion. Comparisons with the quoted row measure overall system performance and cannot isolate executive control.
There is also positive evidence from the new runs. Across the four added VLFM variants, tasks and settings are shared. With SAAC disabled, adding FSEC to PCM raises SR/SPL from 38.2/18.7 to 42.2/22.8 on 2,195 MP3D tasks. This supports the component's contribution in that configuration. We will ground component claims in these within-experiment comparisons and explicitly identify the provenance of every baseline row.
Q6. How was 19.90% obtained?
We took 19.90% directly from ApexNav Fig. 7, where it labels category C (False Positive) on MP3D. You are correct that no integer count out of 2,195 episodes rounds to 19.90% at two decimal places. ApexNav, Fig. 7
1. Inferring the count. The other five categories have published percentages of 39.23%, 13.85%, 4.92%, 14.67% and 7.43%. On a denominator of 2,195 episodes, these imply counts of 861, 304, 108, 322 and 163, respectively. If the six categories partition all episodes, the count left for C is
2. Checking the displayed value. Subtracting the five rounded percentages from 100% reproduces the figure label.
100% − (39.23% + 13.85% + 4.92% + 14.67% + 7.43%) = 19.90%
Using the inferred episode count directly gives
× 100% = 19.9088838…%
The two calculations explain how the displayed percentages can sum to 100%. ApexNav does not specify its rounding procedure or publish a raw count for C, so taking the remainder is one possible explanation. We will credit ApexNav Fig. 7 in our caption and restore its original label, False Positive.
Summary and thanks
The added VLFM comparisons, failure analysis and episode trace clarify our contribution and its limits. Thank you for your careful review and constructive suggestions.
Response to Reviewer tbAp (1/4): Contributions, Comparison with ApexNav, and Ablations(W1-W3)
Response overview
Thank you for recognizing that detecting a target and pursuing it reliably are distinct problems. We now add a second backbone and MP3D component experiments, connecting this evidence to the proposed decision rules. Below, we address transferability, decision rules, failure modes, ACS interpretation, and the sources of our comparisons.
W1. ApexNav inheritance and transfer to VLFM
ConsistNav extends ApexNav's navigation pipeline with PCM/FSEC/SAAC. We also implemented four VLFM variants, all evaluated on the same 2,195 MP3D tasks and settings (SR/SPL, %):
This extends evaluation to a second backbone. Among the comparable new variants, PCM+FSEC exceeds PCM by +4.0/+4.1 points and FSEC by +5.0/+7.8. The original VLFM row is cited; the comparisons among our new variants do not depend on treating it as a rerun. These results provide initial evidence that PCM and FSEC can transfer across backbones, supporting their potential as general-purpose executive components.
W2. Methodological novelty and threshold design
1.Our contribution is a revisable commitment policy for the same object instance. PCM retains identity and evidence; FSEC combines them with verification progress and recovery feedback; SAAC constrains execution.
ApexNav already fuses observations and handles missed detections; CogNav uses LLM-selected states; UniGoal uses graph matching, verification and blacklists. Our executive couples three decisions around the retained candidate:
The contribution lies in these coupled transition rules. At the module level, PCM+FSEC improves both metrics over either module alone with SAAC disabled.
2.Thresholds should be assessed by function, selection protocol and sensitivity. Five relevant systems explicitly use them:
Training-free does not mean parameter-free. These examples serve different functions. Benchmark criteria, sensor/geometry tolerances and executive guards must be distinguished: camera/robot height and depth error can require internal calibration, while the benchmark success radius (e.g. ApexNav's 0.2 m) stays fixed. Explicit disclosure makes these assumptions reproducible and testable. We will clarify inherited versus added parameters, their roles and selection protocol.
W3. SAAC-only and the source of the gains
We added SAAC directly to ApexNav without PCM/FSEC (SR/SPL, %; ✗: absent, ✓: present):
In the new SAAC-only results, SR exceeds the cited baseline on both benchmarks; HM3Dv2 SPL also improves, while MP3D SPL remains nearly unchanged. This trend is consistent with SAAC helping task completion. Furthermore, in the shared VLFM setting, adding SAAC to PCM+FSEC raises SR/SPL from 42.2/22.8 to 45.6/25.7, a gain of +3.4/+2.9 points, providing direct evidence of SAAC's additional contribution to success rate and path efficiency in this setting.