- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper introduces RSI-Nav, a training-free Object-Goal Navigation system that combines retrospective rule memory with prospective “preflection.” After each episode, an LLM converts the trajectory, target category, and binary success outcome into natural-language navigation rules; semantic credit, historical success, and a UCB term are then used to retrieve rules in later episodes. Before each frontier decision, the LLM predicts observations and possible failures for up to three candidates and selects a waypoint using the retrieved rules. The rule memory persists across episodes within each benchmark dataset. RSI-Nav reports 67.3/33.9 SR/SPL on HM3D and 49.0/19.1 on MP3D, with module, retrieval, backbone, and cost analyses.
Critical strengths
-
The same-backbone gains are consistent across two datasets and both standard metrics. Relative to reproduced ASCENT with Qwen2.5-7B, RSI-Nav improves HM3D by 1.8 SR/1.1 SPL points and MP3D by 3.9 SR/2.9 SPL points; the trend remains with Qwen3-8B.
-
The system is efficient and meaningfully decomposed. The paper separately studies prospective and retrospective components and several retrieval strategies, while average LLM calls increase only from 8.7 for ASCENT to 9.6 for RSI-Nav.
Critical weaknesses
-
RSI-Nav consumes validation outcomes across episodes, but the baselines are not shown to operate under the same protocol. Episode can use information derived from earlier validation outcomes, whereas the quoted baselines in Table 1 apparently cannot. This changes the task's information regime and makes the SOTA comparison potentially apples-to-oranges (Section 4.1.4, p. 8; Appendix A, p. 14; Table 1, p. 5).
-
The core self-improvement mechanism is not reproducible from the current description. Rule attribution, statistic updates, category filtering, deduplication, invalid-rule criteria, and the treatment of rewritten rules are central to the method rather than peripheral implementation details (Section 3.4, pp. 5–7; Appendix A, pp. 12–14).
Questions for authors
-
Does the Habitat evaluation policy permit the binary outcome from one validation episode to influence later validation episodes? For each baseline in Tables 1 and 3, was the method given exactly the same feedback, persistent state, official episode order, and reset schedule (Section 4.1.4, p. 8; Tables 1 and 3, pp. 5 and 9)?
-
How exactly are , , and updated for existing rules? How are
Followed_Rule_Indices, conflicting rules, target-category filtering, duplicate or invalid rules, memory capacity, and statistics for rewritten rules handled (Section 3.4, pp. 5–7; Appendix A, pp. 12–14)? -
Why does Table 1 list VLFM in both the learning-based and training-free groups, and what distinguishes those two configurations? Please also clarify the protocol compatibility and provenance of all quoted results (Table 1, p. 5).
We sincerely thank reviewer for your comments. To address your questions, we (1) clarify the feedback and reset protocols used in Tables 1 and 3, (2) explain how rules are attributed, updated, and maintained, and (3) distinguish the two VLFM configurations in Table 1 and document the sources of the baseline results.
Response to W1 & Q1. What feedback does each method receive?
Habitat-Labresets the environment and agent each episode, passes observations toagent.act, and collects metrics for scoring. Feeding the previous episode’s success flag to the agent is an extension of this interface. In our online cross-episode adaptation setting (Tables 1 and 3), each dataset starts with empty memory. After each episode in the official order, its binary outcome updates the memory available to later episodes.Baseline protocols. Table 1 quotes published baseline results. For these rows, we can verify episode order, feedback, and reset schedules only where the sources document them. In Table 3, we ran ASCENT* and RSI-Nav on the same validation episodes in the same order, with the same simulator, perception stack, and language backbone. ASCENT* resets its episodic state and receives no previous episode’s outcome. Full RSI-Nav carries updated memory. We will label this protocol difference in Table 3.
Results with no feedback from earlier validation episodes. On the full benchmarks with Qwen3-8B, preflection alone (Table 2, setting 2) achieves 66.5 / 33.5 on HM3D and 47.4 / 18.4 on MP3D (SR / SPL, %). ASCENT* achieves 65.3 / 33.0 and 45.3 / 16.2. These gains use no cross-episode rules.
For frozen transfer, we collected 1,000 training episodes across 80 HM3D scenes at each checkpoint and evaluated all four checkpoints on the same 200 tasks across 20 held-out scenes
The final snapshot succeeded on 69.00, versus 66.00 with empty memory.
Response to W2 & Q2. When is a rule updated?
1. Attribution and usage. At each frontier decision, the planner retrieves up to five rules from the current target category.
Followed_Rule_Indicesrefers to their 1-based positions. We collect valid cited rule IDs across the episode. If there are no valid citations anywhere in the episode, we use all rule IDs shown. Each attributed rule increments Nr once per episode.2. Outcome and semantic credit. In the manuscript, pr denotes the fraction of attributed episodes that succeeded. The implementation adjusts each binary outcome using the rule’s stored semantic credit and variance, accumulates these adjusted rewards, and divides their sum by Nr for the UCB reward mean.
Semantic credit Sr updates when reflection first proposes a rule or proposes the same rule again. We compute signed changes in goal similarity over up to three support steps. Softmax weighting (temperature 0.30) favors gains on success and drops on failure. Stored credit starts at zero and updates to 80% of the previous value plus 20% of the new weighted credit. Reuse alone updates the count and cumulative reward and preserves the stored semantic credit.
3. Rule maintenance. We accept positive or negative proposals with nonempty text. A reflection labeled
luckyadds no rule. Within a target category, a proposal with identical text and type updates the existing record and retains its history. Rewording creates a new record. UCB ranks coexisting conflicting rules and the planner selects a frontier using current observations. All rules are retained, with no capacity limit or eviction.Response to Q3. Which VLFM results are quoted in Table 1?
The two VLFM rows use the same frontier-selection method but different low-level controllers. The original VLFM uses a pretrained PointNav policy to reach each selected waypoint, so we list it in the learning-based group. ApexNav reports a variant, VLFM*, that replaces PointNav with a shortest-path planner. We list this variant in the training-free group because its navigation controller requires no training.
Table 1 compares published baseline results on the HM3D-v0.1 and MP3D validation splits using success rate (SR) and success weighted by path length (SPL). Each baseline score is quoted for the configuration evaluated in its source paper.