- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper studies zero-shot vision-language navigation with smaller vision-language models and proposes a navigation system that combines task memory, hierarchical temporal memory, and geometric spatial memory. The system tracks progress through ordered subgoals, compresses past observations into summaries at multiple levels, and uses an occupancy map and a geometric controller to execute model-selected waypoints. When the current view provides insufficient information, the agent can acquire additional observations through a PREVIEW action. The authors use logical entailment from a semantic communication perspective to characterize the relationship between memory and decision-making. They also introduce three instruction-perturbation protocols: FLIP, GOAL-ONLY, and PARAPHRASE. Evaluation includes R2R-CE val-unseen, component ablations, and real-world robot experiments.
Critical strengths
S1. The system design aligns well with the requirements of navigation. Task memory maintains instruction order and execution progress, temporal memory preserves past observations, and spatial memory supports geometric planning. Pose information connects past experiences to spatial locations. The system also separates high-level waypoint selection from low-level motion control and includes an active observation action to supplement the current view. Together, these choices form a coherent perception–memory–decision–execution pipeline and provide a useful reference for navigation system design.
S2. The system achieves meaningful navigation improvements supported by component ablations. Among the zero-shot methods using 8B-scale models listed in Table 1, the proposed method achieves the highest SR and SPL, at 0.263 and 0.196, respectively. In Table 2, removing temporal memory, spatial memory, or PREVIEW reduces SR to 0.190, 0.205, and 0.190, respectively, supporting the utility of these components within the full system. Including temporal memory is also associated with substantial reductions in path length and action steps. Furthermore, adding temporal memory to CA-Nav improves SPL from 0.097 to 0.126, providing preliminary evidence that this component can benefit other navigation systems.
Critical weaknesses
W1. There is a gap between the central research question and the implemented method, and the methodological novelty is not sufficiently established. The paper centers on the question of what a navigator should remember, but the method primarily combines a fixed observation window, hierarchical summarization, and task and spatial memory. Although Section 3.2 provides an abstract criterion based on logical entailment, the paper does not translate it into an operational rule for selecting which facts to retain or directly evaluate whether summarization preserves the evidence needed for subsequent decisions. Consequently, the results primarily demonstrate the effectiveness of the integrated system rather than fully answer the central research question. Comparing different information-retention strategies under the same context budget would help substantiate the claimed novelty.
W2. The experiments do not establish the deployment advantage that motivates the work. A key motivation is navigation with smaller models that fit a robot's computational budget. Although the method consumes fewer tokens per decision, Table 3 reports 880.3k total tokens per episode, approximately 3.2 times the 273.3k reported for HSGM. Thus, a more compact input per decision does not necessarily imply lower resource consumption over an entire task. Moreover, the system uses both Qwen3-VL-8B and JoyAI, and the real-world experiments rely on remote inference on an A100 server. The results therefore support navigation effectiveness in real-world environments, but do not establish that the system fits an onboard computational budget.
Questions for authors
-
Do you have controlled comparisons that vary only how historical information is retained and organized, while keeping the context budget and all other modules fixed? Such comparisons would help determine whether the gains arise from the proposed memory design rather than simply providing more historical information.
-
Which specific memory design choices follow from the logical perspective in Section 3.2 and distinguish the method from generic hierarchical summarization? What concrete conclusions does the paper offer about what a navigator should remember, beyond the division into task, temporal, and spatial memory?
-
The paper emphasizes navigation within a robot's computational budget, but the system uses both Qwen3-VL-8B and JoyAI, and the real-world experiments rely on remote A100 inference. Does the stated computational budget refer to the reasoning model alone or to the full system? Do you have measurements of end-to-end latency, peak GPU memory usage, and computational cost over a complete episode, accounting for both models, that demonstrate how fewer tokens per decision translate into practical deployment benefits?
3 comments
Dear Reviewer LpY4,
As the discussion period closes later today, we will be online waiting for your feedback on our rebuttal, in which we respond to each of the concerns you raised. We have also added anonymized code and rollout videos to the supplementary material. We would highly appreciate it if you could take into account our response when updating the rating and having discussions with AC and other reviewers.
Thank you so much for your time and efforts.
Thank you for the clarifications. The comparisons still use different context budgets, the logical perspective does not provide an operational rule for selecting retained information, and full-system efficiency measurements remain unclear. I therefore maintain my current assessment.
Thanks for your Great Point!!!
Thank you for pressing us to make “What should a navigator remember?” a concrete design question. The revision compares three ways to retain the same stream of navigation observations and explains how the retained information serves later decisions.
1. What changes when the system retains history differently?
We compare three temporal inputs to the same JoyAI observer. Sliding supplies the eight most recent archived observation descriptions. Full supplies raw descriptions in chronological order, truncated at the input-token limit. ST-VLN keeps recent observations in detail and organizes older observations into pose-associated hierarchical summaries. Task memory, spatial memory, and PREVIEW are held fixed. Input length follows each retention rule: Full and Sliding consume 1,093k and 114k JoyAI input tokens per episode, respectively.
Sliding, Full, and hierarchical summarization achieve 0.190/0.112, 0.167/0.102, and 0.263/0.196 in SR/SPL, respectively. Extending the raw history substantially increases observer input, yet both SR and SPL decline. The strongest results come from retaining recent detail while consolidating older observations into descriptions of landmarks and their spatial relationships.
2. What should a navigator remember, and how does the logical perspective inform the design?
Our answer is to retain recent observations in detail and preserve three things about earlier experience: which landmarks the agent encountered, where it saw them, and how far it had progressed through the instruction. Consider a robot approaching a doorway along a corridor. Several consecutive observations may describe the same door. After the robot turns, the door’s location relative to the corner and whether the agent has completed the “pass the doorway” subgoal can still affect its next waypoint choice. The repeated descriptions of the approach can be condensed.
The logical perspective gives the agent a concrete question to address: Does the available evidence support the next decision? Task memory tracks completed and pending subgoals, temporal summaries retain landmarks associated with poses, and spatial memory maintains the map and committed waypoint. When the current view and retained evidence leave the next direction unclear—for example, when a referenced landmark lies outside the field of view—the agent can request additional observations through PREVIEW.
3. How do input size, episode cost, and real-world deployment relate?
The 8B designation refers to the Qwen3-VL-8B reasoner. The complete system also uses a JoyAI observer. In the physical trials, the Go2 streams onboard observations to a remote A100 server and executes the returned actions.
Table 3 reports both normalized input size and total episode token use. ST-VLN averages approximately 6.6k input tokens per navigation step; HSGM averages approximately 23.0k per model call. Across a complete episode, ST-VLN uses 880.3k tokens across both models, compared with approximately 273.3k for HSGM. In the Qwen3-VL-8B comparison, ST-VLN also travels farther on average (16.98 vs. 6.27 m) and achieves higher SR (0.263 vs. 0.049). In this comparison, ST-VLN uses about 3.2 times as many tokens per episode and reaches the goal over five times as often.