- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
This paper addresses the issue of inconsistent exploration memory and navigation decision representations in VLM-based Zero-Shot Object Navigation, and proposes MarvisNav. This method constructs a State-aware RVG to directly bind the exploration state to the visual navigation candidates, enabling the VLM to jointly consider target relevance and exploration progress in the same visual input, thereby enhancing the long-term sequence target search ability. The effectiveness of the method is verified on datasets such as HM3D and MP3D, and the influence of different memory representations on the VLM decision is analyzed through ablation experiments. Overall, this work improves the VLM navigation framework from the perspective of memory representation and achieves high-performance ZSON without the need for strategy training.
Critical strengths
- The problem is clearly defined, and the entry point is valuable: It focuses on an actual problem in VLM-based zero-shot object navigation: exploring the inconsistency in representation between memory and the current navigation decision, which makes it difficult for VLMs to effectively utilize historical information. The core viewpoint proposed in the "Motivation" section, "memory representation affects VLM decision-making", is somewhat enlightening.
- The method is well-designed and highly systematic: MarvisNav is proposed. Through State-aware RVG, the exploration states are encoded into the candidate navigation options, enabling the VLM to jointly consider target relevance and exploration states within the same visual input.
- The experiment was thorough, and the verification was comprehensive: It was verified on multiple benchmarks such as HM3D v0.1, HM3D v0.2, and MP3D. It included memory representation ablation, VLM backbone ablation, query efficiency, and real robot experiments. The experimental coverage was quite complete.
- The quality of the paper's writing and presentation is high: The logic among the motivation, methods, and experiments is relatively clear. The design of figures and ablation studies is good, enabling a clear and intuitive explanation of the method's motivation.
Critical weaknesses
- The innovation mainly lies in the design aspect, with limited algorithmic contributions: The core contribution is mainly in redesigning the interaction mode between memory and VLM decision-making. Although the paper discusses and explores the contribution of memory to navigation, clarifying the motivation, the method is essentially a new input representation (To some extent, it is similar to the Navigation Visual Prompt; there have also been similar studies recently, but there is a lack of introduction and distinction explanations for these works). The research paper is based on a considerable amount of existing work in areas such as RVG, frontier exploration, and VLM navigation. The methodological contribution of new algorithms is relatively limited.
- The efficiency analysis is not comprehensive: it mainly compares the number of VLM queries, rather than the overall system operation cost. The additional computational overheads such as RVG construction, map updates, and visual projections have not been fully analyzed. Whether the reduced queries translate into lower actual delay at the robot end still needs verification.
- Minor issue: The PDF of the paper is too large, which is not conducive to the author's reading. Please try to use vector graphics, etc.
Questions for authors
Overall, this paper has a coherent structure and a reasonable motivation. Particularly, the MOTIVATION section further enhances the value of the entire piece. However, in the core design of the method, there is still a lack of innovation, and there is also a shortage of a more comprehensive analysis of the efficiency of the complex ZSON system. If the author could further distinguish the proposed method from similar research approaches and provide an efficiency analysis of the ZSON system, I might raise the score even further.
4 comments
Q: System-level efficiency.
A: Thank you for this constructive suggestion. While the main paper reported query counts because VLM latency varies with serving conditions, we fully agree that breaking down module-level overheads and overall episode runtime provides a much clearer picture of practical efficiency. We have added a comprehensive runtime analysis over 1,000 episodes in Appendix F, summarized below: (Same setting as Fig. 4, with all methods under identical conditions on HM3D v0.2; the VLM is accessed through its remote API).
- (a) Overhead of RVG construction, map updates and visual projection. As shown below, the geometric and projection modules operate in 10–18 ms. Moreover, RVG construction runs asynchronously in a separate process without blocking control. The VLM query (1.84 s) and visual projection are triggered only at sparse decision sites.
| Module | When it runs | Mean time |
|---|---|---|
| Map update (occupancy grid and distance field) | every frame | 10.0 ms |
| RVG construction and state assignment | asynchronously at 10 Hz, in a separate process | 16.6 ms |
| Visual projection | once per sparse decision | 18 ms |
| VLM query | once per sparse decision | 1.84 s (p90 2.40 s) |
- (b) Overall system cost. WMNav and OpenFrontier use their official implementations and default settings. All used the same VLM. The last column reports the mean runtime separately for episodes that successfully reach the target and those that fail. | Method | VLM queries per episode | Time per episode (mean / median) | Successful / failed episodes (mean) | |---|---|---|---| | WMNav | 54.5 | 314 / 182 s | 229 / 532 s | | OpenFrontier | 22.8 | 225 / 156 s | 166 / 388 s | | MarvisNav| 4.1 | 51.0 / 35.0 s | 42 / 83 s |
Over a full MarvisNav episode (51.0 s on average), map updates and visual projection together add only about 1.9 s (4% of the episode), and RVG updates run asynchronously in a separate process, so they do not block the agent while it moves. By comparison, waiting for MarvisNav's 4.1 VLM queries takes ~7.5 s per episode (1.84 s per query), meaning that all RVG construction, map updates, and visual projection over an entire episode (~1.9 s) cost only roughly the same as a single VLM query. MarvisNav issues 4.1 queries per episode, against 54.5 for WMNav and 22.8 for OpenFrontier, and its mean episode time is 51.0 s, against 314 s and 225 s under the same VLM, i.e., 6.2× and 4.4× shorter. On the physical robot (Appendix B), MarvisNav issued 9 queries over a 39.2 m trajectory (about one every 4.4 m), and the robot moved between junctions without waiting for the VLM.
Q: “Minor issue: PDF file size”
A: Thank you for catching this and for inspecting our paper so closely! We fully agree that keeping the PDF lightweight makes the paper much more accessible. For the revised PDF, we will convert our figures and plots into vector formats (SVG).
We hope our response addressed your main concerns, and thanks again for your suggestions. We are excited about how these discussions and improvements can further enhance the quality of this paper, and we look forward to collaborating with you during the discussion phase.
Willing to clarify further concerns you may have.
Dear reviewer tq1T.
Thank you again for your encouraging review and helpful suggestions! We have uploaded the revised PDF (with updates highlighted in blue) incorporating your feedback:
- Contributions and distinctions from prior work (Sec.1, Sec. 2)
- Efficiency analysis of the whole system (Appendix F)
- PDF size reduced from 45.7 MB to 7.4 MB for a much smoother reading experience (Some figures are photographs and simulator renders that cannot be vectorized, so we compressed them instead)
We would greatly appreciate it if you could review our responses and let us know if they address your concerns. Your constructive suggestions have helped us clarify these points and further strengthen the paper. If you have any further suggestions or comments, we would be delighted to engage in discussion. Thank you once again for your time and support!
Thank you for the detailed responses and additional experiments. The responses address my concerns, but the contribution remains primarily representational and system-oriented, which limits my assessment of its methodological novelty. Overall, I have no further concerns and will maintain my score of 3.
Thank you for your valuable feedback. We agree that sharpening our contributions and distinctions from prior work further strengthens the paper. We hope our suggested changes and this individual response will address your concerns in detail:
A: Our contribution consists of two closely connected parts: an empirical study of how memory representation shapes VLM decisions, and the MarvisNav framework that turns this insight into an efficient zero-shot navigator. We have further clarified this in the introduction (line 112).
A finding and insight about memory representation. Prior ZSON methods supply exploration memory in different forms (e.g., candidate filtering, text history, visited markers). We ask whether the form itself matters. Within a single unified framework, we show that the memory format directly shifts VLM route preferences (Fig. 2) and governs navigation performance (Table 2). Specifically, adding text history improves SR by only +1.7% over no memory (reaching 73.6%), whereas projecting binary visited markers reaches 76.5%. Furthermore, using the exact same four exploration states, binding states directly to choices achieves 81.2% SR versus 78.5% for separate markers (an additional ablation). This proves that how memory is represented matters as much as whether it is available.
An approach that puts the finding into practice. Turning this takeaway into a strong navigator (MarvisNav) requires solving four concrete problems: (a) Where to locate choices? We build an online Reduced Voronoi Graph (RVG) where junctions serve as sparse decision sites and outgoing branches define candidate routes. (b) What state should each choice carry? By associating RVG branches with frontiers, each candidate is assigned one of four exploration states, capturing key intermediate cases (e.g., a visited junction still leading to unexplored frontiers) beyond binary visitation. (c) Which choices should remain visible? We design an occlusion-checked egocentric projection that filters obstructed paths, uses directional arrows for out-of-view branches, and keeps candidate letters consistent across views. (d) How to query the VLM efficiently? Since each VLM call is time-consuming, querying only at sparse junctions (jointly inferring a route choice and target visibility) requires just 7.5% of WMNav's VLM calls, enabling low-cost, real-time deployment on a Unitree Go2 across indoor and outdoor scenes.
Distinction from Navigation Visual Prompting Works. While prior methods also project visual markers onto egocentric views, they differ fundamentally in whether exploration memory is provided to the VLM and how it is aligned with route choices. We have made this clearer in Sec. 2 (line 156):
Route choices without exploration history (e.g., OpenFrontier, WMNav, DriveNav, PIVOT): These methods overlay markers onto navigable locations to indicate where the robot can go. The VLM is shown selectable alternatives, such as marked candidates, frontiers, or directions. Exploration memory, where used, is applied outside the VLM by restricting the candidates before the query or adjusting the VLM's preferences after it, so the VLM does not weigh it together with target relevance.
Memory shown as separate visual markers(e.g., Mem2Ego): Mem2Ego projects visited locations as separate markers alongside route candidates. The VLM has to infer from image proximity which candidate each marker concerns, which makes its route preferences less reliable (Fig. 2d), and binary visitation also cannot express that a visited junction may still lead to unexplored space.
Exploration states on the route choices (MarvisNav): Rather than separating actions and history, MarvisNav encodes four frontier-derived exploration states directly into the candidate visual markers themselves. This enables the VLM itself to jointly consider target semantic relevance and remaining exploration value without cross-marker spatial matching.