acceptodds
Under review as a conference paper at ICLR 2027

MarvisNav: Making Memory Visible on Route Choices for Zero-Shot Object Navigation

Abstract

When searching for an object, people choose their next move by considering both likely target locations and places already explored. The current view can cue place-associated memories, bringing target relevance and prior exploration into the same spatial context. In many zero-shot object navigation (ZSON) methods, however, vision-language models (VLMs) infer promising search areas from egocentric images, but exploration history enters the decision through separate representations, e.g., as text or maps. This separation either requires an additional fusion step to align semantic evidence with exploration state, or leaves the correspondence between memory and route choices implicit for the VLM to recover. We instead make exploration memory visible directly on visual route choice. We propose MarvisNav, a ZSON framework that maintains a topological graph and projects candidate nodes together with their exploration states onto the egocentric perception as memory-bearing visual route choices. These topological states capture local exploration progress beyond binary visitation. By binding exploration state directly to each visual candidate, MarvisNav enables the VLM to jointly evaluate target relevance and exploration state and make the final high-level selection without a separate post-hoc fusion or reranking stage. Without policy training, MarvisNav achieves state-of-the-art performance on HM3D ( SR and SPL), while competitive on MP3D. Moreover, it outperforms representative VLM-based methods with far fewer VLM calls (e.g., of WMNav). Real-robot experiments across diverse scenes further validate its practical deployability. Beyond MarvisNav, our study shows that memory representation shapes VLM decisions and ZSON performance, highlighting that effective memory use in ZSON depends not only on its availability, but also on how it is represented.

Primary area:
applications to robotics, autonomy, planning
Keywords:
zero-shot object navigation, vision-language models, embodied navigation, exploration memory, memory representation, visual navigation

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_tq1TOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper addresses the issue of inconsistent exploration memory and navigation decision representations in VLM-based Zero-Shot Object Navigation, and proposes MarvisNav. This method constructs a State-aware RVG to directly bind the exploration state to the visual navigation candidates, enabling the VLM to jointly consider target relevance and exploration progress in the same visual input, thereby enhancing the long-term sequence target search ability. The effectiveness of the method is verified on datasets such as HM3D and MP3D, and the influence of different memory representations on the VLM decision is analyzed through ablation experiments. Overall, this work improves the VLM navigation framework from the perspective of memory representation and achieves high-performance ZSON without the need for strategy training.

Critical strengths
  • The problem is clearly defined, and the entry point is valuable: It focuses on an actual problem in VLM-based zero-shot object navigation: exploring the inconsistency in representation between memory and the current navigation decision, which makes it difficult for VLMs to effectively utilize historical information. The core viewpoint proposed in the "Motivation" section, "memory representation affects VLM decision-making", is somewhat enlightening.
  • The method is well-designed and highly systematic: MarvisNav is proposed. Through State-aware RVG, the exploration states are encoded into the candidate navigation options, enabling the VLM to jointly consider target relevance and exploration states within the same visual input.
  • The experiment was thorough, and the verification was comprehensive: It was verified on multiple benchmarks such as HM3D v0.1, HM3D v0.2, and MP3D. It included memory representation ablation, VLM backbone ablation, query efficiency, and real robot experiments. The experimental coverage was quite complete.
  • The quality of the paper's writing and presentation is high: The logic among the motivation, methods, and experiments is relatively clear. The design of figures and ablation studies is good, enabling a clear and intuitive explanation of the method's motivation.
Critical weaknesses
  • The innovation mainly lies in the design aspect, with limited algorithmic contributions: The core contribution is mainly in redesigning the interaction mode between memory and VLM decision-making. Although the paper discusses and explores the contribution of memory to navigation, clarifying the motivation, the method is essentially a new input representation (To some extent, it is similar to the Navigation Visual Prompt; there have also been similar studies recently, but there is a lack of introduction and distinction explanations for these works). The research paper is based on a considerable amount of existing work in areas such as RVG, frontier exploration, and VLM navigation. The methodological contribution of new algorithms is relatively limited.
  • The efficiency analysis is not comprehensive: it mainly compares the number of VLM queries, rather than the overall system operation cost. The additional computational overheads such as RVG construction, map updates, and visual projections have not been fully analyzed. Whether the reduced queries translate into lower actual delay at the robot end still needs verification.
  • Minor issue: The PDF of the paper is too large, which is not conducive to the author's reading. Please try to use vector graphics, etc.
Questions for authors

Overall, this paper has a coherent structure and a reasonable motivation. Particularly, the MOTIVATION section further enhances the value of the entire piece. However, in the core design of the method, there is still a lack of innovation, and there is also a shortage of a more comprehensive analysis of the efficiency of the complex ZSON system. If the author could further distinguish the proposed method from similar research approaches and provide an efficiency analysis of the ZSON system, I might raise the score even further.

4 comments

Thank you for your valuable feedback. We agree that sharpening our contributions and distinctions from prior work further strengthens the paper. We hope our suggested changes and this individual response will address your concerns in detail:

Q: Contributions and distinctions from prior work

A: Our contribution consists of two closely connected parts: an empirical study of how memory representation shapes VLM decisions, and the MarvisNav framework that turns this insight into an efficient zero-shot navigator. We have further clarified this in the introduction (line 112).

  1. A finding and insight about memory representation. Prior ZSON methods supply exploration memory in different forms (e.g., candidate filtering, text history, visited markers). We ask whether the form itself matters. Within a single unified framework, we show that the memory format directly shifts VLM route preferences (Fig. 2) and governs navigation performance (Table 2). Specifically, adding text history improves SR by only +1.7% over no memory (reaching 73.6%), whereas projecting binary visited markers reaches 76.5%. Furthermore, using the exact same four exploration states, binding states directly to choices achieves 81.2% SR versus 78.5% for separate markers (an additional ablation). This proves that how memory is represented matters as much as whether it is available.

  2. An approach that puts the finding into practice. Turning this takeaway into a strong navigator (MarvisNav) requires solving four concrete problems: (a) Where to locate choices? We build an online Reduced Voronoi Graph (RVG) where junctions serve as sparse decision sites and outgoing branches define candidate routes. (b) What state should each choice carry? By associating RVG branches with frontiers, each candidate is assigned one of four exploration states, capturing key intermediate cases (e.g., a visited junction still leading to unexplored frontiers) beyond binary visitation. (c) Which choices should remain visible? We design an occlusion-checked egocentric projection that filters obstructed paths, uses directional arrows for out-of-view branches, and keeps candidate letters consistent across views. (d) How to query the VLM efficiently? Since each VLM call is time-consuming, querying only at sparse junctions (jointly inferring a route choice and target visibility) requires just 7.5% of WMNav's VLM calls, enabling low-cost, real-time deployment on a Unitree Go2 across indoor and outdoor scenes.

Distinction from Navigation Visual Prompting Works. While prior methods also project visual markers onto egocentric views, they differ fundamentally in whether exploration memory is provided to the VLM and how it is aligned with route choices. We have made this clearer in Sec. 2 (line 156):

  • Route choices without exploration history (e.g., OpenFrontier, WMNav, DriveNav, PIVOT): These methods overlay markers onto navigable locations to indicate where the robot can go. The VLM is shown selectable alternatives, such as marked candidates, frontiers, or directions. Exploration memory, where used, is applied outside the VLM by restricting the candidates before the query or adjusting the VLM's preferences after it, so the VLM does not weigh it together with target relevance.

  • Memory shown as separate visual markers(e.g., Mem2Ego): Mem2Ego projects visited locations as separate markers alongside route candidates. The VLM has to infer from image proximity which candidate each marker concerns, which makes its route preferences less reliable (Fig. 2d), and binary visitation also cannot express that a visited junction may still lead to unexplored space.

  • Exploration states on the route choices (MarvisNav): Rather than separating actions and history, MarvisNav encodes four frontier-derived exploration states directly into the candidate visual markers themselves. This enables the VLM itself to jointly consider target semantic relevance and remaining exploration value without cross-marker spatial matching.

Q: System-level efficiency.

A: Thank you for this constructive suggestion. While the main paper reported query counts because VLM latency varies with serving conditions, we fully agree that breaking down module-level overheads and overall episode runtime provides a much clearer picture of practical efficiency. We have added a comprehensive runtime analysis over 1,000 episodes in Appendix F, summarized below: (Same setting as Fig. 4, with all methods under identical conditions on HM3D v0.2; the VLM is accessed through its remote API).

  • (a) Overhead of RVG construction, map updates and visual projection. As shown below, the geometric and projection modules operate in 10–18 ms. Moreover, RVG construction runs asynchronously in a separate process without blocking control. The VLM query (1.84 s) and visual projection are triggered only at sparse decision sites.
ModuleWhen it runsMean time
Map update (occupancy grid and distance field)every frame10.0 ms
RVG construction and state assignmentasynchronously at 10 Hz, in a separate process16.6 ms
Visual projectiononce per sparse decision18 ms
VLM queryonce per sparse decision1.84 s (p90 2.40 s)
  • (b) Overall system cost. WMNav and OpenFrontier use their official implementations and default settings. All used the same VLM. The last column reports the mean runtime separately for episodes that successfully reach the target and those that fail. | Method | VLM queries per episode | Time per episode (mean / median) | Successful / failed episodes (mean) | |---|---|---|---| | WMNav | 54.5 | 314 / 182 s | 229 / 532 s | | OpenFrontier | 22.8 | 225 / 156 s | 166 / 388 s | | MarvisNav| 4.1 | 51.0 / 35.0 s | 42 / 83 s |

Over a full MarvisNav episode (51.0 s on average), map updates and visual projection together add only about 1.9 s (4% of the episode), and RVG updates run asynchronously in a separate process, so they do not block the agent while it moves. By comparison, waiting for MarvisNav's 4.1 VLM queries takes ~7.5 s per episode (1.84 s per query), meaning that all RVG construction, map updates, and visual projection over an entire episode (~1.9 s) cost only roughly the same as a single VLM query. MarvisNav issues 4.1 queries per episode, against 54.5 for WMNav and 22.8 for OpenFrontier, and its mean episode time is 51.0 s, against 314 s and 225 s under the same VLM, i.e., 6.2× and 4.4× shorter. On the physical robot (Appendix B), MarvisNav issued 9 queries over a 39.2 m trajectory (about one every 4.4 m), and the robot moved between junctions without waiting for the VLM.

Q: “Minor issue: PDF file size”

A: Thank you for catching this and for inspecting our paper so closely! We fully agree that keeping the PDF lightweight makes the paper much more accessible. For the revised PDF, we will convert our figures and plots into vector formats (SVG).

We hope our response addressed your main concerns, and thanks again for your suggestions. We are excited about how these discussions and improvements can further enhance the quality of this paper, and we look forward to collaborating with you during the discussion phase.

AuthorsOpenReview ↗

Willing to clarify further concerns you may have.

Dear reviewer tq1T.

Thank you again for your encouraging review and helpful suggestions! We have uploaded the revised PDF (with updates highlighted in blue) incorporating your feedback:

  • Contributions and distinctions from prior work (Sec.1, Sec. 2)
  • Efficiency analysis of the whole system (Appendix F)
  • PDF size reduced from 45.7 MB to 7.4 MB for a much smoother reading experience (Some figures are photographs and simulator renders that cannot be vectorized, so we compressed them instead)

We would greatly appreciate it if you could review our responses and let us know if they address your concerns. Your constructive suggestions have helped us clarify these points and further strengthen the paper. If you have any further suggestions or comments, we would be delighted to engage in discussion. Thank you once again for your time and support!

Thank you for the detailed responses and additional experiments. The responses address my concerns, but the contribution remains primarily representational and system-oriented, which limits my assessment of its methodological novelty. Overall, I have no further concerns and will maintain my score of 3.

Reviewer_yAENOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes MarvisNav, a training-free framework that projects exploration memory directly onto the visual input of a VLM. Compared with previous work, the method constructs a reduced Voronoi graph from RGB-D observations and assigns each node an exploration state. It then projects the current candidate nodes onto egocentric views and annotates them with different colors to represent their exploration states. A VLM evaluates each option and selects the next route.

Critical strengths
  1. A coherent memory interface. Combining exploration states with candidate route choices is a reasonable design. The VLM does not need to infer the correspondence between past memory and future route choices, allowing it to focus on deciding which route is most valuable for finding the object. The memory representation captures the exploration state precisely, including the fact that a visited junction may still lead to unexplored space.
  2. Broad simulation results and useful diagnostic experiments. The paper evaluates the proposed method on multiple benchmarks and with different VLMs. It also conducts ablation experiments with different memory representations. These experiments provide useful findings about how memory-system design can affect zero-shot object navigation.
Critical weaknesses
  1. The novelty relative to existing work is insufficient. The paper itself describes Mem2Ego as projecting candidate choices and visited locations into egocentric views for VLM-based decision making. MarvisNav adopts a similar approach and attaches exploration memory to the candidates. The improvement from visited markers to an RVG-based representation does not appear substantial enough to support publication as a standalone paper. Moreover, the paper does not directly compare against Mem2Ego with all other components held constant, so the reported gains cannot be fully attributed to the innovation proposed in this paper.
Questions for authors
  1. Can the authors provide a comparison with Mem2Ego using the same VLM, perception module, and all other experimental conditions?
  2. Can the authors clarify the novelty of this work compared with Mem2Ego?
5 comments

Thank you for your thoughtful review of our research and positive remarks about the coherent memory interface and diagnostic experiments. We appreciate your constructive feedback about potential improvements to our paper and hope our response will address your concerns in detail:

Q: Comparison with Mem2Ego’s visual memory interface under identical conditions

A: This controlled comparison is provided by the "Visited Markers" row in Table 2, which implements Mem2Ego’s visual memory interface (projecting visited locations as separate markers alongside candidates) within our pipeline while holding the VLM, perception module, and all other components constant. Under identical settings, MarvisNav improves SR from 76.5% to 81.2% (+4.7 points) and SPL from 40.6% to 42.5%. While Sec. 3 introduced this variant by citing Mem2Ego (line 195), neither Sec. 3 nor Table 2 explicitly named it. For clarity, we have further updated both line 427 and Table 2 (as "Visited Markers (Mem2Ego-style)") to ensure it is as clear as possible.

Implementation details of Visited Markers (Mem2Ego-style): Since Mem2Ego has no public code, we faithfully implemented its visual memory design from the paper within our shared pipeline, matching Mem2Ego across four core aspects: (i) memory content: binary visitation, i.e., where the agent has been; (ii) presentation: visited locations drawn in the egocentric view as circular markers separate from the unchanged candidate markers; (iii) prompt: stating what the marker means and asking the VLM to prefer candidates away from it; and (iv) decision: having the VLM select one marked candidate from the annotated image. The candidates themselves are the same RVG candidates in all variants, rather than Mem2Ego's frontier-based ones, so that only the memory interface differs.

Q2: Core contribution vs prior work (Mem2Ego)

A: Thank you for the question. While both Mem2Ego and MarvisNav project information onto egocentric views, MarvisNav is not an incremental modification of Mem2Ego. The two works differ fundamentally in where memory is shown and what memory is shown, alongside our broader empirical study of memory representations. We have further clarified these in Sec. 1 and Sec. 2 (lines 112 and 156).

Specifically, as you insightfully noted in Strengths 1, our interface (i) "frees the VLM from inferring the correspondence between past memory and future route choices" and (ii) "captures that a visited junction may still lead to unexplored space." These two properties are precisely what Mem2Ego's interface lacks.

To further separate where memory is shown from what memory is shown, we ran two additional variants alongside Visited Markers (Mem2Ego-style) and MarvisNav: placing binary visitation directly on the choices, and drawing our four exploration states as separate markers beside the choices (described in the main paper Sec. 4.1). All four variants share the same settings and navigation stack across all 1,000 HM3D v0.2 episodes (SR / SPL in %):

Where memory is shown \ What memory is shownBinary VisitationFour exploration states
Separate markers beside the choices76.5 / 40.6 (Mem2Ego-style)78.5 / 42.2
On the choices79.6 / 42.481.2 / 42.5 (MarvisNav)
  • Where memory is shown (separate markers vs. on the choices): Mem2Ego draws memory as separate markers alongside candidates, whereas MarvisNav encodes the state directly into each candidate marker. The motivation study shows the same effect on VLM preferences: with separate visited markers, neither VLM reliably prefers the unvisited route (Fig. 2d). In closed-loop navigation, placing the same memory on the choices instead of beside them raises SR from 76.5% to 79.6% (with binary visitation).
  • What memory is shown (binary visitation vs. four exploration states): Mem2Ego records binary visitation (visited or not), whereas MarvisNav associates RVG branches with frontiers to define four exploration states that distinguish partially explored branches from fully exhausted ones. Even when drawn as separate markers, replacing binary visitation with our four exploration states raises SR from 76.5% (Mem2Ego-style) to 78.5%.

Finally, we appreciate your highlight in Strengths 2 that our experiments "provide useful findings about how memory-system design can affect zero-shot object navigation." Beyond proposing a single navigation pipeline, our study evaluates how different memory representations shape both VLM route preferences (Fig. 2) and closed-loop performance (Table 2) within one controlled framework. For instance, text history adds only +1.7% SR over no memory (71.9% to 73.6%), whereas separate visited markers and our choice-bound states reach 76.5% and 81.2%. These findings not only ground the design of MarvisNav, but also offer practical guidance for building future VLM-based ZSON memory interfaces.

We hope our response addressed your main concerns, and thanks again for your suggestions. We are excited about how these discussions and improvements can further enhance the quality of this paper, and we look forward to collaborating with you during the discussion phase.

AuthorsOpenReview ↗

Willing to clarify further concerns you may have.

Dear Reviewer yAEN,

Thank you again for your time and constructive feedback! We have uploaded the revised PDF (with updates highlighted in blue) incorporating your suggestions:

  • Core contributions and distinctions from prior work such as Mem2Ego and OpenFrontier (Sec. 1, Sec. 2)
  • Controlled comparison with the Mem2Ego-style baseline under the identical navigation stack (Sec. 5.2, Table 2)
  • Where the improvement over visited markers comes from (new comparison experiments in Sec. 5.3, Table 3)

We would greatly appreciate it if you could review our responses and let us know if they address your concerns. Your feedback has helped us clarify these points and further strengthen the paper, and if your concerns are resolved, we would be grateful if you would consider updating your assessment. If you have any further questions or comments, we would be delighted to engage in discussion. Thank you once again for your time and consideration!

Reviewer_ahEcOpenReview ↗
Rating
3(+1)
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper investigates how to present exploration memory to the Vision-Language Model (VLM) in an effective format during target navigation to optimize VLM-based selection. The authors propose MarvisNav, which first constructs a Reduced Voronoi Graph (RVG) and defines four exploration states based on the relationship between nodes and frontiers. Subsequently, visible paths and candidate nodes are projected onto the image, with candidate points and exploration states represented by letters and colors. The VLM then selects a candidate by combining visual semantics and exploration states, and the coordinates of the selected node are executed by the navigation module. The authors evaluate the method on HM3D v0.1, HM3D v0.2, and MP3D.

Critical strengths

1.The authors directly annotate candidate nodes with exploration states based on frontiers and local connectivity, enabling the VLM to simultaneously consider scene semantics and remaining exploration value. This design is intuitive and demonstrates a degree of novelty. 2.The experimental design is comprehensive. In comparisons using the same VLM, MarvisNav achieves superior SR and SPL while reducing the number of VLM calls.

Critical weaknesses

1.The existing ablation study does not control for the information provided to the VLM. Textual History provides the ten most recent visit records, Visited Markers indicate whether a location has been visited, while Choice-aligned provides four more specific topological exploration states. Therefore, the results demonstrate the effectiveness of the full design, but it is difficult to determine whether the improvements come from richer state information or from directly aligning the states with the candidate choices. A controlled comparison using the same state information presented as text, independent markers, and candidate coloring is missing. 2.Mem2Ego already projects navigation candidates and visitation memory onto egocentric images, while OpenFrontier also uses visual markers to support VLM-based navigation target selection. The main contribution of this paper is to further encode topological exploration states directly into candidate markers. Although this design is practical, it represents a relatively incremental advance over existing visual memory-based decision-making methods.

Questions for authors

1.Could the authors compare their method with Mem2Ego’s visual memory interface under the same VLM and navigation settings to quantify the additional benefits brought by the proposed topological exploration states and candidate marker design? 2.Could the authors compare three approaches under the same four exploration states and consistent decision prompts: (1) providing textual state descriptions associated with candidate letters, (2) displaying separate state markers next to the candidates, and (3) directly color-coding the candidates, and report the corresponding SR and SPL results?

5 comments

Thank you for your thoughtful review of our research and for recognizing the novelty of state-annotated candidate design and the efficiency of our method. We hope our suggested changes and this individual response will address your concerns in detail:

Q: Comparison with Mem2Ego’s visual memory interface under identical conditions(Q1)

A: This comparison under the same VLM and navigation stack is provided by the "Visited Markers" row in Table 2, which implements Mem2Ego’s visual memory interface (projecting visited locations as separate markers alongside candidates). Under identical settings across all 1,000 HM3D v0.2 episodes, MarvisNav improves SR from 76.5% to 81.2% (+4.7%) and SPL from 40.6% to 42.5% (+1.9%). While Sec. 3 introduced this variant by citing Mem2Ego (line 195), Table 2 did not explicitly name it. To make this explicit, we will further clarify Visited Markers (Mem2Ego-style) in both Table 2 and line 427.

Implementation details of Visited Markers (Mem2Ego-style): Since Mem2Ego has no public code, we faithfully implemented its visual memory design from the paper within our shared pipeline across four core aspects: (i) memory content: binary visitation, i.e., where the agent has been; (ii) presentation: visited locations drawn in the egocentric view as circular markers separate from the unchanged candidate markers; (iii) prompt: stating what the marker means and asking the VLM to prefer candidates away from it; and (iv) decision: having the VLM select one marked candidate from the annotated image.

Q: Quantifying the individual benefits of exploration states vs. candidate marker design (W1)

A: Thank you for this valuable suggestion. To cleanly separate whether the gains come from richer state information or from aligning states with choices, we ran the ablation studies you proposed. The table is shown below. We will include the comparisons and new discussion in line 437.

The table below crosses what memory is shown (columns) with where memory is shown (rows). Across columns, Binary visitation only distinguishes visited from unvisited locations, whereas our Four exploration states classify each route as an unexplored endpoint, an explored endpoint, an open junction, or an explored junction depending on whether unexplored space remains beyond it (Sec. 4.1). Across rows, memory is either drawn as Separate markers beside the choices or encoded directly On the choices. All four variants share the exact same settings and navigation stack across the 1,000 HM3D v0.2 episodes, and the prompts differ only in the sentences describing the markers (SR / SPL in %):

Where memory is shown \ What memory is shownBinary visitationFour exploration states
Separate markers beside the choices76.5 / 40.6 (Mem2Ego-style)78.5 / 42.2
On the choices79.6 / 42.481.2 / 42.5 (MarvisNav)
  • Benefit of candidate marker design (Where): Holding the memory content constant, moving the state from separate markers directly onto the choices raises SR from 76.5% to 79.6% (+3.1%) for binary visitation, and from 78.5% to 81.2% (+2.7%) for our four exploration states.
  • Benefit of four exploration states (What): Holding marker placement constant, replacing binary visitation with our four exploration states raises SR from 76.5% to 78.5% (+2.0%) on separate markers, and from 79.6% to 81.2% (+1.6%) on the choices.

Overall, these results confirm that placing memory directly on the choices is the primary driver of performance, while the four exploration states provide complementary gains.

Q: Comparison across 3 different representation formats under the same four exploration states (Q2)

Thank you for this suggestion. We now report all three variants: (1) and (2) are new, while (3) is MarvisNav. All three use the same four exploration states and consistent decision prompts (only the sentences describing the memory differ). We will further clarify this in Sec. 5.2:

Representation under the same four exploration statesSR (%)SPL (%)
(1) Textual state descriptions associated with candidate letters79.542.2
(2) Separate state markers next to the candidates78.542.2
(3) Directly color-coding the candidates (MarvisNav)81.242.5

In (1), candidate markers are drawn in a single uniform color on the image, while the text prompt explicitly states the exploration state of each candidate letter (e.g., A, B, C). In (2), our four exploration states are drawn in the image as separate color-coded markers beside the candidate markers.

  • Associating exploration memory with candidate choices removes correspondence ambiguity, even in text: Unlike Textual History in Table 2 (73.6% SR), which lists the ten most recent visit records without associating them with the current candidates, approach (1) gives the four exploration states and explicitly binds each to its candidate letter, reaching 79.5%. Notably, with the same four states, this is also higher than the unaligned separate visited markers (Mem2ego-style) in (2) (78.5%): associating states with candidate letters frees the VLM from inferring the spatial relationships between separate markers and candidate choices in the image.

  • Visually choice-aligned states perform best: Even when both approaches already associate the four states with each candidate, directly color-coding the candidate markers in the image (81.2%) is higher than textual state descriptions (79.5%). Color-coding encodes the exploration states directly in the candidate markers, whereas textual state descriptions are associated with the candidates through their letters.

Q: Distinction from Mem2Ego and OpenFrontier and core contributions(W2)

A: While OpenFrontier, Mem2Ego, and ours(MarvisNav) all project visual markers onto egocentric views, they differ fundamentally in what memory is shown and where memory is shown, alongside our broader empirical study of memory representations. We will further clarify these distinctions in Sec. 1 and Sec. 2 (lines 112 and 156).

  • Route choices without exploration history on the markers (e.g., OpenFrontier). These methods overlay markers onto navigable locations to show selectable action goals. Exploration memory, where used, is applied outside the VLM (e.g., by filtering or restricting candidates beforehand), so the VLM cannot weigh exploration history together with target relevance (unlike our design, which as you noted in Strengths 1 enables the VLM to "simultaneously consider scene semantics and remaining exploration value").

  • Memory shown as separate visual markers (Mem2Ego) vs. on the choices (MarvisNav). Mem2Ego brings memory into the image, but in terms of what memory is shown, it only records binary visitation (treating visited junctions that still lead to unexplored areas the same as dead ends); in terms of where memory is shown, it draws separate markers beside the choices (76.5% SR), leaving marker-to-candidate correspondence implicit. Instead of drawing past footprints beside candidates, MarvisNav derives route-level exploration states from our topological graph and encodes them directly on the choices (81.2% SR), querying the VLM only at sparse junctions to achieve higher SR and SPL with substantially fewer VLM calls (as you highlighted in Strengths 2).

  • A finding and insight about memory representation. Prior ZSON methods supply exploration memory in different forms (e.g., candidate filtering, text history, visited markers). Beyond proposing a single navigation pipeline, our study evaluates how different memory representations shape both VLM route preferences (Fig. 2) and closed-loop performance (Table 2 and the controlled ablations above) within one controlled framework. For instance, text history adds only +1.7% SR over no memory (71.9% to 73.6%), whereas separate visited markers and our choice-aligned states reach 76.5% and 81.2%. This shows that how memory is represented matters as much as whether it is available. These findings not only ground the design of MarvisNav, but also offer practical guidance for building future VLM-based ZSON memory interfaces.

We hope our response addressed your main concerns, and thanks again for your insightful suggestions. We are excited about how these discussions and improvements can further enhance the quality of this paper, and we look forward to collaborating with you during the discussion.

AuthorsOpenReview ↗

Willing to clarify further concerns you may have.

Dear Reviewer ahEc,

Thank you again for your time and constructive feedback! We have uploaded the revised PDF (with updates highlighted in blue) incorporating your suggestions:

  • The controlled comparison you proposed: the same four exploration states and consistent prompts, presented as text, separate markers, and candidate coloring(Sec. 5.3, Table 3)
  • Comparison with the Mem2Ego-style visual memory interface under the same VLM and navigation settings (Sec. 5.2, Table 2, Appendix E)
  • Contributions and distinctions from Mem2Ego and OpenFrontier(Sec. 1, Sec. 2)

We would greatly appreciate it if you could review our responses and let us know if they address your concerns. Your proposed comparison now forms a dedicated subsection, and it shows that directly aligning the states with the candidate choices accounts for most of the gain, while richer state information provides complementary gains. If you have any further questions or comments, we would be delighted to engage in discussion. Thank you once again for your time and consideration!

Comments on the paper

1 comment

As the rebuttal phase comes to a close, we would like to sincerely thank all the reviewers for the constructive discussions and invaluable suggestions during this period. Your feedback and engagement have been instrumental in improving our work and strengthening its contribution to the field. We truly appreciate the time and effort you have dedicated to this process! Thank you so much!

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.