- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
AnchorNav studies long-horizon navigation from game images and language instructions. It builds a landmark-and-route map from gameplay videos, uses place recognition to correct online position estimates, and projects the next target into the current image. A local controller follows this waypoint together with a language sub-instruction and produces keyboard–mouse actions. The controller is trained on 350 hours of gameplay. Experiments cover long-horizon traversal and short instruction following in Black Myth: Wukong, together with an evaluation on RxR.
Critical strengths
- The problem is useful, and the system design is clear. Long-horizon navigation requires both remembering visited locations and translating route instructions into actions in the current view. The map and image-space waypoint address these two needs.
- The long-horizon experiment demonstrates system usability. The full system completes the spawn-to-boss traversal in five of eight runs.
- The evaluation includes a public benchmark with a strong reported success rate. On RxR val-unseen, the paper reports a 61.65% success rate, higher than the reference results listed in Table 3.
Critical weaknesses
- Sections 3.2–3.3 select the next target through graph search and project its 3D position into the image. However, Section 4.3 states that the graph contains no nodes for the RxR validation scenes. The paper does not explain how the target's 3D position and image-space waypoint are generated at test time in this setting.
- AnchorNav uses a map built from offline gameplay videos, while the baselines receive map prompts and can consult the in-game map. The differences in coverage, available information, and construction cost are not quantified. Table 1 therefore does not establish whether the advantage persists when the methods receive the same environmental information.
- The game-navigation metrics are not sufficiently specified. The paper does not clearly specify the calculation procedures for PCR and RVR or the arrival criterion for OSR in the game experiments.
- Table 1 reports only the Chapter 1 navigation task, with eight runs per method. The evidence is insufficient to establish reliability across a broader range of long-horizon tasks.
- The long-horizon experiment lacks existing navigation-method baselines. Table 1 compares only four zero-shot general-purpose VLMs. The retrained DualVLN and ActiveVLN baselines in Table 2 are not evaluated on the long-horizon task. The results demonstrate an advantage over these VLMs but do not establish an advantage over existing navigation methods.
Questions for authors
- Without stored nodes for the current RxR validation scene, how are the next target's 3D position and image-space waypoint generated at test time?
- Section 4.2 reports 250 trials per difficulty, but multiplying 250 by the reported success rate of 53.8% gives 134.5 successes, which is not an integer. How was this success rate calculated?
- Are the VLM baselines in Table 1 allowed to build, update, or query a topological map? Can at least one baseline receive the same landmark and route information and query interface as AnchorNav for a navigation comparison?
- How are PCR, RVR, and game OSR calculated?
- How much video is used for map construction, and how long does processing it take?
- Please specify each VLM baseline's prompt, observation history and memory management, action interface, and available tools in Table 1. What reasoning settings, token budgets, and model-call budgets are used?
- Can at least one navigation baseline already retrained for Table 2 be evaluated on the long-horizon task in Table 1, with its adaptation for long-horizon goals and action execution specified?
8 comments
Response to Reviewer i9JJ(2/2)
Q6.What are each Table 1 VLM baseline's prompt, observation history, memory, action interface, tools, and reasoning, token, and model-call budgets?
The four VLM baselines in Table 1 use the same prompt, observation history, memory, tools, and action interface; the complete prompt is provided in the Appendix E.2. It describes the game environment, output format, keyboard–mouse action mapping, recovery behavior when the agent becomes stuck, stopping criterion, and use of the map. Each model receives the 20 most recent frames together with the corresponding action history. Its memory contains a complete game map annotated with the start, goal, and a hand-drawn optimal route. In addition, the system opens the in-game map every two minutes, allowing the model to verify the character’s current position. The action interface outputs strings in the format X Y;k1 k2;k3;k4 k5..., where X and Y specify horizontal and vertical view rotations mapped to mouse movements, and each subsequent segment specifies the keyboard or mouse buttons pressed simultaneously. Every output contains at least five actions, each executed for 100 ms. The reasoning instruction asks the model to briefly describe only the obstacles immediately ahead and the direction indicated by the map. Each request uses approximately 2,900 input tokens and 50 output tokens, while a full-map evaluation run consumes approximately 5 million tokens. The estimated cost per run is \27 for GPT-5.5, \27 for Claude-Opus-5.
Q7.Can at least one navigation baseline retrained for Table 2 be evaluated on the long-horizon task in Table 1?
We thank the reviewer for this suggestion. We conducted an ablation study in which our local anchor was replaced with ActiveVLN while the global anchor remained unchanged. We evaluated each controller over eight independent runs on the same long-horizon route, following the protocol and two-hour per-run time budget of the main Table 1 experiment.
| Case | PCR-F (↑) | Succ. (↑) | TTS (min) (↓) |
|---|---|---|---|
| Local anchor replaced with ActiveVLN | 11.5 | 0 | -- |
| Ours | 96.9 | 62.5 | 62.29 |
The substantial performance drop indicates that ActiveVLN struggles to ground and execute the language-only instructions issued by the global anchor. One key source of error is referential ambiguity. As illustrated in Figure 6, when several similar pillars are visible, the instruction “Move near the stone pillar at the front left” may not uniquely identify the intended target. Our local anchor resolves such ambiguity by jointly grounding the text instruction and an image-space waypoint, leading to substantially better long-horizon navigation performance.
Thank you for the detailed response. I now better understand the experimental setup and the role of each component, and the authors have addressed my original questions.
My remaining concern is the limited evaluation scope. The main experiments are conducted in a single game, while the proposed framework relies on game-specific gameplay data, an offline topological map, and a trained local controller. It would therefore be helpful to clarify whether the framework can transfer to other games, or whether the map and local controller need to be rebuilt and retrained for each new game.
I would also like to ask whether the proposed tools and interfaces can be directly used with stronger frontier models, such as GPT-6 Astra. In particular, can such a model replace the current planner or local controller while using the same topological memory, re-anchoring, and waypoint interface? If so, it would help clarify whether the framework provides a model-agnostic navigation architecture and whether stronger foundation models could achieve substantially better performance.
Response to Reviewer i9JJ(1/2)
We thank Reviewer i9JJ for the valuable comments. We have included examples of actual code execution in the Supplementary Material and revised the manuscript according to the comments. Please find our response below.
For a fair comparison on RxR, we do not provide our model with topological memory or stored 3D node coordinates. At test time, the global-anchor LLM selects the next visual target from the task instruction and current observation, then outputs a language sub-instruction and the target’s image-space coordinates. For example, when the agent must leave a room, the planner places the waypoint on the visible door. The waypoint is then rendered directly onto the current frame and passed, together with the sub-instruction, to the local controller. The controller subsequently issues VLN-CE atomic actions, such as MOVE_FORWARD and TURN_RIGHT.
Thank you for pointing out this issue; the number of trials reported in the paper is a typo. For each difficulty level, we used 50 tasks and tested each task 10 times, i.e., 500 trials per difficulty level (1,500 trials across the three difficulty levels), rather than the 250 trials per difficulty stated in the paper. With 500 trials per difficulty, the reported success rate of 53.8% corresponds to 269 successful trials, which is an integer. This has been corrected in the paper.
The geometry-guided topological memory is a contribution of this work and is not available to the four general-purpose VLM baselines in Table 1; they neither build, update, nor query this graph. Instead, we provide each baseline with position, route, and destination information in its prompt. Specifically, we annotate the route and destination on a saved game map and include it in the VLM’s context. During execution, the baselines may also open the in-game character map to estimate their current position and compare it with the annotated route.
We have provided a more detailed explanation of the evaluation metrics in the Appendix.
PCR denotes the path coverage rate, measuring the proportion of the reference route traversed by the agent. We uniformly sample points along the ground-truth route and count a point as covered if its minimum distance to the test trajectory is within 15m. PCR is computed as the number of covered reference points divided by the total number of sampled reference points. PCR@15 reports coverage after 15 minutes of navigation, whereas PCR-F reports the final coverage at the end of the full evaluation horizon.
RVR is the revisit rate, which measures the proportion of backtracking. It is computed on the test trajectory: we divide the trajectory into N grid cells and traverse the trajectory from start to end, where entering a new cell counts as one entry and later re-entering a previously left cell counts as one revisit. RVR = number of revisits / total number of merged entries. A lower RVR indicates less backtracking.
OSR is the oracle success rate, which measures whether the goal is reached, i.e., whether the character trajectory reaches the goal point. If the agent reaches the goal but does not stop, it is counted in OSR; if it reaches the goal and stops, it is counted in SR. OSR = number of test samples reaching the goal / total number of samples. A higher OSR indicates a higher probability of passing the goal and thus stronger navigation ability.
We used 75 offline videos totaling 1.84 hours for map construction. Processing these videos to generate the geometry-guided topological memory took 4 hours, of which gemini-3.1-pro spent 2.5 hours on node selection, node annotation, and edge annotation, and VGGT-Omega spent 1.5 hours on four GPUs adding geometry information to the topological memory. This information has been added to the Appendix.