acceptodds
Under review as a conference paper at ICLR 2027

VCN-Bench: A Video-Contextualized Navigation Benchmark for Spatial Reasoning over Prior Visual Experience

Abstract

Spatial reasoning is fundamental to embodied agents, yet it remains unclear whether spatial understanding can be carried forward to guide sequential interactions. Existing spatial-reasoning benchmarks typically terminate at offline predictions, while navigation benchmarks evaluate spatial reasoning as part of instruction following and exploration. We introduce **VCN-Bench**, a **V**ideo-**C**ontextualized **N**avigation benchmark for probing closed-loop spatial reasoning over prior visual experience in MLLMs. Given a prior video covering both the initial location and destination, the agent is tasked with reasoning out the instruction-specified target and navigating toward it with the inferred spatial context. Built on Matterport3D, VCN-Bench contains five instruction types, 100k training episodes, and 1,250 evaluation episodes. Navigation serves as the primary evaluation, while diagnostic goal identification helps distinguish destination-resolution errors from subsequent navigation failures. We further propose MV-DualVLN, a planning-oriented baseline that jointly leverages prior video and in-episode observations. Experiments reveal limited navigation performance, a substantial destination-resolution-to-navigation gap, and frequent navigation failures even after correct destination identification.

Primary area:
datasets and benchmarks
Keywords:
spatial reasoning, embodied navigation, multimodel large language model, video-contextualized navigation

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_E9voOpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces VCN-Bench, where an agent uses a prior video to identify a target and navigate to it. It also proposes MV-DualVLN as a baseline.

Critical strengths

Separating target identification from subsequent navigation is useful for analyzing failures.

Critical weaknesses

1.Only a few navigation methods are evaluated. Testing many MLLMs on goal identification does not make up for the limited coverage of navigation baselines, especially for a benchmark paper.

2.I am not convinced that the paper clearly establishes the value of this new task. What useful insight does adding a prior video provide beyond existing VLN-CE and memory-based navigation tasks? The paper needs stronger evidence that this is a meaningful evaluation setting.

Questions for authors

What concrete gap in existing navigation evaluation does this task address, and what evidence shows that the observed difficulty persists across stronger, properly adapted navigation methods?

3 comments
AuthorsOpenReview ↗

Response to Reviewer E9vo (1/3)

W1: Only a few navigation methods are evaluated

We would like to clarify that we have tried our best to evaluate as many navigation models as possible that can be adapted to VCN-Bench, covering both hierarchical and end-to-end paradigms. We also fine-tuned 3D-Mem and MV-DualVLN to better adapt them to VCN-Bench. We welcome the reviewer to provide specific models suitable for evaluation on VCN-Bench; we would be happy to include their evaluation results.

VCN-Bench differs from conventional route-following VLN tasks in several aspects. First, the destination is not explicitly specified by a route description or an object category. Instead, it must be resolved from a prior video according to spatial or temporal relations. Second, the prior video serves as persistent scene context that should be leveraged throughout navigation, rather than a sequence of observations to be directly followed. Third, the agent must make sequential navigation decisions based on this context and its evolving egocentric observations. In contrast, many publicly available VLN models are primarily designed for R2R/RxR-style route-following instructions. Although some goal-oriented navigation models support object-centric tasks, they typically assume an explicitly specified object category, instance description, or target representation, rather than a destination that must first be inferred from a prior video.

Therefore, we select methods for which a meaningful adaptation to our evaluation protocol is feasible, rather than reporting results from incompatible settings. The table below reports the performance of NaVILA on VCN-Bench. As NaVILA is designed exclusively for instruction-following navigation, its results on VCN-Bench are inherently limited. For meaningful comparison, we excluded models that support instruction following only, which accounts for the relatively small number of methods in Table 1.

SPLSRRSR
NaVILA2.694.327.36

Our current evaluation covers complementary navigation paradigms: (1) a hierarchical approach that combines MLLM-based target-frame selection with MTU3D, which is a SoTA model for goal-oriented navigation; (2) end-to-end methods, including zero-shot 3D-Mem and Uni-NaVid, which are inherently capable of both video understanding and navigation; and (3) task-adapted end-to-end methods, including 3D-Mem and our MV-DualVLN, fine-tuned on VCN-Bench training episodes. In particular, the fine-tuned 3D-Mem and MV-DualVLN achieve 18.1% and 27.6% SR, respectively. These comparisons provide evidence across different navigation paradigms.

AuthorsOpenReview ↗

Response to Reviewer E9vo (2/3)

W2: The value of this new task.

The contribution of VCN-Bench is to evaluate whether an agent can resolve a destination from supplied visual experience and use that experience to guide subsequent closed-loop navigation. Its value lies in the new navigation paradigm and the capability being examined, rather than the additional input modality alone.

1. Existing navigation performance does not directly establish this capability.

As stated in L57-63, conventional route-following VLN benchmarks provide instructions describing paths, landmarks, and actions. In many goal-oriented navigation settings, the destination is explicitly specified, while scene knowledge is acquired through exploration. Lifelong navigation further evaluates the reuse of accumulated experience, but the available experience depends on the agent’s previous trajectories and coverage. These settings certainly involve spatial reasoning; however, their aggregate scores do not directly reveal how well an agent can reason over a supplied prior visual experience and translate it into navigation decisions.

2. VCN-Bench makes the role of prior experience explicit.

Each episode supplies a prior video covering the initial location and destination, reducing variation in experience acquisition. In particular, Room-to-Object instructions define the destination through relations within that experience—for example, the second bedroom visited or the farthest bedroom among those shown in the video. The video therefore participates in determining which destination is intended, rather than merely helping the agent search for an already specified target. Moreover, its trajectory is not a prescribed route to replay: the agent must navigate from its actual initial state using changing online observations. The benchmark thus examines both destination resolution and the subsequent use of experience for action.

3. The experiments provide insights beyond showing that the task is difficult.

  • Prior scene context is useful beyond the destination frame. As shown in Table 3 in the main paper, the full-video MV-DualVLN achieves 27.6% SR, compared with 19.7% for the variant trained and evaluated with only the ground-truth target frame. Thus, a correct target reference does not substitute for the broader scene context available in the prior video.

  • Correct destination information does not eliminate navigation failures. As shown in Appendix D.2, replacing predicted goal frames with ground-truth frames throughout navigation in the same trained MV-DualVLN improves SR from 27.6% to 33.1%. With model parameters and the remaining evaluation setup unchanged, substantial failures persist, showing that inaccurate destination resolution is not the sole bottleneck.

Taken together, we believe VCN-Bench provides a meaningful evaluation setting beyond conventional route-following or goal-oriented navigation, by explicitly examining whether spatial knowledge inferred from prior visual experience can be carried forward to guide navigation.

AuthorsOpenReview ↗

Response to Reviewer E9vo (3/3)

Q1: What concrete gap in existing navigation evaluation does this task address, and what evidence shows that the observed difficulty persists across stronger, properly adapted navigation methods?

The gap between VCN-Bench and existing navigation benchmarks is discussed in response to W2. The key distinction is that strength on existing navigation benchmarks does not automatically imply strength in reasoning over prior visual experience. Existing tasks certainly involve spatial reasoning, but evaluate it jointly with route following, exploration, and memory acquisition. Their aggregate scores do not directly establish whether an agent can resolve an experience-dependent destination from a supplied video and use that experience to guide subsequent closed-loop navigation. This is the complementary capability that VCN-Bench is designed to evaluate.

We would like to clarify that our evaluation already includes strong, representative navigation methods—not merely generic MLLMs applied to navigation without adaptation. We select MTU3D, Uni-NaVid, and 3D-Mem, and build MV-DualVLN, covering complementary approaches to goal-oriented navigation, video-based action prediction, explicit memory, and waypoint planning. These methods are explicitly adapted to VCN-Bench, as detailed in Appendix C.3. Beyond zero-shot evaluation, we fine-tune 3D-Mem on VCN-Bench and extend DualVLN into MV-DualVLN to jointly process prior video, navigation history, and multi-view observations. The latter two achieve 18.1% and 27.6% SR, respectively. Thus, substantial difficulty remains even with task-specific training, rather than only under unadapted cross-task transfer.

Importantly, our argument does not rely on low success rates alone. In MV-DualVLN, providing ground-truth goal frames throughout navigation increases SR from 27.6% to 33.1%, showing that incorrect destination identification is not the sole bottleneck. Meanwhile, MV-DualVLN outperforms the target-frame-only variant (27.6% vs. 19.7% SR), demonstrating the utility of prior scene context beyond the target frame itself. Together, these results show that the benchmark probes a meaningful capability not established by performance on conventional navigation tasks, and that the challenge persists across the evaluated navigation paradigms, including task-fine-tuned methods.

Reviewer_tLngOpenReview ↗
Rating
4(+1)
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces VCN-Bench, a video-contextualized navigation benchmark for evaluating MLLMs' closed-loop spatial reasoning over prior visual experience. The benchmark contains five instruction types covering different spatial reasoning capabilities, including counting, ordering, distance, orientation, and instance grounding. The authors introduce goal identification as a diagnostic task and propose MV-DualVLN as a reference baseline. Extensive experiments reveal limited navigation performance and a substantial gap between correct goal identification and successful navigation.

Critical strengths

1. Novel Benchmark and Diverse Task Design. VCN-Bench introduces a meaningful setting that connects spatial reasoning over prior visual experience with closed-loop navigation. The five instruction types cover different spatial reasoning capabilities, providing a diverse and systematic evaluation of MLLMs.

2. Comprehensive Evaluation and Analysis. The paper evaluates multiple proprietary and open-source MLLMs, along with hierarchical and end-to-end navigation baselines. The diagnostic evaluation, controlled ablations, and analysis of the gap between goal identification and navigation provide valuable insights into existing models' limitations.

3. Open-source Resources and Reproducibility. The authors provide anonymized links to benchmark annotations, data generation and evaluation pipelines, and the MV-DualVLN implementation. These resources can facilitate reproducibility and support future research on spatial reasoning and embodied navigation.

Critical weaknesses

1. Insufficient Analysis of Navigation Failures. While conditional success rates and oracle target-frame experiments reveal a substantial identification-to-navigation gap, the causes remain unclear. Analyzing waypoint selection, localization, and stopping errors would provide deeper diagnostic insights.

2. Limited Analysis of Relational Reasoning. Although counterfactual and shuffled-video experiments probe relational sensitivity, counterfactual evaluation retains the original target despite changing the instruction. Performance drops therefore do not establish correct relational understanding. Evaluating against updated targets would provide stronger evidence.

3. Limited Investigation of Task Difficulty Factors. Despite task-wise comparisons, video correlations, and goal-identification error analyses, the effects of distractor similarity, navigation distance, and trajectory complexity remain unclear. Controlled analyses would better explain performance differences across tasks.

Questions for authors

Please see Weaknesses 1–3 for my main questions and concerns.

4 comments
AuthorsOpenReview ↗

Response to Reviewer tLng (1/2)

W1: Insufficient Analysis of Navigation Failures

Thank you for the advice. To investigate the source of the performance gap between goal identification and navigation, we conducted a fine-grained failure analysis on 345 episodes where goal identification succeeded but navigation failed, as shown in figure. Categorizing failures according to the agent’s spatial relationship with the destination, we observed three distinct failure modes:

  • Failure to enter the target room (277 episodes, 80.3%);
  • Entry into the target room without reaching the target object (54 episodes, 15.7%);
  • Failure to stop after reaching the target (14 episodes, 4.1%).

These results suggest that successful navigation to the target room strongly correlates with overall task success. The primary bottleneck resides in how to navigate to the target room.

We further analyse the causes of target-room entry failure. Failure modes are categorized as follows:

  • Decision limit (4 episodes, 0.9%): exceeding the maximum allowed number of navigation steps
  • Execution exception:
    • Pixel out of bounds (63 episodes, 18.3%): the predicted pixel goal is out of range;
    • Execution stuck (20 episodes, 5.8%): the agent becomes immobilized somewhere and fails to reach the model-predicted pixel goal;
  • Path planning error:
    • Approaching target without room entry (7 episodes, 2.0%): the agent continuously approaches the target room but ultimately fails to enter it;
    • Repeatedly backtracking (18 episodes, 5.2%): the agent repeatedly shuttles back and forth within a certain area;
    • Ends in a distractor room (60 episodes, 17.4%): although the selected visual frame correctly identifies the target room type, the agent ends up in a wrong room with the correct category;
    • Other (105 episodes, 30.4%): the agent stops in a region completely unrelated to the target room.

Taken together, these results clarify the nature of the identification-to-navigation gap: selecting the correct destination frame does not imply that the agent can translate that reference into effective navigation from its current state. Among the analyzed failures following correct initial identification, 80.3% occur before the agent ever enters the target room. Thus, the dominant observed failure stage is reaching the target room, rather than only approaching the object or stopping after arrival. The observed backtracking and navigation to same-category distractor rooms are consistent with difficulties in establishing online spatial correspondence and selecting effective routes or waypoints. This analysis is added in Appendix D.3.


W2: Limited Analysis of Relational Reasoning

Thanks for your advice. We agree (as stated in Appendix D.2) that the instruction intervention modifies only the language input to the model, aiming to assess the model's sensitivity to relational keywords. To further evaluate the model's understanding of the relations in the modified instruction, we correspondingly update the navigation goal to reflect the revised relational constraint. However, such modifications do not guarantee a valid goal, and there are three issues:

  1. The room indicated by the modified instruction does not exist. For instance, replacing “before” with “after” in an instruction referencing “the living room visited in the video before the current room” yields no valid target if no living room appears later in the video.
  2. The modified instruction cannot guarantee unambiguity. For instance, substituting “farthest” with “nearest” may yield multiple bedrooms at nearly identical distances, preventing unique identification.
  3. The modified target room does not contain the required target object.

To ensure interpretability and validity of analysis, we retain only those episodes for which the modified instruction admits a valid solution—yielding a curated set of 110 episodes.

InstructionTargetSPLSRRSR
1OriginalOriginal19.5026.3631.81
2ModifiedOriginal3.967.2711.82
3ModifiedModified17.1423.6427.27

The table above presents evaluation results on the 110 instruction-modified episodes, reporting success rates under the original navigation targets (row # 2) and the targets modified according to the instructions (row # 3). It can be observed that SR achieves 23.64% with the modified targets, which is close to the model's 26.36% SR on the original R2O instructions (row # 1). These results are added in Appendix D.2.

AuthorsOpenReview ↗

Response to Reviewer tLng (2/2)

W3: Limited Investigation of Task Difficulty Factors

Thank you for this insightful advice. Task difficulty is quantified along three dimensions: (i) shortest-path navigation distance (in meters), (ii) shortest-path trajectory complexity, defined as the count of directional turns exceeding 30$^\circ$, and (iii) the number of room distractors of the same type as the target room in the prior video.

For each task type, we stratify episodes by difficulty level across these dimensions and report episode counts per stratum in the figure. As shown, Distance and Orientation tasks exhibit longer navigation distances, especially the Orientation task, where more than half of the episodes have a navigation distance greater than 20m and also involve more turns. Order tasks contain substantially fewer room distractors, as the requirement in many episodes is temporal reasoning over visitation sequences of rooms with different categories.

The figure presents the goal identification performance of Qwen3.8-flash, DeepSeek-v4.1-flash, and MV-DualVLN across the three difficulty dimensions. As shown, performance exhibits negligible correlation with navigation distance and trajectory complexity, and only a marginal correlation with room distractor count. In contrast, performance exhibits systematic variation across task types, reflecting intrinsic differences in the model’s capacity to reason over distinct task semantics.

The figure presents the navigation performance of MV-DualVLN across the three difficulty dimensions. In contrast to goal identification, navigation success rates across all task types decline with increasing difficulty along all three dimensions. Since Distance and Orientation episodes feature longer navigation distances, and Orientation episodes also have higher trajectory complexity, this decline has a larger impact on these two task types and helps explain the lower navigation performance on Distance and Orientation tasks.

The analysis is added in Appendix D.4.

I have reviewed the authors’ responses to my comments as well as to those raised by other reviewers, which adequately address my concerns. Accordingly, I am willing to raise my rating.

Thank you for your thorough review and for dedicating time to evaluate our responses and supplementary analyses. Your constructive feedback has strengthened our analysis of the benchmark’s challenges and the capabilities of baseline models.

Reviewer_ZjhCOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces VCN-Bench, which evaluates whether multimodal models can use prior visual experience to identify a destination and navigate to it. Built on Matterport3D, the benchmark provides prerecorded RGB tour videos and five instruction types covering instance identification, visitation count and order, distance, and orientation. It contains approximately 103K training episodes and 1,250 evaluation episodes, with disjoint scenes across the training and evaluation splits. Navigation is the primary task, supplemented by frame-based goal-identification diagnostics.

The proposed MV-DualVLN baseline combines prior video context, navigation history, and multiview observations to predict goal frames and pixel-space waypoints, which are executed by an oracle controller. The experiments compare navigation and goal-identification baselines and analyze the use of prior context, the role of navigation history, and the difficulty of resolving the intended destination.

Critical strengths
  1. The benchmark provides a useful setting for studying navigation based on prior visual experience. Agents receive a shared prior video, and the relational instructions and controlled distractors make this context relevant beyond object-category matching. The five task types, disjoint-scene split, and documented automated and human validation make the benchmark a useful resource for studying this capability.

  2. The evaluation provides useful analysis beyond aggregate benchmark scores. The target-frame-only and history ablations, conditional navigation analyses, and oracle-goal intervention using the same model help separate destination resolution from subsequent navigation. In particular, supplying oracle goal frames improves success rate from 27.6% to 33.1%, while substantial navigation failures remain. This provides a clearer picture of the bottlenecks than the overall success rate alone.

Critical weaknesses

None. I do not identify major weaknesses that would justify rejection.

My remaining concerns are minor clarifications regarding the visual-input protocol, the adaptation of the 3D-Mem baseline, and the distinction from closely related benchmarks.

Questions for authors
  1. Clarification of the visual-input protocol: What frame counts, sampling rules, image resolutions, and frame-index mappings are used for each evaluated model? After sampling, how often are the target and the evidence needed for room-visitation or geometric reasoning retained? A focused audit using only the inputs actually visible to each model would help clarify the scope of the reasoning conclusions.

  2. Clarification of the 3D-Mem adaptation: Does the adapted baseline stop at the centroid of all objects rather than select an individual target object, as in the original navigation protocol? Please clarify whether this accurately describes the implementation. If so, what motivates this change, and how does restoring object-level selection affect the comparison?

  3. Distinction from related benchmarks: How does the capability measured by VCN differ from ESI-Bench’s interactive spatial reasoning and VideoNIG’s tour-video-to-navigation pipeline? Please clarify the additional contributions of video-relative destination resolution and repeated online use of prior context. The blanket characterization of prior spatial benchmarks as offline should also be qualified.

4 comments
AuthorsOpenReview ↗

Response to Reviewer ZjhC (1/2)

Q1: Clarification of the visual-input protocol.

Thank you for raising this question, and we apologize for the missing details.

About visual inputs:

As shown in the table below, goal identification evaluation is conducted on the original video sequence without temporal downsampling; frame indices correspond directly to their positions in the prior video. During navigation evaluation, all end-to-end models use videos uniformly downsampled to 50 frames, and the model receives the downsampled frame indices within this compressed sequence.

When computing the goal identification accuracy of navigation models, the same downsampling procedure is applied, and the indices of the target frames retained after downsampling are recorded (the downsampling process is deterministic). The proportion of frames selected by the model that fall within these indices is then computed. These are updated in Appendix C.3 and C.4.

EvaluationSampling ruleImage resolutionFrame index
Goal identificationNo downsample384 ×\times 384Original index in the prior video
NavigationUniformly downsample to 50 frames384 ×\times 384Index in the downsampled video

Target and evidence retention:

After downsampling, the target frame is preserved in 91.3% of episodes. The criteria for retaining key evidence are as follows:

  • All rooms involved and observations of the initial room are retained;
  • Inter-frame mutual visibility (each frame falls within the field of view of the other frames);
  • The sequential count/order of room appearance in Count and Order instructions is preserved.

Across the test set, 85.6% of episodes satisfy all three criteria and thus retain sufficient key evidence for valid evaluation after downsampling. The target retain rate and evidence retain rate for each task type are shown in the table below.

The retain rate for R2O instructions is relatively high, while that for Instance instructions is lower because the definition of target frames is more stringent. To investigate whether evidence retention affects the final navigation performance, we visualize the navigation performance in the 1,070 episodes in which evidence is retained after downsampling, as shown in figure. The performance on each subtask is similar to that on the full test set (Figure 6 in the main paper), so it does not affect the final conclusion. These are added in Appendix C.3.5.

Task typeTarget retain rate (%)Evidence retain rate (%)
Count95.685.2
Order92.086.4
Distance96.888.8
Orientation98.494.8
Instance73.672.8
Overall91.385.6

Q2: Clarification of the 3D-Mem adaptation

Thanks for your careful reading. We sincerely apologize for the inaccurate description of 3D-Mem’s implementation details in our Appendix C.3.3. We have checked our implementation code. Both the zero-shot and fine-tuned variants adhere to the original 3D-Mem design principles. The confusion arose from an inaccurate account of the waypoint selection procedure: while the navigation waypoint selection function computes the centroid of all objects present in the snapshot (3dmem), the object list within the snapshot is first filtered—prior to invoking this function—to retain only the model-selected target object (3dmem). Consequently, the actual implementation selects the center of an individual target object, aligning with the intended behavior described in the original 3D-Mem implementation.

AuthorsOpenReview ↗

Response to Reviewer ZjhC (2/2)

Q3: Distinction from related benchmarks

We thank the reviewer for this valuable suggestion. We have revised our related work section.

Comparison with ESI-Bench. ESI-Bench primarily evaluates active spatial reasoning, focusing on whether agents can acquire task-relevant evidence through active perception and finally answer the question. In contrast, VCN-Bench focuses on reasoning over an already available visual experience and leveraging the inferred spatial context to support subsequent navigation. Given a prior video covering the initial location and destination, the agent must establish spatial correspondence between its current state and the prior video, resolve the instruction-specified destination, and use the inferred context to guide navigation.

While both benchmarks involve perception-action loops, ESI-Bench emphasizes active information acquisition, whereas VCN-Bench emphasizes context-grounded spatial reasoning and its subsequent utilization in navigation.

Comparison with VideoNIG. VideoNIG formulates a goal-oriented navigation instruction generation task, where the model generates step-by-step instructions for an optimal route conditioned on a given goal description, initial observation, and tour video. VCN-Bench differs from VideoNIG along three key dimensions:

  1. Trajectory alignment: In VideoNIG, the tour video and the instruction specify the same trajectory—the same trajectory that the agent is required to execute during deployment. In VCN-Bench, however, the video trajectory is not the trajectory the agent ultimately needs to follow. The navigation destination must be resolved from the prior video according to video-relative spatial and temporal relations depicted in the instruction. And the agent needs to plan its path towards the destination.
  2. Role of video during navigation: In VideoNIG, the video is used only during instruction generation and plays no role in the navigation phase; the agent relies exclusively on the generated instructions for execution. In VCN-Bench, the prior video remains a persistent context throughout navigation to inform sequential waypoint selection.
  3. Evaluation objective: VideoNIG assesses the agent’s ability to generate fine-grained, step-by-step navigation instructions that faithfully reconstruct the observed video trajectory. VCN-Bench, by contrast, evaluates the agent’s capacity to jointly interpret visual and linguistic inputs to infer a navigation goal and subsequently plan an executable route toward it.

Thank you for the detailed responses and additional analyses. The clarifications regarding the 3D-Mem implementation and the distinctions from related benchmarks adequately address Q2 and Q3.

For Q1, I appreciate the additional sampling analysis. However, only 85.6% of episodes retain all key evidence after downsampling, dropping to 72.8% for Instance tasks. This raises a remaining concern about whether failures on episodes with missing evidence reflect limitations in spatial reasoning or insufficient visual input. The similar aggregate performance on the evidence-retained subset does not fully resolve this issue.

Could the authors clarify how they ensure the validity of these episodes under the actual navigation input protocol?

Overall, the responses are helpful, and I maintain my current rating.

AuthorsOpenReview ↗

Response to Reviewer ZjhC

Thank you for your further feedback. Your comment highlights an important trade-off in long-video processing: reducing computational cost while retaining task-relevant visual evidence. We agree that this trade-off must be considered when interpreting model failures. To fully address your concerns, we (i) first discuss this trade-off, (ii) then clarify the impact of downsampling on our experimental conclusions, and (iii) finally provide an explanation for the relatively low evidence retention rate in instance retention.

1. Trade-off discussion

VCN-Bench provides the full prior video, while the evaluated baselines use temporal downsampling to accommodate computational and context budgets. This introduces a trade-off between processing cost and the retention of task-relevant evidence. VSI-Bench similarly uses 16/32 sampled frames for its evaluated models and explicitly treats frame sampling as a model-design choice separate from benchmark design. The full-set results evaluate the navigation systems under their stated input-processing configurations, including the effects of evidence selection and retention. The benchmark leaves frame selection and memory compression to the evaluated methods, encouraging approaches that better balance computational cost and evidence preservation.

2. Impact of downsampling

2.1 Impact on performance

To better answer this question, we evaluate Qwen3-VL-4B-Thinking on goal identification under three input settings: uniform downsampling to 50 frames, uniform downsampling to 100 frames, and full prior video. Each setting is evaluated on its corresponding retained subset drawn from the same original test set. The figure reports the number of successful samples under each task type. Results demonstrate that, despite variation in evidence retention rate across these conditions, the numbers of successful predictions remain similar.

2.2 Impact on conclusions

Sharing your concern, our paper does not attribute all navigation failures to spatial reasoning alone. Our main conclusions concern the observed difficulty of both goal identification and video-contextualized navigation, as well as the gap between them (L91-92), rather than a claim that every failure has an isolated reasoning cause. Indeed, Appendix E already acknowledges that aggressive downsampling may remove destination evidence or disrupt trajectory structure.

First, destination resolution remains challenging. Our standalone goal-identification evaluation uses the full prior video, rather than the 50-frame navigation input. Its results therefore provide evidence for difficulty in resolving video-grounded destinations that cannot be explained by the navigation baselines’ temporal downsampling.

Second, video-contextualized navigation remains challenging on the retained subset. The navigation performance on the subset in figure is comparable to that on the full dataset.

Third, correct destination identification does not guarantee successful navigation even on the evidence-retained subset. Our subset analysis (figure) is intended to establish this persistence of the identification-to-navigation gap, not to prove that all excluded episodes remain sufficiently specified.

Accordingly, the audit qualifies the attribution of full-set failures, rather than overturning these central findings.

3. Instance retention uses a strict visibility criterion

For Instance tasks, the target retention check requires at least one sampled frame to satisfy the object-visibility definition in Appendix B.3, including the 3 m proximity requirement and sufficient unoccluded visibility. The evidence-retention audit also incorporates this requirement. Consequently, an episode may fail the audit even when the target remains observable from farther away or useful contextual evidence is preserved. The reported 72.8% therefore does not imply that the remaining 27.2% of Instance episodes contain no usable evidence or are necessarily unsolvable.

Instance instructions provide detailed object attributes and spatial relations to surrounding landmarks. For example, for "approach the black stool near the table in the office", downsampling may remove the qualifying close-up view of the stool while retaining views of the table and office, and possibly a more distant view of the stool. This information may still support navigation toward the relevant area, where online observations can provide additional evidence for identifying and approaching the object. Importantly, the close-range visibility criterion used for frame-based goal identification is therefore not a necessary condition for successful closed-loop navigation.

Comments on the paper

2 comments

We sincerely thank the reviewers for their careful evaluation and constructive feedback. We appreciate their recognition of VCN-Bench’s meaningful task setting and diverse task design (ZjhC, tLng), the diagnostic value of distinguishing destination identification from subsequent navigation (ZjhC, tLng, E9vo), the informative evaluation and analyses (ZjhC, tLng), and the released resources supporting reproducibility and future research (tLng). All the suggestions have helped us clarify our contributions and strengthen the experimental analysis. We hope that our point-by-point responses and additional results address the reviewers’ concerns. The corresponding changes are highlighted in blue in the revised manuscript.

Dear Reviewers,

The authors have submitted their rebuttal to address your concerns. As the discussion deadline approaches, please review their response and share any remaining concerns or questions soon so the authors have enough time to respond.

Thank you, -AC

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.