acceptodds
Withdrawn from ICLR 2027

SEDualVLN: A Spatially-Enhanced Dual-System Framework for Vision-Language Navigation

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Vision-Language Navigation (VLN) approaches have currently followed two primary paradigms: the end-to-end Vision-Language Model (VLM) policy fine-tuned on navigation trajectories to directly predict actions, and the zero-shot modular pipeline integrating pre-trained Multimodal Large Language Model (MLLM) for training-free generalization to unseen environments. However, end-to-end methods struggle with long-horizon navigation and lack dynamic reasoning, whereas zero-shot methods are constrained by limited spatial grounding for reliable planning and also require substantial reasoning time. To bridge this gap, we introduce SEDualVLN, a spatially-enhanced dual-system VLN framework. System 1 is a VLM model enhanced with both global and local spatial awareness, used for action generation. System 2 integrates a general MLLM with a mapping module, wherein the MLLM plans waypoints by leveraging top-down views of the real-time 3D map alongside streams of rendered path images. Both systems leverage different forms of spatial enhancement to cultivate the agent's sense of direction in VLN tasks. Ultimately, they cooperate to complete the navigation task through a fast-slow coordinated approach. SEDualVLN achieves state-of-the-art performance on VLN-CE benchmarks, and further ablation studies demonstrate the effectiveness of each system and module.

Primary area:
applications to robotics, autonomy, planning
Keywords:
Vision-Language Navigation, Dual System, Spatially-Enhanced

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_2mQmOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes SEDualVLN, a dual-system framework for vision-language navigation in continuous environments. System 1 builds on StreamVLN and incorporates geometry-feature supervision from a frozen VGGT model, together with passage-focused visual representations obtained using GroundingDINO and SAM. System 2 constructs an online 3D map, renders views along candidate frontier paths, and uses a general-purpose multimodal model to select navigation waypoints. The two systems operate at different frequencies to combine action generation with deliberative planning. Experiments on R2R-CE and RxR-CE report improvements over the included baselines, supported by component ablations and qualitative real-world demonstrations.

Critical strengths
  1. A well-motivated combination of complementary spatial representations. Geometry supervision and passage-focused features target local action decisions, while map-based candidate-path visualization supports higher-level planning. Providing the planner with rendered route observations is an interesting way to connect language instructions with spatial choices.

  2. Consistent improvements on two established VLN-CE benchmarks. Compared with the reported DualVLN baseline, SEDualVLN improves success rate by 3.0 percentage points on R2R-CE and 2.5 points on RxR-CE. The component ablations also suggest that geometry supervision and passage-focused representations contribute to performance.

Critical weaknesses
  1. The methodological novelty is insufficiently established. Fast–slow navigation architectures already exist in DualVLN and R3. Moreover, the global spatial enhancement closely resembles GLaD [1], which aligns the LLM’s visual-token hidden states with frozen VGGT features for VLA. GLaD is neither cited nor discussed, and the paper does not establish a substantive methodological advance over this prior geometry-distillation approach.

  2. The ablations do not adequately isolate the core spatial-enhancement mechanisms. System 1 ablations demonstrate the utility of geometry supervision and passage-focused features, but do not distinguish their benefits from generic auxiliary supervision or additional visual features. More critically, System 2 lacks comparisons isolating the top-down map, candidate-path rendering, and two-stage reasoning. The standalone-versus-combined system comparison supports collaboration, but does not establish the contribution of the proposed rendering-based planning mechanism.

[1] GLaD: Geometric Latent Distillation for Vision-Language-Action Models

Additional Concerns

  1. Real-world evaluation is purely qualitative. The real-world section reports only a few qualitative rollouts in Figure 4, with no success rate, trial count, or baseline comparison. Claims such as “conclusively demonstrate” and “bridges the sim-to-real gap” are not supported by this evidence and should be substantially softened or backed by quantitative trials.
  2. The comparison table is narrow. Table 1 includes only a handful of baselines, while the related-work section cites many recent VLN-CE methods. Including more of these—or justifying their exclusion—would make the state-of-the-art claim more convincing.
Questions for authors
  1. What substantive technical advances distinguish the global spatial enhancement from GLaD-style geometry distillation?
  2. What evidence isolates the contributions of top-down maps, candidate-path rendering, and two-stage reasoning in System 2 under comparable model and inference budgets?
  3. For System 1, what evidence shows that the gains specifically arise from geometric supervision and passage information, rather than generic auxiliary supervision or additional visual tokens?
  4. How are selected waypoints translated into actions, and how are plans updated when the agent moves during asynchronous reasoning?
  5. Are all geometric inputs, including those used for voxel pruning, mapping, and traversability estimation, derived exclusively from observed RGB frames?
  6. Answers to the additional concerns.
Reviewer_3gJhOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

SEDualVLN combines a StreamVLN-based fast policy with a slower mapping and frontier-selection system. The fast policy incorporates VGGT feature alignment and passage masks from Grounding DINO and SAM. The slow system uses LingBot-Map, frontier planning and an MLLM to choose targets. The paper reports strong R2R-CE and RxR-CE val-unseen performance, component ablations and a qualitative robot demonstration.

Critical strengths
  1. The reported success rates on R2R-CE and RxR-CE val-unseen are strong. Combining fast action prediction with a slower spatial planning system is a reasonable way to address long-horizon navigation.

  2. The paper attempts to examine the contributions of spatial enhancement and the dual-system design through ablations, and includes a robot demonstration. If the comparisons can be clarified, these could provide useful evidence for the integrated design.

Critical weaknesses
  1. Several apparent numerical inconsistencies need clarification before the ablations can be interpreted confidently. The reported RxR improvement of 14.4% SR and 11.3% SPL does not match Table 2's 58.5→63.9 and 49.5→52.4 values. The RxR row for 'w/o GSES' in Table 2 also matches 'System 1 only' in Table 4, despite apparently different configurations. Other repeated entries and the Table 4 AT values need explanation. These observations are requests to check the reporting, not evidence by themselves of misconduct, but they materially affect confidence in the component-level conclusions.

  2. The technical contribution and the source of the gains remain insufficiently isolated. The system combines several existing geometry, perception, mapping and navigation components, with limited comparison to closely related geometry-enhanced VLN methods. Switching the MLLM changes SR little, but this alone cannot establish whether the gain comes from MLLM reasoning or the shared mapping/frontier machinery. A non-MLLM frontier-selection control and explicit pose/odometry assumptions are needed to interpret the reported advantage. I currently lean toward rejection pending these clarifications.

Questions for authors
  1. Can you reconcile the textual RxR improvements with Table 2, explain which Table 2/4 configurations share evaluations, and check the repeated RxR rows and AT column? Please provide corrected results where necessary.

  2. What pose information and low-level controller does System 2 use? Does it use simulator ground-truth pose or odometry, and are competing methods given the same information? Please define the 20:1 scheduler and explain how the systems' actions and waypoints are reconciled.

  3. How does the full system compare with the same map/frontier planner using a simple non-MLLM selection rule? Which comparisons isolate the proposed spatial enhancement from related methods such as JanusVLN/DyGeoVLN? Please also identify the model and representation meant by 'layer 32'.

Reviewer_XSCvOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary
  1. Existing VLN either fine-tunes end-to-end models to predict actions directly, which is weak on long-horizon and dynamic reasoning, or uses zero-shot MLLM planning, which has limited spatial grounding and is slow. This paper proposes SEDualVLN, which connects the two in a fast–slow dual system for continuous VLN.
  2. System 1 is a spatially enhanced VLM that generates low-level actions at high frequency, while System 2 follows a Mapping–Rendering–Reasoning pipeline and uses 3D maps with path renderings for GPT-4o to perform low-frequency waypoint planning.
  3. On R2R-CE/RxR-CE Val-Unseen, it achieves strong SR/SPL and surpasses DualVLN and other compared methods.
Critical strengths
  1. Addressing the limited long-horizon spatial awareness of end-to-end VLMs and the slow, weakly grounded planning of zero-shot MLLMs, SEDualVLN separates high-frequency atomic actions from low-frequency waypoint planning in a fast–slow dual system, retaining both execution efficiency and global spatial reasoning for continuous VLN.
  2. On top of StreamVLN, it adds global VGGT feature alignment and local passage-connectivity enhancement, strengthening spatial representations without using depth as an inference-time input.
  3. System 2 builds a map, renders views along frontier paths, and has GPT-4o select targets against the instruction, supplying 3D geometry and visualized paths directly to the slow planner and thereby supporting spatial decisions that are hard to make from text alone or fragmented 2D cues.
Critical weaknesses
  1. The fast–slow dual-system design is not particularly novel, StreamVLN backbone, GPT-4o, and mapping are existing components. What is newly added is mainly VGGT alignment and DINO+SAM passage tokens in System 1, plus the map–render–reason pipeline in System 2. Relative to the prior dual-system DualVLN, the gains are only about 3 / 2.5 SR, the main contribution of the method is the introduction of spatial alignment.
  2. The slow system uses an online 3D map, frontiers, A* paths, and can render novel views along those paths for GPT-4o. This provides much richer information than most RGB-only VLM baselines. The gains may therefore come mainly from mapping + rendering + a strong MLLM, rather than from the spatially enhanced System 1.
  3. The experiments mainly report scheduler frequency ratios for the fast and slow systems. How System 2’s selected waypoint constrains System 1’s atomic actions, which system has priority under conflict, and whether replanning occurs are not clearly specified, so it is unclear whether the two systems are truly coupled or merely time-switched modules.
Questions for authors
  1. System 1 alone already reaches 62.8% SR, and adding System 2 raises it to 67.3%; the improvement over DualVLN (64.3%) is limited. It would help to separate spatially enhanced System 1 alone vs DualVLN, and the same System 1 with different slow systems.
  2. Local Spatial Enhancement runs Grounding DINO + SAM at inference time, but latency, failure rate, and causal contribution to navigation errors are not analyzed; it is also unclear whether passage detection errors can mislead the agent.
  3. Do the Table 2/3 ablations always keep System 2 enabled? If so, the gains of the local/global spatial modules are entangled with the slow system.
  4. The scheduler’s inputs and outputs, and whether it is learnable, are unspecified, this determines how the fast and slow systems are connected.
  5. Local enhancement is presented as a core module, yet how the passage-mask encoding is fused with the original RGB tokens is unclear, the main paper only mentions an MLP projection.
  6. The decision rules of System 2 are underspecified, for example how GPT-4o compares many frontiers and whether it falls back to System 1 upon failure.
  7. For VGGT alignment, which tokens are supervised, how VtV_t and StS_t are spatially–temporally aligned, and how α\alpha is chosen, are not clearly stated.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.