acceptodds
Withdrawn from ICLR 2027

Scalable Native Agentic Intelligence for Long-Horizon Vision-and-Language Navigation

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Vision-and-language navigation (VLN) requires agents to operate in unknown environments over long horizons while coordinating perception, memory, planning, and action. Existing agentic approaches typically orchestrate multiple specialized models, resulting in substantial pipeline complexity and inference overhead. We present *SNAIL-VLN*, a native agentic framework in which a shared multimodal language model jointly learns active exploration, action prediction, spatial perception and memory, and task planning and scheduling, with a runtime harness integrating these capabilities in a closed loop. To enable joint capability learning, we construct a spatially annotated dataset from 153 indoor scenes, including region-topology supervision and 4,742 DFS-guided exploratory expert trajectories, and curate five types of capability-specific question-answer pairs for joint training. Experiments on LH-VLN demonstrate that SNAIL-VLN achieves a new state of the art, improving the multi-stage task success rate from 2.44% to 11.40%. Ablation studies confirm the contribution of each atomic capability, while scaling experiments show that increased model capacity generally improves navigation performance, particularly on multi-stage tasks and subtask success. Code, model checkpoints, and data will be publicly released to facilitate further research.

Primary area:
applications to robotics, autonomy, planning
Keywords:
Vision-and-Language Navigation, Embodied AI, Agentic Models, Active Exploration, Long-Horizon Planning

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 4 reviews, above 27% of the others.
Reviewer_8niUOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes SNAIL-VLN for long-horizon vision-and-language navigation. The framework jointly trains a shared multimodal language model for exploration, action prediction, spatial understanding, and task planning, while a runtime harness maintains spatial memory and coordinates subtask execution. To support training, the authors construct spatial-topology annotations and DFS-guided exploration trajectories, from which they derive five types of QA supervision. Experiments on LH-VLN show improved navigation performance, with a multi-stage success rate of 11.40%. The paper also provides runtime component ablations and model-scaling experiments to study the contributions of the system components and model capacity.

Critical strengths
  1. The paper involves substantial work in data construction, model training, and system implementation. It provides useful practical experience on adapting pretrained multimodal models to VLN and building a harness to support long-horizon navigation.
Critical weaknesses
  1. The experiments are limited to training and evaluation in Habitat, without evaluation in other simulators or on real robots. Although the method improves over the reported baselines, the absolute success rate remains low: most configurations have an SR below 10%, and the full system reaches 11.40%. These results are still far from reliable task completion. Given the small test set and low success rates, the results may also be sensitive to randomness, and repeated runs or confidence intervals would help assess their stability.
  2. Although the related work section covers several agentic VLN methods, the specific differences between the proposed harness and existing VLN harnesses are not sufficiently clear. A more detailed comparison of their designs and capabilities is needed to clarify the novelty of the proposed system.
Questions for authors
  1. Is DFS the most suitable strategy for constructing exploration trajectories? For example, when searching for food, an agent should reasonably prioritize the kitchen or living room, whereas DFS may visit the bathroom first. Although the dataset includes semantic region annotations, DFS itself does not prioritize regions according to their relevance to the target. Could this limit the model's ability to learn goal-directed semantic exploration? A comparison with a target-aware exploration strategy would be helpful.
  2. What challenges do the authors expect when transferring this system to the real world?
  3. With close to 90% of full episodes still failing, what are the main causes of failure? It would be useful to break down failures into categories such as exploration, spatial memory, subtask management, and action execution, and discuss which direction is most important for further improvement.
Reviewer_xBvuOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

SNAIL-VLN trains a shared multimodal model for long-horizon navigation using five QA supervision tasks derived from region-topology annotations and DFS-guided exploratory trajectories. At inference, a runtime harness maintains spatial memory and manages subtasks by calling the model's learned capabilities. The paper reports gains on a filtered LH-VLN test set, along with runtime component ablations, model-size comparisons, and request-level latency measurements.

Critical strengths
  1. The supervision tasks have clear roles in the navigation system: region identification and same-region judgment support spatial memory, while target visibility judgment supports stopping and subtask transitions.
  2. DFS-guided supervision exposes the model to exploration rather than only shortest paths. The runtime ablations also show that spatial memory, subtask decomposition, and target visibility judgment each contribute to performance.
Critical weaknesses
  1. Sections 3.3 and 4.1 describe endpoint-visibility filtering and the resulting 411/54/114 train/validation/test episodes. What remains unclear is whether every baseline in Table 1 was run on the same 114 test episodes with the same observation and action budgets. The paper should identify the rerun results and release the retained episode IDs and filtering implementation. Results on the unfiltered test set, or at least a characterization of the excluded episodes, are also needed: target visibility determines both inclusion in the test set and the proposed stopping decision.

  2. Table 3 removes runtime components from a trained system. As the authors acknowledge, this tests their use at inference, not the benefit of learning the capabilities jointly. The joint-learning claim needs training-objective ablations and a navigation-only model evaluated with the same harness, controlling training budget and data exposure. A comparison with specialist models under a matched budget would also help separate shared learning from additional supervision and orchestration.

  3. Evaluation is limited to one multi-stage benchmark, so it is hard to tell whether the gains extend to long individual routes. An additional compatible benchmark, such as R2R-CE or RxR-CE, with results broken down by route length would address this gap. On LH-VLN, stage count and within-stage route length should be analyzed separately. These factors test different abilities: managing a sequence of subtasks and navigating a long route within one subtask.

  4. Which backbone, model size, training setup, and evaluation conditions produce the 11.40% SR in Table 1? Table 4 reports 8.77% for InternVL3.5-8B, and the reason for the difference is not clear. A smaller reporting issue concerns efficiency: Table 5 gives request-level latency, but spatial-memory matching can require multiple calls. The number of calls and wall-clock time per episode would be more informative about system cost.

Questions for authors

Please see the weaknesses.

Reviewer_iGW5OpenReview ↗
Rating
2
Confidence
4
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

SNAIL-VLN aims to address issues such as context redundancy and task management difficulties faced by existing VLN methods in long-horizon, multi-stage navigation tasks. The authors decompose navigation capabilities into five QA-supervised tasks and jointly train them using a unified Causal Language Modeling objective. The model first breaks down the long-horizon navigation instruction into ordered sub-tasks, while the harness maintains a sub-task queue and a spatial topological memory graph. During each navigation decision cycle, the model performs region identification and panoramic matching; the harness then updates the spatial memory based on the matching results and subsequently invokes the model to predict action sequences. The authors conduct experiments on the LH-VLN benchmark, with multiple ablation studies validating the effectiveness of the proposed method.

Critical strengths

This paper investigates multi-capability collaboration in long-horizon VLN, further abstracting long-horizon navigation into multiple interrelated atomic capabilities and attempting joint learning via a unified model. The work proposes a relatively clear multi-capability joint learning framework and constructs an effective dataset for training. The experimental results preliminarily demonstrate the importance of task decomposition, spatial memory, and completion status verification in long-horizon navigation.

Critical weaknesses

In addition to insufficient completion, the paper also suffers from the following issues:

  1. The paper summarizes its main contribution as "Native Agentic Intelligence," yet it still exhibits prominent similarities in system-level capability orchestration compared to existing agentic VLN methods.
  2. The relationship among different training tasks remains unclear. If two capabilities are trained using the exact same type of inputs, outputs, and supervision signals, how does the model distinguish between active exploration and routine navigation decisions?
  3. The current spatial memory mechanism relies too heavily on region categories and panoramic matching, while region categories lack uniqueness.
  4. Details regarding the construction of the five types of supervised datasets are lacking.
  5. The experiments are insufficient; current experiments only preliminarily validate the importance of exploratory trajectories and runtime modules.
Questions for authors

In addition to the issues mentioned in the weaknesses, I would like the authors to address the following questions:

  1. Compared with existing VLN methods adopting the agent paradigm, what is the essential innovation of this paper?
  2. Does the training phase use ground truth spatial topology? How do you avoid the information discrepancy between training and inference?
  3. How does the same-region judgment moudule ensure matching reliability in repeated rooms and visually similar scenes?
Reviewer_xYCHOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

SNAIL-VLN trains a single multimodal model to explore, predict actions, recognize places, and manage multi-stage navigation instructions. A runtime controller maintains a graph of visited regions across subtasks and checks whether the current target is visible before executing a predicted stop. The authors annotate 153 indoor scenes and use 4,742 exploratory trajectories to construct five types of question-answer supervision. They report 11.40% full-task success on a visibility-filtered LH-VLN test set and study the effects of exploration data, runtime components, model size, and inference latency.

Critical strengths
  1. A shortest-path expert knows where the target is, while a deployed agent must decide which unfamiliar room to enter. Using the same 411 source episodes, DFS-guided supervision raises full-task success from 1.82% to 3.64% (Table 2). This is a modest but direct test of the motivation for collecting exploratory trajectories.
  2. Recognizing a room from its panorama lets the agent reuse what it learned on an earlier subtask, for example when an instruction sends it back to an office. Disabling memory in the same trained checkpoint lowers full-task success from 11.40% to 4.39% (Table 3). This gives the memory design empirical support independent of the comparison with prior systems.
Critical weaknesses
  1. Section 4.1 retains 114 test episodes for which the target is visible at every subtask endpoint. All four FantasyVLN entries in Table 1 match the values published under its original evaluation protocol. For a single unweighted pass over 114 episodes, 13 successes round to 11.40%, whereas no integer success count rounds to 2.44%. An averaging procedure could explain this. The episode set and aggregation rule matter because the claimed improvement from 2.44% to 11.40% depends on the two rates being comparable. FantasyVLN, §4.1 and Table 1
  2. The paper shows that a single model can perform all five supervised tasks. Table 3 measures what happens when the controller loses access to a capability after training. That leaves open whether joint training improves the individual capabilities or their coordination compared with training them separately on the same data. This distinction matters for the paper's conclusion about the benefits of shared representations.
Questions for authors
  1. Which Table 1 baselines were rerun on the same 114 episodes? For SNAIL-VLN and FantasyVLN, could you give the success counts, denominators, and any averaging procedure? Please also clarify whether the agents had equivalent visual access and how the panorama capture in Algorithm 1 counted toward the action budget.
  2. Do you have a comparison that keeps the supervision, backbone initialization, and runtime controller fixed while varying whether the tasks share trainable parameters? If this comparison is unavailable, would you limit the claim to the demonstrated feasibility of serving these capabilities from one model?
  3. When decomposition is disabled, what happens at the first executed STOP? Can the agent mark an intermediate target as complete and continue with the rest of the instruction? This detail changes how I interpret the 0.00% full-task success alongside 14.97% independent subtask success in Table 3.
  4. Table 5 reports 0.983 seconds per model request. A decision cycle can involve several panorama comparisons before the model predicts an action. Do the evaluation logs give the mean number of calls per episode and wall-clock time per episode under a common step budget? These figures would help assess the practical cost of the controller.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.