acceptodds
Under review as a conference paper at ICLR 2027

NaviVLN:Learning Task-Centric Hierarchical Representations via Embodied Alignment for Dual-System Vision-Language Navigation

Abstract

Vision-Language Navigation (VLN) integrates spatial perception, instruction understanding, and sequential decision-making. Foundation-model-based policies benefit from multimodal supervision, yet insufficient task relevance and cross-modal correspondence can cause task-agnostic intra-modal redundancy and cross-modal semantic misalignment, limiting sample efficiency and decision consistency. We present NaviVLN, a dual-system policy built on THREAD (Task-centric Hierarchical Representations via Embodied Alignment for Dual-system), aligning vision, language, and trajectories through data, model, and training. THREAD fuses navigation information into mutually informative visual and textual targets grounding instruction semantics and task progress. Joint Attention integrates implicit visual and textual chain-of-thought during training, shaping shared representations through auxiliary and navigation supervision. Staged training connects these representations to a local trajectory expert; deployment discards auxiliary decoders, using asynchronous intention querying for efficient execution. On R2R-CE Val-Unseen, NaviVLN achieves approximately 10% higher SR than Uni-NaVid using roughly 10% of its reported training data. Ablations show absolute SR gains of 8.49% and 4.30% over action-only and unaligned multimodal baselines, respectively, supporting the effectiveness of full-pipeline task alignment. Further gains with increasing training data suggest strong scaling potential.

Primary area:
applications to robotics, autonomy, planning
Keywords:
Vision-Language Navigation, Multimodal Representation Alignment, Implicit Reasoning

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_UZA2OpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes NaviVLN, a dual-system navigation framework combining task-aligned visual/textual auxiliary targets, Joint Attention, and staged training. Auxiliary decoders shape navigation representations during training and are removed at deployment; latent intention queries condition a local trajectory expert. With 330K reported decision anchors, the method improves over NaVid and Uni-NaVid on R2R-CE and RxR-CE.

Critical strengths
  1. The framework is well motivated. Progress-aware text and task-aligned semantic visual targets provide complementary supervision, while removing auxiliary decoders at inference is a practical design choice.
  2. The architecture comparisons support Joint Attention. With auxiliary targets and loss settings held fixed, Joint improves R2R-CE SR from 47.09% for Separate to 51.39%, and SPL from 44.52% to 48.03%. Representation analyses provide supporting evidence, and the 110K/220K/330K experiments demonstrate gains from additional training data.
Critical weaknesses
  1. The integrated architecture is supported more clearly than its specific design choices. Separate/Merged/Joint comparisons do not independently isolate the navigation anchor or stop-gradient. The contribution of task-aligned targets relative to less task-specific auxiliary targets, and that of staged training, also remains uncertain. This makes it difficult to determine which choices account for the advance beyond the broader auxiliary-supervision and hierarchical-navigation recipe.
  2. Comparative sample efficiency is not established under matched conditions, and missing configuration details hinder assessment. Table 1 contrasts decision-anchor counts with reported or estimated counts from methods with different data compositions, models, and inputs. The internal scaling curve shows improvement with more data, but not superiority over a matched baseline. The provided PDF also lacks the exact VLM checkpoint, numerical loss weights, optimizer settings, and training steps or epochs for each stage.

Overall, the ablations and results support a useful integrated method, but the contribution of its specific choices and the comparative sample-efficiency claim remain insufficiently established. Together with the missing training configuration, these concerns lead me to a weak reject.

Questions for authors
  1. Could the authors prioritize matched ablations of the navigation anchor and stop-gradient? If available, comparisons with less task-specific targets or a simpler training schedule would help explain the sources of the gains.
  2. Please define a decision anchor and report data use and repeated exposure across all stages, together with the missing model/training configuration and the exact representation-scoring protocol. What evidence supports comparative sample efficiency under matched backbone, supervision, and training-budget conditions?
5 comments
AuthorsOpenReview ↗

Response to Reviewer UZA2 — W1 & Q1

We thank the reviewer for the balanced assessment, and we agree with the framing: the integrated architecture is better supported than its individual choices, and the comparative sample-efficiency claim was not established under matched conditions. We have tried to close both gaps with evidence rather than with description. Table 2 removes each coupling mechanism singly, Table 3 varies the targets under a fixed architecture, Table 4 removes training stages, and Section 3.1 defines the counting unit and applies it to every number. Appendix A now carries the configuration table. We also agree with your two strengths: progress-aware text and task-aligned visual targets are complementary, and removing the auxiliary decoders at inference is a practical choice rather than a hedge.

ConcernRevisionWhere
Specific design choices not isolatedComponent, target and stage ablations, each removing one itemTable 2 / Table 3 / Table 4
Comparative sample efficiency not establishedDecision anchor defined, one counting unit, one monocular-RGB baselineSection 3.1, Table 1 caption, Appendix D
Training configuration missingPer-stage configuration table; three remaining values markedAppendix A

[W1 & Q1] Navigation anchor, stop-gradient, and a simpler baseline

Table 2 removes each mechanism singly with data, backbone and targets fixed: 68.3 falls to 63.2 without cross-branch K/V, 61.8 without the navigation anchor and 67.5 without cross-branch stop-gradient. The anchor is the dominant term and the stop-gradient the smallest; we report that ordering rather than presenting them as equal. Table 4 addresses simpler training: skipping Stage 1 (0→2) gives 62.7/55.43 SR / SPL and skipping Stage 2 (0→1) gives 57.9/51.21, against 68.3/63.55 for the full schedule, so Stage 1 contributes 5.6 points. Stage 0 with explicit CoT reaches 60.3/54.88, but that is a decoding comparison, not the Stage 0 plus same trajectory-expert control you specify; that matched control is still missing.

The Stage 0 plus local-trajectory-expert variant you specify is the direct control still missing: we report no number for it rather than a partial one, and explicit CoT does not stand in for it. Table 3 completes the target comparison under a fixed architecture: task-aligned semantics with progress description 64.2, against 57.2 for RGB with scene CoT, 60.2 for semantics with scene CoT and 62.6 for RGB with progress description. Together the tables attribute the gain to specific choices — anchor coupling 6.5 points, cross-branch attention 5.1, Stage 1 5.6, target alignment up to 7.0 — rather than to the integrated recipe as a whole.

ConfigurationR2R SRR2R SPLDelta vs full modelIsolated component
Joint (full schedule)68.363.55——
w/o cross-branch K/V63.258.81−5.1Cross-branch attention
w/o navigation anchor61.857.65−6.5Anchor coupling
w/o cross-branch stop-gradient67.562.87−0.8Gradient blocking
Skip Stage 162.755.43−5.6Representation alignment stage
Skip Stage 257.951.21−10.4Joint optimization stage
Stage 0 + explicit CoT60.354.88−8.0Single-system baseline
AuthorsOpenReview ↗

Response to Reviewer UZA2 — W2 & Q2

[W2 & Q2] Sample efficiency, per-stage data, and repeated exposure

Section 3.1 defines the decision anchor and the unit. An anchor is one effective trajectory sample carrying both task-aligned annotations; every count uses the effective-trajectory-sample unit (one trajectory, one view, one height, one instruction). The dataset has about 4.0M such samples: 0.8M anchors and 3.2M instruction-following samples without both annotations. Stage 0 uses the 4.0M; Stages 1--2 reuse the 0.8M anchors, a subset rather than an additional pool, so later stages do not add to the dataset total. STOP oversampling is reported separately, so repeated exposures are not new samples. The table below records how each data component enters training; per-stage source composition and the exact counts are also spelled out in the data-accounting paragraph of Appendix A.

On matched conditions: input modality is fixed and the baseline's own reported count is used — 68.3 vs 66.9 SR on R2R-CE Val-Unseen for Qwen-RobotNav-4B, with 4.0M vs 15.6M samples (about 25.6%). The Table 1 caption states counts follow each method's unit and are not interchangeable, and Table 1 now lists every backbone. The +10.2-point gain (58.1% to 68.3%) is Stage 0 navigation SFT versus full Joint under like-for-like accounting — a method gain, not a data-volume gain. Scale results are in Appendix D; the model and training configuration is in Appendix A.

Data componentSamplesTask-aligned annotationsUsed in
Effective trajectory samplesabout 4.0M—Stage 0
Decision anchors0.8MYes (visual and textual)Stages 1 and 2
Instruction-following samples3.2MNoStage 0
STOP oversamplingCounted separately—All stages
AuthorsOpenReview ↗

Response to Reviewer UZA2 — Overall

[Overall] The three points in your assessment

The revision adds evidence on your three points. (i) Which choices produce the gain: Tables 2--4 remove each mechanism, target and stage singly, with margins of 6.5 points for the anchor, 5.1 for cross-branch attention, 5.6 for Stage 1 and up to 7.0 for target alignment. (ii) Whether sample efficiency holds: the counting unit is defined and applied throughout, and the only comparative claim is against one monocular-RGB baseline on R2R-CE Val-Unseen with an explicit count — 1.4 points at about 25.6% of its reported samples, with the caveat that the two counts use different units; scale results are separate in Appendix D. (iii) Configuration: Appendix A reports backbone, stride, GPUs/batch, epochs, steps, learning rates, auxiliary weights and per-stage update scope. The Stage 0 plus same trajectory-expert control remains the one comparison not completed.

What remains open is stated explicitly: multi-seed variance in simulation and the Stage 0 plus trajectory-expert baseline that would directly test the simpler-training alternative. We name these rather than let the paper imply more than the tables show. In summary the revision adds single-mechanism, target and stage ablations (Tables 2--4), a defined counting unit, a per-stage configuration table in Appendix A and complete real-world success counts in Table 5; the missing comparison is Stage 0 with the same local trajectory expert.

Reviewer_TgZzOpenReview ↗
Rating
3(+1)
Confidence
2(−1)
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes a dual system VLN model called NaviVLN built on THREAD. In training, the navigation data is turned into 2 targets, a progress-aware text description and a semantic image which mark the task-related objects and the expert path. Two auxiliary predictors learn both targets and a joint attention module share information with each other and with the navigation hidden state. After staged training, the VLM is connected to a DiT by latent queries.At test time the auxiliary parts are removed, and the VLM and action expert run asynchronously at different speed.

Critical strengths
  1. The design of using auxiliary text and visual predictors only in training and remove them at inference is practical and efficient
  2. The ablation in Table 2 is informative, the authors also report that the extra alignment loss does not help.
Critical weaknesses
  1. The main results is not strong and the claims are a bit misleading. “Approximately 10% higher SR than Uni-NaVid” is actually +4.39 points (51.39 vs 47.0). In Table 1, NaviVLN is lower than most of the recent methods, but these are not discussed and NaviVLN row is still bolded.
  2. The data-efficiency comparison seems not well controlled. The data units are mixed, NaviVLN’s 330K is called “decision anchors” in text but Fig. 2 says the dataset have 492K samples, and some baseline numbers seem not match their own paper.
  3. Slow VLM latent + fast DiT action expert with asynchronous running is very similar to DualVLN and InternVLA-N1, and progress text/visual CoT targets are also explored in CoT-VLA and FantasyVLN. The paper do not clearly say what is the difference in the related work,thus the novelty over existing dual-system VLN is not clear.
  4. The paper is almost not reproducible. The VLM backbone, size of DiT, loss weights, learning rates, training epochs, compute, and which model is used as the offline annotator are all not given, the appendix only contain qualitative cases. There is also no ablation for the data grounding pipeline and the staged training.
  5. No multiple seeds or std is reported, so the small gap like Joint vs online text reasoning is hard to judge.The real-world part only show 4 successful cases, without success rate over trials or comparison to any baseline.
Questions for authors

See weakness. And additional questions:

  1. What is the VLM backbone and its size, and are the baselines in Table 1 using comparable backbones? Please also give the main training details
  2. What exactly is a “decision anchor”, and how does 330K relate to the 492K samples in Fig. 2?
  3. In Table 2, does the AR answers only baseline also use the DiT action expert and the same staged training?
  4. For the real-world experiment, how many trials are run for each instruction and what is the success rate? How do other methods perform in the same scenes?
7 comments
AuthorsOpenReview ↗

Response to Reviewer TgZz — W1

We thank the reviewer for the direct assessment. Four of the five weaknesses say that the paper overstates what the numbers show, or omits what would be needed to check them, and we treat them that way rather than as disagreements. Concretely: the unqualified 10% claim is gone and the claim is now stated in percentage points against a named baseline; one counting unit is defined and applied to every number; Section 2.3 states the difference from DualVLN, InternVL-N1, CoT-VLA and FantasyVLN one method at a time; the configuration table is in Appendix A; and the real-world section reports trials rather than only successful cases. We also state plainly where NaviVLN is not the best: on RxR-CE we trail both Qwen-RobotNav-4B (71.3) and ABot-N0 (69.3).

[W1] The 10% claim, Table 1 emphasis, and the RxR-CE gap

You are right on both counts. The old sentence mixed a relative and an absolute gain without naming the baseline: the 10% came from Uni-NaVid at 47.0 SR, whereas the like-for-like monocular comparison is 68.3 versus 66.9, i.e. 1.4 percentage points (about 2.1% relative). It is removed from the Abstract and Introduction, which now read: 68.3% SR on R2R-CE Val-Unseen, 1.4 points above monocular Qwen-RobotNav-4B with about 25.6% as many samples (4.0M vs 15.6M). The claim is scoped to R2R-CE, where input modality is fixed, not to overall superiority.

Table 1 is now grouped by input modality (panoramic/multi-view, monocular RGB, our model), with every backbone filled in and only the best value per column bolded; the NaviVLN row is not presented as an across-the-board winner. We also state where we are behind: on RxR-CE NaviVLN gets 63.7 SR / 55.53 SPL, below Qwen-RobotNav-4B at 71.3 / 61.5 (monocular, 15.6M) and ABot-N0 at 69.3 / 60.0 (panoramic, 16.9M). We keep R2R-CE as the headline because modality is fixed there and do not claim the advantage transfers to RxR-CE; both settings appear side by side so the asymmetry is visible.

MethodInputTraining dataR2R SRR2R SPLRxR SR
NaviVLN (Joint)Monocular RGBabout 4.0M68.363.5563.7
Qwen-RobotNav-4BMonocular RGB15.6M66.960.571.3
ABot-N0Panoramic16.9M66.463.969.3
DualVLNMonocular RGBabout 12M64.358.561.4
AuthorsOpenReview ↗

Response to Reviewer TgZz — W2 & Q2

[W2 & Q2] The 330K / 492K / 287K counts and the decision anchor

A decision anchor is one effective trajectory sample with both task-aligned annotations: textually the completed subtask plus next movement direction (no action coordinates), visually instruction-relevant landmarks plus the projected expert path (Section 3.1). Every number uses the effective-trajectory-sample unit — one trajectory under one view, height and instruction. The dataset is about 4.0M such samples; the 0.8M anchors are a subset, and the remaining 3.2M are instruction-following samples without both annotations.

330K and 492K were not two views of one quantity: 330K was the alignment subset under a narrower unit (now 0.8M), 492K a total omitting effective samples (now 4.0M), and the 287K Figure 2 slice used the same old unit. All three were too small because one trajectory yields several effective samples under different descriptions, views and heights, and that density had not been counted; we therefore give them as they were rather than as conversions of the current figures. The table below records each earlier number against its replacement. Table 1 keeps each method's reported count and states units are not interchangeable. The only like-for-like comparison is the baseline reporting both monocular RGB and an explicit count: 68.3 vs 66.9 SR with 4.0M vs 15.6M samples. STOP oversampling is separate, so repeated exposures are not new samples.

ItemEarlier draftRevisedRelation
Decision anchors330K0.8MSubset carrying both annotations
Dataset total492K4.0MTotal effective trajectory samples
Instruction-following slice287K (Fig. 2 pie)3.2MThe part without annotations
Baseline sample count—15.6MReported by Qwen-RobotNav-4B
AuthorsOpenReview ↗

Response to Reviewer TgZz — W3

[W3] Difference from DualVLN, InternVL-N1, CoT-VLA and FantasyVLN

Section 2.3 now groups prior work into prompted agents, autoregressive action policies and hierarchical planning-execution systems, and states the difference from each close method explicitly rather than claiming novelty in general. Against DualVLN and InternVL-N1 we do not claim the dual-system structure — both already explore a slow VLM plus a faster asynchronous action module. We claim the representation: two implicit CoT streams, visual and textual, coupled by Joint Attention around a detached navigation anchor, with the action expert querying only the resulting latents. Against CoT-VLA and FantasyVLN we do not claim auxiliary reasoning targets as such: our visual target is a task-aligned semantic view (navigable region, landmarks, projected expert path), not future-frame reconstruction, and our textual target is navigation progress without action coordinates.

The claim is narrow and testable, and Table 3 is the test: under a fixed Separate architecture, task-aligned pairs give 64.2 SR against 57.2 for RGB with scene CoT, 60.2 for semantics with scene CoT and 62.6 for RGB with progress description. Table 2 adds coupling evidence — 63.2 without cross-branch K/V and 61.8 without the anchor, against 68.3 for the full model. If the gain came from the skeleton or auxiliary targets in general, these single-mechanism changes would not move SR by 5 to 7 points.

MethodShared with this paperWhat is different here
DualVLNSlow VLM with a faster action module, asynchronous executionThe contribution is not the asynchronous structure but the representation it learns: two implicit CoT streams coupled by Joint Attention around the navigation anchor, queried by the action expert
InternVL-N1Dual-system decompositionAuxiliary targets carry task-aligned annotations during training; the auxiliary decoders are removed at deployment
CoT-VLAVisual chain-of-thought targetsThe visual target is a navigable region with instruction-relevant landmarks and the projected expert path, not future-frame reconstruction
FantasyVLNTextual chain-of-thought targetsThe textual target is navigation progress without action coordinates
Reviewer_tEmKOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

This paper presents NaviVLN, a model that follows navigation instructions using current and past RGB images. During training, it learns to describe task progress and predict simplified scene images showing relevant landmarks and expert paths. Two prediction branches share information through Joint Attention to help train the VLM. The trained VLM then guides a separate model that generates local trajectories. The two branches are removed at inference, while planning and motion generation run at different rates. Results on R2R-CE and RxR-CE show gains over some baselines, with further support from ablations and robot demonstrations.

Critical strengths
  1. The training targets are closely tied to navigation: one describes which parts of the instruction have been completed, while the other highlights relevant landmarks and paths. The ablations show benefits from both.

  2. The system removes the two prediction branches at inference and allows the motion model to update more frequently than the VLM. Joint Attention raises success rate by 4.30 percentage points over separate branches on R2R-CE.

Critical weaknesses
  1. The design builds on ideas explored in OneVL and DualVLN, but the ablations do not clearly establish the value of the new design choices. They show that the prediction tasks and Joint Attention help, yet do not compare the proposed targets with simpler alternatives or separately assess cross-branch interaction, trajectory conditioning, and stop-gradient. It therefore remains unclear which specific choices account for the improvements. Comparisons with relevant training strategies using the same data and backbone would further clarify the technical contribution.

  2. The sample-efficiency claim compares undefined ‘’decision anchors’’ with sample counts from other papers, without establishing a common counting rule or clarifying whether Stage 0 data are included. Comparisons under matched data budgets are needed. Figure 4’s fitted comparison and illustrative extrapolation provide limited additional support for the claimed advantage..

  3. The paper does not specify the VLM backbone, exact losses, or training settings in enough detail. Comparisons that keep the VLM frozen or remove individual training stages are also missing, leaving limited evidence that the progress and scene prediction tasks improve the VLM itself.

Questions for authors

The citation for ABot-N0 appears to contain two errors: “Guoqing Liu” is written as “Qingguo Liu,” and “Hongyu Pan” is missing.

No other questions.

6 comments
AuthorsOpenReview ↗

Response to Reviewer tEmK — W1

We thank the reviewer for a careful and constructive reading. Three of the four concerns are about attribution: which specific design choices, rather than the dual-system recipe as a whole, produce the gain; whether the sample-efficiency claim rests on a defined and comparable counting unit; and whether the configuration is reported in enough detail to be checked. We answer them with component ablations, one counting unit, and a per-stage configuration table. The fourth point is a citation error, which is corrected. The two strengths you note, namely targets tied to navigation progress and the removal of the prediction branches at inference, are unchanged, and the new ablations are organised so that each mechanism can be checked on its own.

[W1] Attribution: which mechanism produces the gain

Table 2 removes each coupling mechanism from the full Joint model with data, backbone, targets and losses fixed: 63.2/58.81 SR/SPL without cross-branch K/V (−5.1/−4.7), 61.8/57.65 without the navigation anchor (−6.5/−5.9), and 67.5/62.87 without cross-branch stop-gradient (−0.8/−0.7). The anchor is the dominant term and cross-branch attention second; the 0.8-point stop-gradient difference is one run and we do not claim it stable across seeds. Joint plus an explicit alignment loss reaches 66.3/61.62, 2.0 points below Joint, so the negative result stays in the table.

Trajectory conditioning is tested where it is introduced. Stage 1 freezes the VLM and anchors both auxiliary branches to the navigation hidden state: skipping it (Table 4, 0→2) gives 62.7 SR and skipping Stage 2 (0→1) gives 57.9, against 68.3 for the full schedule (0→1→2). A VLM frozen throughout cannot train the representation and falls back to the Stage 0 level (58.1 without auxiliary targets, 60.3 with explicit CoT). The Stage 0 plus the same local trajectory-expert control you specify is still missing; the explicit-CoT variant is a decoding comparison and does not replace it.

Table 3 varies only the targets under a fixed Separate architecture: task-aligned semantics with progress description 64.2, navigation semantics with scene CoT 60.2, RGB reconstruction with progress description 62.6, and RGB with scene CoT 57.2. The 7.0-point spread between best and worst pairings exceeds the 4.1-point difference between Separate and Joint.

Configuration (R2R-CE / RxR-CE Val-Unseen)R2R SRR2R SPLRxR SRDelta vs full model
Joint (full model)68.363.5563.7—
w/o cross-branch K/V63.258.8161.6−5.1
w/o navigation anchor61.857.6558.2−6.5
w/o cross-branch stop-gradient67.562.8762.7−0.8
Joint + alignment loss66.361.6263.4−2.0
AuthorsOpenReview ↗

Response to Reviewer tEmK — W2

[W2] Decision anchor, counting unit, and the measured gain

Section 3.1 now defines a decision anchor as one effective trajectory sample carrying both task-aligned annotations: textually the completed subtask plus next movement direction (no action coordinates), visually instruction-relevant landmarks plus the projected expert path. The counting unit for every number in the paper is the effective trajectory sample — one trajectory under one camera view, height and instruction description. In that unit Navi-Dataset has about 4.0M samples: 0.8M anchors and 3.2M instruction-following samples without both annotations. The anchors are a subset of the same 4.0M pool.

The earlier figures counted different things and we explain them separately rather than convert between them: 330K was the alignment subset, 492K a dataset total that omitted effective samples, and 287K the instruction-following slice in the Figure 2 breakdown. All three predate this unit and were too small because one trajectory yields several effective samples under different descriptions, views and heights; we do not treat them as conversions of 0.8M / 4.0M / 3.2M. Figure 2 and its caption now show all three in one unit, and STOP oversampling is reported separately because repeated exposures are not unique samples.

Anchor counts cannot be traded against other papers' sample counts, so Table 1 keeps each method's own unit and the caption states they are not interchangeable. The only cross-paper comparison is against the baseline reporting both monocular RGB and an explicit count: 68.3 vs 66.9 SR on R2R-CE Val-Unseen with 4.0M vs 15.6M samples (about 25.6%), a 1.4-point lead. We claim no ordering where counting rules cannot be reconstructed. The +10.2-point gain (58.1% to 68.3%) is Stage 0 navigation SFT versus the full model under one unit, a method gain rather than a data-volume gain; scale results are in Appendix D at evaluated scales only.

ItemEarlier draftRevisedMeaning
Decision anchors330K0.8MSamples carrying both task-aligned annotations
Dataset total492K4.0MEffective trajectory samples
Instruction-following slice287K (Fig. 2 pie)3.2MSamples without the two annotations
Counting unitMixedEffective trajectory sampleOne trajectory x one view x one height x one instruction
AuthorsOpenReview ↗

Response to Reviewer tEmK — W3

[W3] Training configuration and stage-removal controls

Appendix A now gives a per-stage configuration table. Backbone Qwen3-VL-4B (also a new column of Table 1), training stride 8 frames, 40 GPUs with per-GPU batch 10 in Stage 0 and 6 in Joint stages, gradient accumulation 1, and checkpoint steps 11,089 / 3,748 / 4,627 for Stages 0 / 1 / 2 using first-epoch weights. Per-stage update scope: vision encoder frozen in Stage 0; VLM frozen with auxiliary predictors, projections and Joint Attention trained in Stage 1; joint optimization in Stage 2 with the encoder frozen. Base learning rates 3e-5 / 1e-4 / 2e-5, warmup about 2% / 1% / 0.3%, auxiliary weights 0.1 visual / 1.0 text. Section 3.2 writes the objective as L_nav + lambda_progL_prog + lambda_semL_sem with both targets defined.

These settings are no longer left open: Appendix A reports the auxiliary weights (0.1 visual / 1.0 text) and per-stage learning rates (3e-5 / 1e-4 / 2e-5). Frozen-VLM training is Stage 1 itself; Table 4 quantifies skipping it (0→2) at 62.7 SR and skipping Stage 2 (0→1) at 57.9, against 68.3 for the full schedule. The Stage 0 plus same trajectory-expert control you ask for is still missing: the explicit-CoT variant (60.3) is a decoding comparison and does not replace it.

SettingStage 0Joint Stage 1Joint Stage 2
VLM backboneQwen3-VL-4Bsamesame
Training stride (frames)888
GPUs / batch per GPU40 / 1040 / 640 / 6
Gradient accumulation111
Epochs used for results111
Optimizer steps at ckpt11,0893,7484,627
VLM updateVision encoder frozenVLM frozenLM/projector/aux updated; vision encoder frozen
Base learning rate3e-51e-42e-5
Auxiliary weights (visual / text)—0.1 / 1.00.1 / 1.0

Comments on the paper

1 comment
AuthorsOpenReview ↗

General Response

We thank all three reviewers for a careful and constructive assessment. The reports recognise three design choices: auxiliary predictors removed at inference (R1--R3), progress-aware text paired with task-aligned visual targets (R1, R3), and ablations that include the negative alignment-loss result (R2). Three questions recur: what in the integrated recipe produces the gain, whether the data-efficiency comparison uses one counting unit, and whether the configuration and real-world protocol can be checked. We answer with single-mechanism, target and stage ablations, one counting unit, a per-stage configuration table, and an explicit account of what remains open.

  1. Attribution. Tables 2--4 vary one mechanism, target pair or stage at a time with other settings fixed. Removing the navigation reference, cross-branch K/V and Stage 1 costs 6.5, 5.1 and 5.6 R2R-CE SR points; changing the auxiliary targets moves SR by up to 7.0 points. The 0.8-point stop-gradient difference is from one run and is not claimed stable across seeds.

  2. Counting and comparison. Section 3.1 defines the effective trajectory sample: about 4.0M in total, of which 0.8M carry both annotations and 3.2M do not; the 0.8M are a subset of the same pool. The only cross-paper comparison holds input modality fixed: 68.3 versus 66.9 SR on R2R-CE Val-Unseen against Qwen-RobotNav-4B, with 4.0M versus 15.6M samples. The +10.2-point figure compares Stage 0 navigation SFT with the full model under the same unit; data-scale results appear in Appendix D at evaluated scales only.

  3. Reproducibility. Appendix A reports backbone, stride, GPUs, batch sizes, effective batch, epochs, checkpoint steps, learning rates, warmup, auxiliary weights and per-stage update scope. Section 3.2 states the objective; Section 3.3 places the DiT action expert at Stage 3 with identical use across variants.

  4. Real-world evidence. Section 4.3 and Table 5 report 15 trials per case over five cases: 75 per configuration, 375 in total, with success defined as completing the instruction. Table 5 gives summed successes and rates without a rank column: Sem-Joint 52/75 (69.3%), Sem-Separate 46/75 (61.3%), Sem-Merged 40/75 (53.3%), RGB 16M 38/75 (50.7%) and RGB 0.8M 26/75 (34.7%). No external baseline was deployed in the same scenes, so this validates the supervision design rather than giving a head-to-head comparison.

  5. Open items. Simulation tables are single runs without multi-seed mean or standard deviation; sub-point differences are not interpreted. The Stage 0 policy followed by the same local trajectory expert remains unmeasured, and Stage 0 plus explicit CoT is a decoding comparison, not that control. On RxR-CE Val-Unseen we are below Qwen-RobotNav-4B and ABot-N0, as the revision states explicitly.

  6. Presentation and citations. The unqualified 10% claim is removed; Section 2.3 separates our setting from DualVLN, InternVL-N1, CoT-VLA and FantasyVLN one method at a time; and the ABot-N0 author list is corrected.

Revisions.

All text and table changes are marked in blue: the ablations in Tables 2--4, the counting-unit definition in Section 3.1, the per-stage configuration table in Appendix A, the real-world protocol and success counts in Section 4.3 and Table 5, and the revised claims in the Abstract and Introduction. Table 1 fills in every backbone, adds nDTW, groups rows by input modality and bolds only the best value per column; Table 2 puts the full model last after the single-mechanism removals; Tables 3--4 cover targets, stages and decoding; Table 5 reports 75 trials per configuration with no rank column; Appendix D reports only evaluated scales; and references.bib corrects two ABot-N0 author-list entries. An anonymised project page with qualitative videos and the revised tables is at https://navivln.github.io/navivln-review/#simulation.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.