- Rating
- 1
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
Aligned-VLN augments Qwen2.5-VL-3B with progress and action query tokens to model navigation progress through instruction-token alignment. Content matching and a Gaussian positional bias jointly produce an alignment distribution, whose expected token position and attended instruction representation form a progress context for action decoding. Training combines action imitation with trajectory-derived progress, alignment, increment, and monotonicity losses, using the fraction of completed forward primitives as weak supervision. Each forward pass predicts three action symbols, with the first executed before observing and replanning.
Critical strengths
-
Meaningful problem and concrete mechanism. For multi-stage instructions, locally feasible actions can lead to errors such as premature turns. Linking navigation progress to the instruction content relevant at the current stage is a well-motivated direction.
-
Informative design comparisons. Full improves over PositionOnly by 2.72 SR and 2.63 SPL percentage points, while removing monotonicity supervision reduces SR by 2.72 points. These results suggest benefits from content-based scoring and temporal-order supervision.
-
Simple deployment pipeline. Progress estimation and action prediction share a single VLM forward pass, without generating progress text or invoking a separate progress model. Demonstrations on two robot platforms provide preliminary evidence of deployment feasibility, although inference speed and deployment success rates are not quantified.
Critical weaknesses
W1. Training and validation primarily assess trajectory progress rather than semantic instruction progress. The current evidence does not directly establish whether the model tracks the semantic execution stage of the instruction.
W2. The incremental contribution of token alignment is small and insufficiently isolated by the ablations. ScalarOnly improves SR over ActionOnly by 6.96 percentage points, compared with Full’s total gain of 7.56 points. The former accounts for approximately 92% of the latter, although this is a descriptive ratio rather than a decomposition of independent causal contributions. Full improves over ScalarOnly by only 0.60 SR points, equivalent to approximately 11 additional successful episodes out of 1,839. Its 1.67-point SPL improvement may nevertheless have practical value.
W3. The main results lack key controls under the same training regime, the training corpus is insufficiently specified for reproduction, and comparisons with the latest state-of-the-art methods are missing. The main experiments combine R2R, RxR, and ScaleVLN, whereas the ablations use only R2R. These ablations support component benefits in the R2R-only setting but do not establish whether token alignment is similarly necessary for the main results. Aligned-VLN exceeds the reported SR of the 3B NaVIDA model by 2.1 points on R2R and 2.2 points on RxR. However, as the authors acknowledge, cross-paper protocols may differ, so these differences cannot be interpreted as controlled evidence of gains from progress alignment.
Questions for authors
-
Could the authors provide a small-scale semantic-stage or premature-action diagnostic to demonstrate the added value of token alignment beyond trajectory-progress prediction and a scalar baseline?
-
How does the proposed method differ from the instruction-attention/progress coupling in Self-Monitoring? Which gains arise from the shared expected-position readout rather than standard auxiliary scalar-progress supervision?
-
What are the size, mixture proportions, and conversion rules of the aggregated training corpus, and how is the model initialized? How does ScalarOnly perform under the same aggregated-training configuration?
4 comments
Dear Reviewer,
I hope this message finds you well. As the discussion period is nearing its end, I wanted to ensure we have addressed all your concerns satisfactorily. If there are any additional points or feedback you'd like us to consider, please let us know. Your insights are invaluable to us, and we’re eager to address any remaining issues to improve our work.
Thank you for your time and effort in reviewing our paper!
Thank you for the clarification and additional experiments. The RxR ablations and semantic-stage analysis strengthen the empirical evidence. However, it remains unclear whether the method tracks instruction semantics beyond trajectory progress, and concerns about novelty and the attribution of the gains remain insufficiently addressed. I therefore maintain my original rating.
Thank you for carefully reviewing the additional experiments. You observed that ScalarOnly already achieves most of the SR gain on R2R. This is precisely why it is the relevant baseline for assessing Full: its backbone already encodes the complete instruction, and its action decoder already receives scalar progress. Against this baseline, Full gains a further 0.60 SR points and 1.67 SPL points on R2R. On RxR, both models were retrained with the same backbone, trajectories, and monotonic training pairs. Full raises SR from 19.58 to 22.59 and SPL from 16.14 to 18.75. These comparisons measure the navigation gain of the complete design over scalar progress conditioning.
In Full, the token weights that determine the expected instruction position also determine which instruction features the action decoder reads. The fraction of expert forward actions completed provides weak positional supervision for the alignment distribution. For a given positional bias, the match between the current navigation state and each token’s content can still change the relative token weights (Eq. 3). The model uses these weights to compute both the expected instruction position and the pooled instruction features. Gradients from action imitation also reach the weights through the decoder (Eq. 4 and Sec. 3.5). Figure 3(b) illustrates this readout on one unseen route, where the weights shift from the words about turning and leaving the pantry toward those about reaching the patio and stopping.
PositionOnly addresses the contribution of content matching to navigation more directly. It can already shift its weights with predicted progress and pool instruction features by position. Against this baseline, Full gains a further 1.34 SR points and 1.28 SPL points on RxR. The gain over ScalarOnly establishes the performance advantage of the complete design in these experiments, while the PositionOnly comparison supports the additional navigation value of matching instruction tokens to the current state. We respectfully ask you to reconsider your assessment of the method’s incremental contribution and your rating in light of these two comparisons.
Response to Reviewer fdwm
Thank you for suggesting that we look at when the agent turns.
W1 and Q1: Semantic stages and early turns
Figure 7 (Appendix D) shows the doorway sequence in episode 898 on the R2R-CE unseen validation split. Panels A and B are consecutive expert observations. Panel C is a Habitat rendering after the expert's turn. It illustrates the timing problem in Fig. 1. At the bedroom doorway, the expert moves forward 25 cm before turning right 15°. Full matches the expert's action at both A and B. ActionOnly and PositionOnly predict the turn one step early.
We use the fraction of completed forward steps to supervise the expected token position and center the soft target over tokens (Sec. 3.5). Figure 3(b) shows the weights moving from the opening clause to the destination clause on a held-out route. On 2,290 expert replay states with a clear FGR2R stage away from a clause boundary, summing Full's weights by clause matches the label 53.0% of the time. The expected position alone matches 48.7% of the labels. The fraction of forward steps from the complete expert route matches 60.4%.
W2: What token alignment adds to scalar progress
ScalarOnly uses route progress as a cue to where the agent is in the instruction. With that cue, it scores 6.96 SR points above ActionOnly on R2R. Full uses the same token weights to estimate the current instruction position and pool the features the decoder uses to choose its next action (Appendix A.5).
Full improves over ScalarOnly by 0.60 SR and 1.67 SPL points on R2R, and by 3.01 SR and 2.61 SPL points on RxR (Table II). On RxR, nDTW rises from 0.543 to 0.593. Full, PositionOnly, and ScalarOnly train on the same RxR trajectories and monotonic pairs.
Table II. Ablations with one training source at a time
The R2R scores come from Table 2 of the manuscript. We retrained the RxR variants on RxR only.
Q2: Self-Monitoring and shared weights
Self-Monitoring uses instruction attention to predict progress for ranking trajectories or backtracking. In Full, the decoder gets both an expected position and pooled instruction features from the same token weights (Eqs. 3–4). Self-Monitoring, Secs. 2.2–2.4.
Full and PositionOnly both share weights across the two readouts. Full also scores tokens by content. Relative to PositionOnly, Full gains 2.63 SPL points on R2R (95% paired interval over routes: [0.03, 5.22], with checkpoints fixed). On RxR, it gains 1.34 SR and 1.28 SPL points. To measure what sharing contributes, we would keep both readouts and compute each from its own distribution. To test progress as auxiliary supervision alone, we would keep the loss and remove the decoder's progress input.
W3 and Q3: Recent baselines and pooled training
Table I. Published scores on the unseen validation splits
Sources: NaVIDA, Tables 1–2 and Secs. 3.3, 4.1; MapDream, Table 1 and Sec. 4.2; Progress-Think, Tables 1, 2, and 5.
Progress-Think reports RxR-CE in a transfer setting. We execute the first predicted action symbol and replan after the next observation. NaVIDA executes the first two predicted chunks and trains with DAgger and inverse-dynamics supervision. MapDream trains with non-oracle samples and reinforcement fine-tuning.
For Table 1 of the manuscript, we trained Full with Qwen2.5-VL-3B on pooled trajectories from R2R, RxR, and ScaleVLN. Table III gives the counts from the archived source lists before conversion to action examples.
Table III. Source records before action-level conversion
Each symbol encodes one to three consecutive forward steps or same-direction turns. We fill unused target slots with STOP and label progress by the fraction of forward steps completed (Appendices A.3–A.4). Table II compares Full and ScalarOnly separately on R2R and RxR. Table 1 of the manuscript reports Full after pooled training.