- Rating
- 3(+1)
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2(+1)
Read review
Paper summary
This paper proposes ReDual-VLN, a dual-system framework for long-horizon VLN with deviation recovery. The upper VLM is designed for route planning, context management, and recovery decisions, and the lower VLN executes local navigation and backtracking instructions. The authors also introduce ReTrace-RxR for training short-horizon navigation and backtracking. The results show improvements on both R2R-CE and RxR-CE.
Critical strengths
S1. The paper focuses on an important and practical problem in long-horizon VLN. The motivation is clear, and the framework is sound. The results look pretty good, especially the improvements brought by the recovery mechanism on both benchmarks.
S2. I also like the idea of constructing paired forward and backtracking data with instruction-specific termination labels. This idea seems useful for training the VLN executor, and might be able to be applied to other VLN methods.
Critical weaknesses
W1. I think the novelty of the proposed framework could be better clarified. Some very recent works, such as ReflectVLN, Route2Step, and Uni-LaViRA, have already explored similar ideas, including hierarchical reasoning, deviation correction, and backtracking. Although these works are also quite recent, I think it would be helpful if the authors could better discuss the differences and highlight the unique contributions of their approach.
W2. The recovery mechanism should have a more detailed evaluation. The overall improvement is clear, but it would be helpful to know how well each part works, such as deviation detection, anchor selection, backtracking, and task resumption. Also, the backtracking training data seems to mainly cover returning from the endpoint of a segment to its starting point. I am not sure how well this generalizes to actual navigation errors, especially when the agent deviates in the middle of an instruction. Some experimental results are not well explained. For example, in Table 2, BE-VLN achieves a much higher SR than SPAN-Nav, but the improvement in OSR is quite small. I was wondering whether the improvement mainly comes from better stopping predictions rather than better navigation. I think this deserves some further discussion.
Questions for authors
Q1. How does the proposed recovery mechanism compare with simpler approaches, such as replanning from the current position or returning to the last completed sub-instruction? I think this comparison would help clarify the actual benefit of the proposed recovery strategy.
Q2. Can the authors provide a more direct ablation study to show the independent contribution of HCM, while keeping the upper VLM and instruction decomposition unchanged?
Q3. How well does the backtracking model generalize to actual route deviations, especially when the agent deviates in the middle of an instruction or needs to return to an intermediate anchor rather than the starting point of a segment?
Q4. In Table 2, BE-VLN improves SR from 48.25% to 81.01% compared with SPAN-Nav, while OSR only increases from 83.16% to 84.11%. Could the authors clarify how much of this improvement comes from better instruction termination rather than navigation itself?
7 comments
Response to Questions: Recovery, HCM, and Local-Task Learning
Comment 2: Recovery, HCM, and Local-Task Learning
Thank you. We address Q1-Q4 below; W1-W2 are answered separately.
Q1. What do adaptive anchors add over simpler recovery strategies?
We improve final-goal completion from matched recovery states. We achieve 14/18 successes, versus 12/18 for fixed-boundary return and 8/18 for direct replanning. A valid within-step anchor can restore a route reference without returning to an earlier boundary.
From 27 natural events in a previously screened pool, we retain the earliest entry per episode, yielding 18. We restore pose, decomposition, progress and history, fixing VLM evidence, executor and remaining budget. Each strategy starts with one VLM call, including adaptive selection/return-instruction generation. Strategy rules apply to the unfinished entry step; later steps share a controller. Direct replanning permits return motion. Success requires final STOP at the original goal (<3 m).
| Strategy | SR (%) | NE (m) | Actions | Path (m) |
|---|---|---|---|---|
| Direct replanning | 44.44 | 11.17 | 154.11 | 23.05 |
| Fixed-boundary return | 66.67 | 8.21 | 150.22 | 21.48 |
| Adaptive anchor | 77.78 | 7.73 | 142.94 | 21.17 |
We average actions and travel over all entries, including return, continuation and failures. These results support restoring a route reference before continuing the task (Appendix C.6, Table 20).
Q2. What does HCM contribute with the Supervisor and decomposition fixed?
We isolate an 11.25-point SR gain from context organization. On the same 80 random RxR-CE val-unseen tasks, we fix the VLM, executor, cached decomposition, image-selection policy/budget, review triggers, recovery and Controller safeguards.
Both receive the original instruction and basic dispatch state (objective, command and return target), retaining full supervisory/recovery capabilities. Our Chronological History control reconstructs relations from ordered commands/reviews. With HCM, we add instruction-window, attempt, boundary and visual-reference associations linking temporary returns to original obligations. Appendix D provides both complete prompts.
| Metric | Chronological History | HCM |
|---|---|---|
| SR (%) | 68.75 | 80.00 |
| SPL x100 | 45.36 | 55.27 |
| OSR (%) | 75.00 | 85.00 |
We also reduce cumulative review tokens by 6.85% and service time by 7.28%, averaged across all 80 tasks and excluding initial decomposition (Appendix B.7.2, Table 11).
Q3. Can BE-VLN handle intermediate interruptions and anchors?
Yes: we initiate recovery before instruction completion and return to online-selected intermediate anchors. We analyze all 45 natural attempts from 18 of 300 random RxR-CE val-unseen episodes; manual inspection verifies every entry precedes genuine completion of the active forward instruction.
| Natural recovery subset | Anchor return | Task recovery |
|---|---|---|
| All mid-instruction attempts | 40/45 | 35/45 |
| Within-segment anchors | 39/44 | 35/44 |
| Anchors outside executor history | 3/6 | 2/6 |
Rows are nested. Anchor return uses exit geodesic distance <=3 m; we manually verify correction and completion of the interrupted requirements. For outside-history anchors, we confirm 5/6 arrivals from trajectories/landmarks, 3/6 meet the exit-pose criterion, and 2/6 complete the objective. Our Supervisor sees the historical image; BE-VLN follows its instruction with recent context (Appendix C.4, Tables 17-18).
Training demonstrations start at segment endpoints, but trajectory/ITF supervision covers intermediate states under the same return instruction. Online, we generate commands from the actual interruption and traversal, supporting these state-dependent returns (Appendix A.2.2).
Q4. How much of the SR gain comes from stopping versus navigation?
We learn effective local execution and gain a further 4.88 SR points from selecting ITF. ReTrace-RxR trains motion and terminal boundaries relative to each local instruction. Adding its forward subset, with other sources and ITF supervision retained, raises short-horizon SR from 55.74% to 81.01% (Appendix B.8, Table 12).
We then compare stopping criteria on the same short-horizon test, fixing BE-VLN weights, the trajectory predictor, input interface and budget. Both conditions retain the same training, including ITF supervision.
| Criterion | SR (%) | OSR (%) | SPL x100 | NE (m) | nDTW x100 |
|---|---|---|---|---|---|
| Small motion | 76.13 | 83.59 | 71.45 | 2.13 | 77.01 |
| ITF | 81.01 | 84.11 | 76.43 | 2.00 | 80.22 |
With small-motion stopping, we already obtain 76.13% SR versus SPAN-Nav's 48.25%; selecting ITF raises SR to 81.01%. We narrow BE-VLN's OSR-SR gap from 7.46 to 3.10 points, converting more goal-region visits into completed instructions. B.8 tests motion/boundary training jointly; B.9 directly measures the additional inference-time stopping benefit with the trained executor fixed.
Thank you very much for the detailed response and additional experiments. I can see quite a lof of effort here. The HCM ablation, stopping analysis, and natural recovery results address several of my concerns.
My main concern is still how reliably the system decides when recovery is needed. The recovery analysis covers cases where recovery was triggered, but how often does the system miss a deviation or trigger recovery when it is not needed?
Moreover, the results for anchors outside the executor history are not yet so convincing. From the table, only 2/6 attempts achieved task recovery. The evidence for reliable recovery might not be strong enough. But I am open to discussion on this concern.
Additionally, the results in W1 and Q1 seem to be different. Is this a typo, or am I missing anything?
I am open to further discussion, and keep my original score for now.
Response to Critical Weaknesses: Contributions and Recovery Evidence
Comment 1: Contributions and Recovery Evidence
Thank you. We make long-horizon navigation recoverable through learned local return execution, HCM-managed route context and verified completion of unfinished requirements. We address Q1-Q4 separately.
W1. Contributions and practical advantages
ReflectVLN and Route2Step advance hierarchical correction and progress tracking; Uni-LaViRA supports map-planned returns, replanning and pending-goal memory. Our contribution combines learned language-conditioned return execution, instruction-relative termination and verified completion of the interrupted objective.
We train the local capabilities required by recovery. A landmark passed on a full route can be a short instruction's endpoint. ReTrace-RxR aligns forward/return instructions with direction-specific observations, motion and terminal boundaries, making both commands executable through one policy. With the backbone, heads and update budget fixed, adding forward data raises short-horizon SR from 55.74% to 81.01%; adding return data raises backtracking SR from 47.80% to 63.38%, each against the full corpus lacking that subset (Appendix B.8).
We restore a usable route reference through language and RGB history. After a wrong turn, the current scene may no longer align with the remaining instruction. We select an identifiable visited anchor on the intended route, generate a return instruction from the actual traversal, including its off-route portion, and execute it with BE-VLN. Unlike a map-planned waypoint return, this uses the same language/RGB interface as forward execution. In the 18-entry paired comparison, we achieve 77.78% final-goal SR, versus 66.67% with fixed-boundary return and 44.44% with direct replanning, with the lowest mean actions and travel. This tests return and continuation from matched recovery states (Appendix C.6, Table 20).
We preserve the original obligation while restoring the route. HCM separates the unfinished objective from the temporary return command and links both to execution attempts, boundaries and visual evidence. We keep route-level history with the Supervisor, while BE-VLN uses its current instruction and 32-observation visual window. We verify landmark identity, topology and a navigable reentry region, then regenerate the original objective's remaining requirements. With VLM, cached decomposition, executor and recovery capability fixed, HCM raises SR from 68.75% to 80.00% on 80 tasks (Appendix B.7.2).
These components turn local return into long-horizon task recovery. With the same Supervisor, HCM, BE-VLN checkpoint, review settings and action budget, enabling recovery raises full-route SR from 69.6% to 76.7% on R2R-CE and 72.3% to 78.5% on RxR-CE (Table 1).
W2. Recovery stages, natural interruptions, and local completion
We evaluate recovery through completion of the interrupted requirements. We run 300 random RxR-CE val-unseen episodes once each and analyze all 45 natural attempts from 18 episodes. Independent manual inspection confirms that every intervention precedes genuine spatial/semantic completion of the active sub-instruction.
We manually check post-return trajectories against the interrupted sub-instruction. Of the 35 completions, 32 meet the geometric criterion and three complete from a suitable reentry state beyond it (Appendix C.3). Thus, we distinguish geometric return, execution handoff and objective completion.
The ten failures have one primary attribution each: detection (1), anchor selection (2), return execution (0), verification (2), or post-return objective completion (5), including two budget-exhausted cases. This diagnosis identifies the importance of connecting a returned state to the unfinished requirements.
We recover from natural intermediate states using historical evidence. We use within-segment anchors in 44 attempts, obtaining 39 returns and 35 objective completions. For the six anchors outside BE-VLN's recent history, trajectory/landmark review confirms arrival at the intended anchor in 5/6; 3/6 satisfy the exit-pose criterion and 2/6 complete the interrupted objective. HCM supplies the historical anchor image to the Supervisor, which turns it into a language instruction for the recent-window executor (Appendix C.4, Tables 17-18).
We separate local-task learning from the benefit of selecting ITF. On the same short-horizon test with BE-VLN fixed, small-motion stopping achieves 76.13% SR and ITF achieves 81.01%. ITF adds 4.88 SR points; OSR is 83.59%/84.11%, narrowing the OSR-SR gap from 7.46 to 3.10 points. These controls connect task-specific motion/boundary training to reliable local completion (Appendix B.8-B.9; Q4).