- Rating
- 4(+1)
- Confidence
- 3
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
This paper addresses the challenge of effective recovery after an agent deviates from the instructed route in vision-and-language navigation (VLN). By investigating the impact of visual history on navigation decisions and deviation recovery, the authors propose two lightweight modules for deviation detection and navigation recovery. Specifically, the Deviation Judge determines whether the agent has deviated from the instructed route based on the navigation instruction, current observation, and historical information, and clears the visual history when an alarm is triggered. The Recovery Policy employs Group Relative Policy Optimization (GRPO) to optimize recovery trajectories sampled from deviation states using reward signals, thereby improving navigation recovery success rates. Experimental results demonstrate that the proposed method effectively improves navigation success rates while introducing only a small number of additional model parameters.
Critical strengths
-
The paper presents a clear research motivation, focusing on the impact of historical information on deviation recovery in vision-and-language navigation. Through paired intervention experiments on different navigation policies, the authors demonstrate that historical information plays different roles during normal navigation and deviation recovery, providing valuable insights into historical dependence and error recovery in long-horizon navigation.
-
The proposed MOON framework is relatively well-designed, achieving performance improvements through a lightweight deviation judge and a reinforcement learning-based recovery policy while keeping the parameters of the base navigation model frozen.
Critical weaknesses
-
The paper takes behavioral inertia caused by historical information as its core motivation and attempts to improve navigation recovery by clearing visual history. However, as shown in Table 2, after introducing the GRPO-based recovery policy, the model without history clearing already achieves a success rate of 67.4%, which is not statistically significantly different from the 69.0% achieved by the full method.
-
The training results in the appendix further show that the performance gap between retaining and clearing historical information decreases as reinforcement learning training progresses. The authors should further clarify the independent contribution of the history-clearing mechanism after sufficient training.
-
The paper presents a compelling research motivation and provides relatively comprehensive analyses that effectively support its underlying motivation. However, the recovery policy primarily relies on standard trajectory-level rewards and group-relative advantage optimization, which are already widely adopted in reinforcement learning. The authors should further clarify the core technical novelty of MOON compared with existing navigation recovery and reinforcement learning methods.
Questions for authors
-
According to Table 2, the GRPO-based recovery policy without history clearing already achieves performance close to that of the full method, with no statistically significant difference between them. The authors should further investigate whether a consistent performance gap remains between recovery policies with and without history clearing under the same training budget and after sufficient convergence.
-
The paper argues that retaining historical information causes navigation policies to continue executing previous actions. However, is this phenomenon specific to the evaluated VLN models, rather than a general issue in VLN? Furthermore, have existing methods explicitly addressed or mitigated this tendency of navigation policies to persist in previous actions when historical information is retained?
5 comments
Response to Reviewer 4WsE (Part 2/3): Recovery-Specific Technical Contribution Beyond GRPO (W3)
Response to W3: Technical Contribution of Recovery Learning
We thank the reviewer for raising this important point. While we intentionally adopt standard GRPO as a clean, critic-free optimizer, naively applying standard trajectory-level RL/GRPO to navigation recovery fails. In standard full-trajectory RL, rare deviation states are under-sampled, accumulated history locks exploration, and global trajectory rewards misattribute pre-deviation penalties.
MOON's core technical novelty lies not in modifying the GRPO loss formula, but in reformulating the RL Markov Decision Process (MDP) specifically for post-deviation recovery through three tightly coupled technical mechanisms:
| Recovery RL Bottleneck | Standard Trajectory RL / Naive GRPO | MOON's Recovery-Specific Formulation | Empirical Impact |
|---|---|---|---|
| 1. State Distribution | Rollouts start from (mostly on-route) | First-Alarm State Branching: Branches rollouts directly from online-detected deviation states | Concentrates 100% compute on hard recovery states |
| 2. Exploration Collapse | Samples conditioned on full corrupted history | Context-Reset Exploration: Clears history prior to group sampling to break behavioral inertia | Doubles initial valid gradient groups (: 22.7% 50.8%) |
| 3. Credit Contamination | Global SPL penalizes pre-deviation detours | Decoupled Suffix-SPL Objective: Evaluates path efficiency strictly over the post-alarm suffix | Boosts greedy recovery SR on 527 alarmed episodes (35.1% 47.1%) |
Specifically, MOON introduces three recovery-specific contributions that make GRPO viable and effective:
- Context-Intervened Group Exploration (Solving Gradient Starvation): Standard GRPO requires intra-group reward variance (at least one successful rollout among samples) to compute non-zero advantages. Under standard retained history, behavioral inertia locks the policy into repeating the same error, yielding only 22.7% initial (leaving 77.3% of states with zero gradients). By coupling selective history clearing directly with first-alarm state branching, MOON alters the conditioning distribution before rollout sampling, surging initial to 50.8% and unlocking latent recovery trajectories for RL to reinforce.
- Decoupled Post-Alarm Credit Assignment (Suffix-SPL): Standard trajectory-level SPL evaluates the entire path from . If a base policy takes a long detour before triggering an alarm, standard SPL heavily penalizes the rollout even if the subsequent recovery is optimal—corrupting the advantage estimate. Our Suffix-SPL isolates the exact trajectory segment controlled by the recovery policy (from the cleared alarm pose to termination), ensuring clean, unbiased credit assignment for recovery efficiency.
- Bridging Sampling Potential to Greedy Execution (Without External Experts): Unlike prior navigation recovery methods (e.g., DAgger, CorrectNav) that require external expert oracles to synthesize corrective trajectories, MOON is completely self-supervised by execution outcomes. By aligning cleared-history group rollouts via Suffix-SPL, MOON successfully distills multi-sample recovery potential into single-run greedy execution—boosting greedy success on 527 alarmed R2R-CE episodes from 35.1% to 47.1%.
We will explicitly contrast our recovery-specific MDP formulation against standard trajectory-level RL in Section 3 to make this technical contribution unmistakable.
Response to Reviewer 4WsE (Part 3/3): Generality of Behavioral Inertia Across VLN Architectures (Q2)
Response to Q2: Generality of Behavioral Inertia
We thank the reviewer for this insightful question. Below, we address both the generality of behavioral inertia across VLN architectures and why existing training-time mitigations fail to resolve it during recovery:
- Generality Across Diverse VLN History Architectures: Rather than an artifact of a specific model, behavioral inertia is a systemic, architecture-agnostic phenomenon across modern VLN policies. Our four evaluated models deliberately span the primary paradigms for encoding visual history in VLN—from explicit frame sampling to implicit KV-cache and spatial-temporal token merging. Evaluating matched deviation-onset states (following a forward step where the shortest-path expert turns) reveals severe action-locking across all memory mechanisms when history is retained, which immediately collapses upon clearing history:
| VLN Policy | History Representation Paradigm | Keep History (No-Turn ) | Clear History (No-Turn ) | Expert |
|---|---|---|---|---|
| NaVILA [3] | Fixed-frame uniform sampling | 68.7% | 5.5% () | 1.1% |
| Uni-NaVid | Online spatial-temporal token merging | 52.1% | 12.6% () | 3.3% |
| NaVIDA | Hierarchical historical token compression | 35.0% | 11.9% () | 1.2% |
| StreamVLN | Streaming KV-cache memory | 33.1% | 13.6% () | 1.8% |
(Note: "No-turn" denotes failing to execute any turning action in the subsequent action chunk, where the expert turns of the time.)
- Relation to Prior Work & Why Existing Mitigations Fall Short:
Prior literature has examined related phenomena from two angles, neither of which resolves post-deviation context locking:
- Unconditional Action Bias vs. Context-Conditioned Inertia: Prior VLN analysis [1] identifies a global training-induced forward bias due to action imbalance, and NaVILA [3] explicitly mitigates this during training via action-label rebalancing. Crucially, our table above proves that data rebalancing does not solve behavioral inertia: despite balanced training, NaVILA still suffers a massive 68.7% no-turn rate under retained history, which plummets to 5.5% immediately upon clearing history using the exact same model weights. This proves the bottleneck is test-time historical context conditioning, not static parameter bias.
- Global History Removal vs. Dynamic Selective Intervention: In general imitation learning, works like Fighting Copycat Agents [2] combat temporal action-copying shortcuts by penalizing or discarding historical information globally during training (e.g., via adversarial representation learning). However, in long-horizon VLN, history is essential when on-route to track instruction progress (as shown in Sec. 2, globally removing history degrades on-route SR).
Summary of MOON's Distinction: Prior methods either rebalance static action priors [1, 3] (which leaves context-induced inertia intact upon deviation) or suppress reliance on history globally [2] (which hurts on-route navigation). MOON is the first to formulate and resolve this as a state-dependent context pathology—preserving history on-route while dynamically resetting it upon deviation, followed by GRPO recovery alignment. We will expand our related work and discussion sections to highlight this comparison and include [1–3].
[1] Yang et al., Behavioral Analysis of Vision-and-Language Navigation Agents, CVPR 2023. [2] Wen et al., Fighting Copycat Agents in Behavioral Cloning from Observation Histories, NeurIPS 2020. [3] Cheng et al., NaVILA: Legged Robot Vision-Language-Action Model for Navigation, RSS 2025.
Response to Reviewer 4WsE (Part 1/3): History Clearing in Training, Deployment, and Iter 30 Convergence (W1 / W2 / Q1)
Thank you for recognizing the motivation, analysis, and lightweight design of MOON. Due to the 5,000-character limit per comment, we organize our response to Reviewer 4WsE into three parts.
Response to W1 / W2 / Q1: Clearing During Training and Deployment
We respectfully clarify a critical nuance regarding Table 2: the 67.4% variant is NOT trained without history clearing. Both the 67.4% and 69.0% models use the exact same recovery policy trained with history clearing; they differ only in whether clearing is switched on or off during test-time deployment. Thus, the strong 67.4% baseline already embodies the massive benefits gained from cleared-history GRPO training.
Specifically, history clearing provides indispensable contributions across both training (as a learning catalyst) and deployment (as a persistent inertia breaker):
While the Keep-trained policy temporarily narrows the gap at Iter 19 (Table 11: 67.8% vs. 68.1%), it plateaus around 67.48% SR / 60.45% SPL through Iter 30, whereas the Clear-trained policy continues improving to 70.47% SR / 63.20% SPL (+2.99 pp SR / +2.75 pp SPL, exact McNemar p=2.6×10−4). Clearing history during training unlocks sustained optimization beyond the plateau of retained-history training, and clearing during deployment permanently removes the residual attention distraction that persists even after convergence.
We will make this distinction between training-time catalysis, post-convergence capacity, and test-time intervention crystal clear in Section 4.