acceptodds
Under review as a conference paper at ICLR 2027

MOON: Learning When to Forget and How to Move On in Vision-and-Language Navigation

Abstract

Recent vision-and-language navigation (VLN) policies act on their visual history, and an incorrect action can take them off the instructed route. Paired interventions on four VLN policies show that clearing the history doubles pooled success at the onset of a deviation but lowers it on route, while later in a deviation, removing only the off-route frames helps much less. After a deviation, the retained history keeps the policy from turning where a turn is needed, which we call behavioral inertia. We propose MOON, a framework that equips a frozen VLN policy with two lightweight modules, adding less than 1% of its parameters. A deviation judge decides when to forget, raising an alarm that clears the history when it predicts that the agent is off route. A recovery policy, trained with GRPO from alarm states, learns how to move on, turning more of the successes found only by sampling into a single greedy rollout. Experiments on R2R-CE and RxR-CE show that MOON reaches the highest R2R-CE success rate among the compared methods and raises the success rate of its frozen base by 5.2 and 6.0 points, and an ablation with a random trigger indicates that this gain depends on when the history is cleared. Without retraining, MOON also improves zero-shot object-goal navigation on HM3D-OVON and runs on a real robot. Together, these results suggest that deviation recovery depends on knowing when to forget, not only how to act.

Primary area:
applications to robotics, autonomy, planning
Keywords:
vision-and-language navigation, vision-language-action models, visual history, reinforcement learning

Reviews

Among:
Shape:
1234this paper, Top 1%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.50 over 2 reviews, above 99% of the others.
Reviewer_4WsEOpenReview ↗
Rating
4(+1)
Confidence
3
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

This paper addresses the challenge of effective recovery after an agent deviates from the instructed route in vision-and-language navigation (VLN). By investigating the impact of visual history on navigation decisions and deviation recovery, the authors propose two lightweight modules for deviation detection and navigation recovery. Specifically, the Deviation Judge determines whether the agent has deviated from the instructed route based on the navigation instruction, current observation, and historical information, and clears the visual history when an alarm is triggered. The Recovery Policy employs Group Relative Policy Optimization (GRPO) to optimize recovery trajectories sampled from deviation states using reward signals, thereby improving navigation recovery success rates. Experimental results demonstrate that the proposed method effectively improves navigation success rates while introducing only a small number of additional model parameters.

Critical strengths
  1. The paper presents a clear research motivation, focusing on the impact of historical information on deviation recovery in vision-and-language navigation. Through paired intervention experiments on different navigation policies, the authors demonstrate that historical information plays different roles during normal navigation and deviation recovery, providing valuable insights into historical dependence and error recovery in long-horizon navigation.

  2. The proposed MOON framework is relatively well-designed, achieving performance improvements through a lightweight deviation judge and a reinforcement learning-based recovery policy while keeping the parameters of the base navigation model frozen.

Critical weaknesses
  1. The paper takes behavioral inertia caused by historical information as its core motivation and attempts to improve navigation recovery by clearing visual history. However, as shown in Table 2, after introducing the GRPO-based recovery policy, the model without history clearing already achieves a success rate of 67.4%, which is not statistically significantly different from the 69.0% achieved by the full method.

  2. The training results in the appendix further show that the performance gap between retaining and clearing historical information decreases as reinforcement learning training progresses. The authors should further clarify the independent contribution of the history-clearing mechanism after sufficient training.

  3. The paper presents a compelling research motivation and provides relatively comprehensive analyses that effectively support its underlying motivation. However, the recovery policy primarily relies on standard trajectory-level rewards and group-relative advantage optimization, which are already widely adopted in reinforcement learning. The authors should further clarify the core technical novelty of MOON compared with existing navigation recovery and reinforcement learning methods.

Questions for authors
  1. According to Table 2, the GRPO-based recovery policy without history clearing already achieves performance close to that of the full method, with no statistically significant difference between them. The authors should further investigate whether a consistent performance gap remains between recovery policies with and without history clearing under the same training budget and after sufficient convergence.

  2. The paper argues that retaining historical information causes navigation policies to continue executing previous actions. However, is this phenomenon specific to the evaluated VLN models, rather than a general issue in VLN? Furthermore, have existing methods explicitly addressed or mitigated this tendency of navigation policies to persist in previous actions when historical information is retained?

5 comments
AuthorsOpenReview ↗

Response to Reviewer 4WsE (Part 1/3): History Clearing in Training, Deployment, and Iter 30 Convergence (W1 / W2 / Q1)

Thank you for recognizing the motivation, analysis, and lightweight design of MOON. Due to the 5,000-character limit per comment, we organize our response to Reviewer 4WsE into three parts.

Response to W1 / W2 / Q1: Clearing During Training and Deployment

We respectfully clarify a critical nuance regarding Table 2: the 67.4% variant is NOT trained without history clearing. Both the 67.4% and 69.0% models use the exact same recovery policy trained with history clearing; they differ only in whether clearing is switched on or off during test-time deployment. Thus, the strong 67.4% baseline already embodies the massive benefits gained from cleared-history GRPO training.

Specifically, history clearing provides indispensable contributions across both training (as a learning catalyst) and deployment (as a persistent inertia breaker):

Evaluation DimensionKeep HistoryClear History (MOON)Net Impact of History Clearing
1. Initial Training Exploration (pass@8↑\text{pass}@8 \uparrow)22.7%50.8%>2.2×>2.2\times valid GRPO training signals
2. Post-Training Inertia (No-Turn Error ↓\downarrow)30.9%6.4%∼5×\sim 5\times reduction in action locking
3. Test-Time Deployment (R2R-CE SR / SPL ↑\uparrow)67.4 / 60.569.0 / 62.3+1.6% SR / +1.8% SPL (0 extra params)
  1. Training Contribution: Unlocking GRPO Cold-Start (22.7% →\rightarrow 50.8%): GRPO relies entirely on sampling at least one successful rollout within a group to generate non-zero reward contrast. Without clearing, behavioral inertia severely suppresses exploration at initialization, yielding a dismal 22.7% pass@8\text{pass}@8 (leaving 77.3% of alarm states with all-failure rollouts and zero gradients). Clearing history instantly surges initial pass@8\text{pass}@8 to 50.8%, more than doubling the density of effective learning signals and making GRPO recovery training viable in the first place.
  2. Why the Deployment Gap (+1.6% SR / +1.8% SPL) is Highly Significant: First, in R2R-CE, a parameter-free test-time gain of +1.6% SR and +1.8% SPL is substantial, as episode-level metrics average across all trajectories (including easy, non-deviated episodes where the recovery policy is never triggered). Second, when we zoom into the actual deviation-onset states (110 matched states requiring a corrective turn) after full GRPO convergence (Iter 19), retaining history still locks the trained policy into a 30.9% no-turn failure rate, whereas clearing history slashes this failure rate by nearly 5×5\times to 6.4%.
  3. Persistent Advantage After Sufficient Convergence (Iter 30 Matched-Training Comparison): Even though parameter updates during GRPO partially teach the policy to resist historical interference, parameter learning alone cannot fully eradicate context-induced behavioral inertia. To directly answer W2 and Q1 regarding separately trained Clear vs. Keep policies under the same training budget and after sufficient convergence, we extended matched-iteration training to Iteration 30 on R2R-CE (N=1,839N = 1,839):
Iter.History (Training & Deployment)SR (%) ↑\uparrowSPL (%) ↑\uparrow
30Keep (GRPOkeep\text{GRPO}_{\text{keep}})67.4860.45
30Clear (MOON)70.4763.20

While the Keep-trained policy temporarily narrows the gap at Iter 19 (Table 11: 67.8% vs. 68.1%), it plateaus around 67.48% SR / 60.45% SPL through Iter 30, whereas the Clear-trained policy continues improving to 70.47% SR / 63.20% SPL (+2.99 pp SR / +2.75 pp SPL, exact McNemar p=2.6×10−4p = 2.6 \times 10^{-4}). Clearing history during training unlocks sustained optimization beyond the plateau of retained-history training, and clearing during deployment permanently removes the residual attention distraction that persists even after convergence.

We will make this distinction between training-time catalysis, post-convergence capacity, and test-time intervention crystal clear in Section 4.

AuthorsOpenReview ↗

Response to Reviewer 4WsE (Part 2/3): Recovery-Specific Technical Contribution Beyond GRPO (W3)

Response to W3: Technical Contribution of Recovery Learning

We thank the reviewer for raising this important point. While we intentionally adopt standard GRPO as a clean, critic-free optimizer, naively applying standard trajectory-level RL/GRPO to navigation recovery fails. In standard full-trajectory RL, rare deviation states are under-sampled, accumulated history locks exploration, and global trajectory rewards misattribute pre-deviation penalties.

MOON's core technical novelty lies not in modifying the GRPO loss formula, but in reformulating the RL Markov Decision Process (MDP) specifically for post-deviation recovery through three tightly coupled technical mechanisms:

Recovery RL BottleneckStandard Trajectory RL / Naive GRPOMOON's Recovery-Specific FormulationEmpirical Impact
1. State DistributionRollouts start from t=0t=0 (mostly on-route)First-Alarm State Branching: Branches rollouts directly from online-detected deviation statesConcentrates 100% compute on hard recovery states
2. Exploration CollapseSamples conditioned on full corrupted historyContext-Reset Exploration: Clears history prior to group sampling to break behavioral inertiaDoubles initial valid gradient groups (pass@8\text{pass}@8: 22.7% →\rightarrow 50.8%)
3. Credit ContaminationGlobal SPL penalizes pre-deviation detoursDecoupled Suffix-SPL Objective: Evaluates path efficiency strictly over the post-alarm suffixBoosts greedy recovery SR on 527 alarmed episodes (35.1% →\rightarrow 47.1%)

Specifically, MOON introduces three recovery-specific contributions that make GRPO viable and effective:

  1. Context-Intervened Group Exploration (Solving Gradient Starvation): Standard GRPO requires intra-group reward variance (at least one successful rollout among G=8G=8 samples) to compute non-zero advantages. Under standard retained history, behavioral inertia locks the policy into repeating the same error, yielding only 22.7% initial pass@8\text{pass}@8 (leaving 77.3% of states with zero gradients). By coupling selective history clearing directly with first-alarm state branching, MOON alters the conditioning distribution before rollout sampling, surging initial pass@8\text{pass}@8 to 50.8% and unlocking latent recovery trajectories for RL to reinforce.
  2. Decoupled Post-Alarm Credit Assignment (Suffix-SPL): Standard trajectory-level SPL evaluates the entire path from t=0t=0. If a base policy takes a long detour before triggering an alarm, standard SPL heavily penalizes the rollout even if the subsequent recovery is optimal—corrupting the advantage estimate. Our Suffix-SPL isolates the exact trajectory segment controlled by the recovery policy (from the cleared alarm pose to termination), ensuring clean, unbiased credit assignment for recovery efficiency.
  3. Bridging Sampling Potential to Greedy Execution (Without External Experts): Unlike prior navigation recovery methods (e.g., DAgger, CorrectNav) that require external expert oracles to synthesize corrective trajectories, MOON is completely self-supervised by execution outcomes. By aligning cleared-history group rollouts via Suffix-SPL, MOON successfully distills multi-sample recovery potential into single-run greedy execution—boosting greedy success on 527 alarmed R2R-CE episodes from 35.1% to 47.1%.

We will explicitly contrast our recovery-specific MDP formulation against standard trajectory-level RL in Section 3 to make this technical contribution unmistakable.

AuthorsOpenReview ↗

Response to Reviewer 4WsE (Part 3/3): Generality of Behavioral Inertia Across VLN Architectures (Q2)

Response to Q2: Generality of Behavioral Inertia

We thank the reviewer for this insightful question. Below, we address both the generality of behavioral inertia across VLN architectures and why existing training-time mitigations fail to resolve it during recovery:

  1. Generality Across Diverse VLN History Architectures: Rather than an artifact of a specific model, behavioral inertia is a systemic, architecture-agnostic phenomenon across modern VLN policies. Our four evaluated models deliberately span the primary paradigms for encoding visual history in VLN—from explicit frame sampling to implicit KV-cache and spatial-temporal token merging. Evaluating matched deviation-onset states (following a forward step where the shortest-path expert turns) reveals severe action-locking across all memory mechanisms when history is retained, which immediately collapses upon clearing history:
VLN PolicyHistory Representation ParadigmKeep History (No-Turn ↓\downarrow)Clear History (No-Turn ↓\downarrow)Expert
NaVILA [3]Fixed-frame uniform sampling68.7%5.5% (↓12.5×\downarrow \mathbf{12.5\times})1.1%
Uni-NaVidOnline spatial-temporal token merging52.1%12.6% (↓4.1×\downarrow \mathbf{4.1\times})3.3%
NaVIDAHierarchical historical token compression35.0%11.9% (↓2.9×\downarrow \mathbf{2.9\times})1.2%
StreamVLNStreaming KV-cache memory33.1%13.6% (↓2.4×\downarrow \mathbf{2.4\times})1.8%

(Note: "No-turn" denotes failing to execute any turning action in the subsequent action chunk, where the expert turns >96%>96\% of the time.)

  1. Relation to Prior Work & Why Existing Mitigations Fall Short: Prior literature has examined related phenomena from two angles, neither of which resolves post-deviation context locking:
    • Unconditional Action Bias vs. Context-Conditioned Inertia: Prior VLN analysis [1] identifies a global training-induced forward bias due to action imbalance, and NaVILA [3] explicitly mitigates this during training via action-label rebalancing. Crucially, our table above proves that data rebalancing does not solve behavioral inertia: despite balanced training, NaVILA still suffers a massive 68.7% no-turn rate under retained history, which plummets to 5.5% immediately upon clearing history using the exact same model weights. This proves the bottleneck is test-time historical context conditioning, not static parameter bias.
    • Global History Removal vs. Dynamic Selective Intervention: In general imitation learning, works like Fighting Copycat Agents [2] combat temporal action-copying shortcuts by penalizing or discarding historical information globally during training (e.g., via adversarial representation learning). However, in long-horizon VLN, history is essential when on-route to track instruction progress (as shown in Sec. 2, globally removing history degrades on-route SR).

Summary of MOON's Distinction: Prior methods either rebalance static action priors [1, 3] (which leaves context-induced inertia intact upon deviation) or suppress reliance on history globally [2] (which hurts on-route navigation). MOON is the first to formulate and resolve this as a state-dependent context pathology—preserving history on-route while dynamically resetting it upon deviation, followed by GRPO recovery alignment. We will expand our related work and discussion sections to highlight this comparison and include [1–3].

[1] Yang et al., Behavioral Analysis of Vision-and-Language Navigation Agents, CVPR 2023. [2] Wen et al., Fighting Copycat Agents in Behavioral Cloning from Observation Histories, NeurIPS 2020. [3] Cheng et al., NaVILA: Legged Robot Vision-Language-Action Model for Navigation, RSS 2025.

Reviewer_YsLROpenReview ↗
Rating
3(+1)
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper addresses the deviation problem in Vision-Language Navigation (VLN). Through an empirical study, it finds that historical observations prevent agents from recovery after deviation, while clearing the history after deviation helps. Motivated by this, it proposes MOON, a VLN framework consisting of a judge module and a recovery module. The judge module estimates a deviation score, which triggers history clearing when it exceeds a calibrated threshold. The recovery module is optimized with GRPO-based reinforcement learning on trajectories sampled from deviation states, with a dedicated Suffix-SPL reward. Experimental results on R2R-CE and RxR-CE demonstrate consistent improvements, and ablation studies validate the effectiveness of key designs.

Critical strengths

This paper provides a comprehensive analysis of visual history for deviation recovery. In particular, the finding that removing off-route observations is substantially less effective than clearing the entire history is interesting and nontrivial, which provides a fresh perspective of the fundamental compounding error problem in VLN, and valuable insights for designing more robust history-conditioned VLN agents.

Critical weaknesses
  1. The contribution of the proposed MOON framework is incremental, as it is essentially a data augmentation method with history clearing mechanism. However, the central idea detect-then-recovery has already been widely explored in literature (e.g., ReflectVLN), while learning corrective behaviors from off-route trajectories is also well established (e.g., Dagger, CorrectNav).
  2. Section 2 presents extensive empirical studies, but several experiments (e.g, History Length, History Content) primarily report quantitative observations without sufficiently explaining their motivation, or drawing clear conclusions. Moreover, the connection between these empirical findings and the subsequent method design is not adequately established, for example, it finds retaining on-route history is less effective than clearing all history on a frozen policy. However, this finding is not further validated within the proposed framework, where the recovery policy is explicitly optimized.
  3. The main text frequently refers readers to the appendix for important methodological details and experimental explanations, undermining the self-containedness and coherence of the paper. Additionally, there are many grammar issues and unclear terms that affect understanding, e.g., 'pooled success', 'Pooled rates weight every state of the four policies equally.', 'The judge alarms in 56.3% of the episodes the base fails on R2R-CE and 45.8% on RxR-CE.'.
Questions for authors
  1. Could the authors clarify the methodological novelty and significance of MOON compared with existing deviation detection and recovery approaches?
  2. Do the findings from the frozen-policy analyses in Section 2 still hold after recovery-policy training?
  3. Could the authors better explain the motivation and conclusions of each experiments in Section 2?
  4. Why is deviation detection disabled after the first alarm during recovery rollouts? How does the method handle subsequent deviations, and what impact might they have on the sampling efficiency of GRPO-based recovery training?
4 comments
AuthorsOpenReview ↗

Response to Reviewer YsLR (Part 1/4): Novelty Beyond Corrective Data (W1 / Q1)

Thank you for recognizing our history-intervention analysis and providing thoughtful feedback. Due to the 5,000-character limit per comment, we organize our response to Reviewer YsLR into four parts.

Response to W1 / Q1: Novelty Beyond Corrective Data

The core distinction is that existing methods treat recovery as a supervision/guidance deficit, whereas MOON formulates it as a context-conditioning pathology (history-induced behavioral inertia).

MethodRoot Cause AddressedTreatment of History ContextRecovery Mechanism
ReflectVLNDelayed plan correctionRetains full history + appends feedbackPrompt-based reflection
DAggerState-distribution shiftRetains full historyExpert label aggregation
CorrectNavPerception/action errorsRetains full historyCorrective trajectory SFT
MOON (Ours)History-induced inertiaSelectively clears history upon deviationContext reset + Self-rollout GRPO
  1. Orthogonal Problem Formulation: Prior methods assume full history is benign and add corrective signals or trajectories on top of it. However, our matched-state interventions across four frozen policies reveal that accumulated history actively hinders recovery: removing only post-deviation frames yields 16.3% success, vs. 26.3% when clearing all history at the same states. MOON is the first to isolate and intervene on this context-level failure mode.
  2. Mechanism (Subtraction vs. Addition): Rather than appending corrective prompts (ReflectVLN) or expanding training distributions (CorrectNav), MOON's judge performs selective forgetting—preserving history on-route while resetting the context upon deviation to break behavioral inertia.
  3. Learning Paradigm (Latent Alignment vs. External Supervision): Clearing history frees the policy from inertia, exposing latent recovery capabilities during sampling that still fail under greedy decoding. Unlike DAgger/CorrectNav which inject external expert trajectories, MOON applies GRPO to the policy's own rollouts from cleared states, directly distilling its own sampled recovery potential into reliable greedy execution.

We will sharpen this conceptual and methodological contrast in the revision.

AuthorsOpenReview ↗

Response to Reviewer YsLR (Part 2/4): History Effects After Recovery Training (W2 / Q2)

Response to W2 / Q2: History Effects After Recovery Training

Yes, all core findings from the frozen-policy analyses in Section 2 consistently hold after GRPO recovery training.

To verify this directly, we repeated the exact matched-state causal interventions using the fully trained recovery policy (fixing model weights, simulator pose, instruction, and current observation while varying only historical context). Across all dimensions—on/off-route selectivity, content scope, window length, and behavioral inertia—the post-training patterns mirror Section 2:

Analysis DimensionHistory ConditionBefore Training (SR %)After Training (SR %)Post-Training Conclusion (Consistent with Sec. 2)
Baseline (Off-Route, N=175N=175)Full History (Keep)18.338.3Suffers severe interference even after RL training
1. History Content (Off-Route, N=175N=175)Prefix (up to deviation onset)20.640.0Removing only off-route frames remains insufficient
1. History Content (Off-Route, N=175N=175)On-route (strict on-route prefix)26.339.4Pre-deviation history still anchors failed trajectories
2. History Length (Off-Route, N=175N=175)Last 16 steps29.741.0Shortening the memory window partially mitigates
2. History Length (Off-Route, N=175N=175)Last 8 steps30.342.3interference, but no fixed truncation window
2. History Length (Off-Route, N=175N=175)Last 4 steps37.744.0matches full history clearing
2. History Length (Off-Route, N=175N=175)Last 2 steps36.045.1
MOON InterventionClear All History44.646.3Achieves highest recovery SR across all conditions

Furthermore, evaluating on-route selectivity and action-level behavioral inertia after full training confirms:

  1. On/Off-Route Selectivity Persists: On 396 matched on-route states, retaining full history achieves 75.5% SR vs. 59.6% when cleared (↓15.9%\downarrow 15.9\%), whereas on the 175 off-route states above, clearing beats full history by 46.3% vs. 38.3% (↑8.0%\uparrow 8.0\%). This confirms that selective, deviation-triggered intervention remains essential after training.
  2. History-Content Finding Persists: After training, deleting only post-deviation frames (On-route prefix: 39.4%, Prefix: 40.0%) still substantially underperforms clearing all history (46.3%), proving that even clean pre-deviation history continues to hinder recovery once off-route.
  3. History-Length Finding Persists: Across all tested sliding windows (Last 2, 4, 8, 16), performance monotonically improves as less history is kept (from 41.0% at Last 16 up to 45.1% at Last 2), yet full clearing (46.3%) remains strictly superior to any fixed window truncation.
  4. Behavioral-Inertia Finding Persists: On 110 eligible deviation-onset states (where the expert turns), retaining history still locks the trained policy into a 30.9% no-turn error rate, which plummets to 6.4% (↓24.5\downarrow 24.5 percentage points) upon clearing history.

(Note: While RL parameter updates also improve the policy's robustness to corrupted history on this 175-state subset, full clearing consistently provides the upper bound—and on all 527 alarmed R2R-CE episodes, GRPO training boosts cleared-history greedy success from 35.1% →\rightarrow 47.1%.) We will include this complete post-training diagnostic table in the revised manuscript.

AuthorsOpenReview ↗

Response to Reviewer YsLR (Part 3/4): From Section 2 Analysis to Method Design (W2 / Q3)

Response to W2 / Q3: From Analysis to Method Design

We thank the reviewer for pointing this out. Section 2 is designed as a progressive diagnostic chain across four frozen policies (evaluating pooled matched-state interventions). Rather than isolated tests, the seven experiments systematically rule out naive heuristics and directly dictate each architectural choice in MOON (Section 3).

Specifically, the experiments unfold in two logical phases: Phase I (Exps 1–5) determines when and how to intervene on history, while Phase II (Exps 6–7) determines why and how to train the policy after intervention.

Phase & ExperimentCore Question AnsweredEmpirical Finding (Pooled across 4 Policies)Direct Design Implication for MOON (Sec. 3)
I-1. On/Off-RouteIs history globally harmful or state-dependent?Clearing history boosts off-route SR (↑\uparrow), but degrades on-route SR (↓\downarrow).Requires a Deviation Judge: Cannot clear blindly; must preserve history on-route and intervene only upon deviation.
I-2. Window LengthIs naive sliding-window truncation sufficient?Full clearing outperforms all tested fixed window lengths off-route.Rules out simple truncation: A fixed short memory window cannot resolve recovery failure.
I-3. Content ScopeIs removing only post-deviation (corrupted) frames enough?Removing only off-route frames yields 16.3% SR, vs. 26.3% when clearing all history.Mandates Full Context Reset: Even pre-deviation history anchors the agent to the wrong path; intervention must clear all prior frames.
I-4. PerturbationsIs failure caused by specific frame order/noise?Shuffling, swapping, repeating frames, or adding noise fails to resolve recovery deficits.Identifies a systemic bottleneck: The issue is the mere presence of accumulated context, motivating a behavioral examination.
I-5. Behavioral InertiaHow does history physically block recovery at the action level?At matched states requiring a turn, full history locks agents into moving forward; clearing significantly reduces no-turn errors (↓\downarrow).Confirms "Behavioral Inertia": Explains why full reset works—it frees the policy from momentum-like action locking.
II-6. Greedy OracleIs dynamic history clearing alone sufficient for full recovery?Even an oracle choosing the best greedy rollout (retained vs. cleared) fails on 65.0% of hard states.Necessitates Recovery Learning: Context intervention alone hits a ceiling under greedy decoding; parameter updates are required.
II-7. Sampling (pass@8\text{pass}@8)Do we need external expert labels, or does latent capability exist?On 320 greedy-failed states, sampling 8 rollouts with cleared history boosts success (pass@8\text{pass}@8) from 30.3% →\rightarrow 61.6%.Directly motivates Self-Rollout GRPO: Clearing history unlocks massive latent recovery potential; GRPO distills these successful self-rollouts into reliable greedy execution without external experts.

Summary of the Section 2 →\rightarrow Section 3 Pipeline:

  • Exps 1–5 prove that history causes behavioral inertia upon deviation that cannot be fixed by window truncation or partial deletion   ⟹  \implies motivating MOON's Judge-Triggered Full History Clearing.
  • Exps 6–7 prove that while clearing history unlocks latent recovery capability in multi-sample rollouts, single-run greedy execution remains brittle   ⟹  \implies motivating MOON's GRPO Recovery Learning on cleared first-alarm states.

We will restructure Section 2 with explicit "Question–Finding–Implication" callout boxes and add a summary transition diagram to make this deductive flow unmistakable in the revised manuscript.

AuthorsOpenReview ↗

Response to Reviewer YsLR (Part 4/4): Main-Text Clarity and Single-Alarm GRPO Sampling (W3, Q4)

Response to W3: Main-Text Clarity and Metric Definitions

We will revise the manuscript to make the main text fully self-contained and resolve all terminology and clarity issues:

  1. Self-Contained Methodological Details: We will consolidate the state-selection and paired-intervention protocols directly into Section 2, move first-alarm execution and recovery training start-state construction into Section 3, and separately describe the data sources used for threshold calibration vs. alarm-level selection in the main text.
  2. Exact Definition of "Pooled Success Rate": We will define "pooled success rate" at first use and apply it consistently: it divides the total number of successful continuations across all four policies by the total number of evaluated state samples under each analysis condition, weighting every state sample equally (rather than macro-averaging policy-level rates).
  3. Explicit Denominators for Alarm Coverage: The reported alarm percentages—56.3% on R2R-CE (375/666375 / 666 base-failure episodes) and 45.8% on RxR-CE (589/1,286589 / 1,286 base-failure episodes)—measure alarm coverage (recall) over episodes where the frozen base policy alone fails, not the fraction successfully rescued by MOON.
  4. Terminology & Copyediting: We will make all denominators explicit, add sample counts to all relevant captions, standardize "state," "episode," and "alarm," and thoroughly proofread the manuscript.

Response to Q4: Single-Alarm Recovery and GRPO Sampling

  1. Why one alarm? The first clearing breaks pre-existing behavioral inertia, after which the agent must accumulate fresh observations to re-establish spatial tracking. A second reset wipes out this newly rebuilt recovery context, inducing spatial amnesia. Empirically (using the same base/recovery policies), allowing 2 clearings hurts SR compared to 1 clearing, whereas using a ground-truth single trigger raises R2R-CE SR from 69.00% →\rightarrow 71.23%, proving that initial trigger precision matters more than multiple resets:
Max ClearingsGRPO RLR2R-CE SR (%)RxR-CE SR (%)
1No65.5865.69
1 (Ours)Yes69.0070.90
2Yes68.2467.65
  1. Handling later deviations: After the single alarm, the policy processes new observations continuously until termination. Because GRPO optimizes Suffix-SPL over the entire remaining trajectory suffix, subsequent deviations or detours directly penalize the rollout's reward, training the policy to correct secondary drifts organically without hard resets.
  2. Impact on GRPO sampling: Later deviations create detours or failures within the 8 sampled suffixes. Whenever a group contains at least one success, these suboptimal rollouts provide valuable negative contrast via lower Suffix-SPL rewards (while all-failure groups yield zero contrast). Tracking complete-suffix pass@8\text{pass}@8 (percentage of alarm states with ≥1\ge 1 successful rollout) across training batches shows that valid contrastive signals are abundant and steadily improve as secondary deviations are suppressed:
IterationStatesComplete-Suffix pass@8\text{pass}@8 (%)
Iter 0 / 9 / 1912850.8% →\rightarrow 69.5% →\rightarrow 77.3%

We will clarify the single-alarm rationale, full-suffix credit assignment, and sampling dynamics in the revision.

Comments on the paper

8 comments
AuthorsOpenReview ↗

Response to Reviewer wVDj (Part 1/2): Paired Evidence and Iter 30 Convergence Results (W1 / Q2)

Thank you for the detailed assessment of the matched-state analysis and system improvements. Due to the 5,000-character limit per comment, we organize our response to Reviewer wVDj into two parts.

Response to W1 / Q2: Paired Evidence for Clearing

We thank the reviewer for this rigorous statistical scrutiny. To properly evaluate the contribution of history clearing ("when to forget"), it is essential to separate global dilution across un-alarmed episodes from actual recovery states, and to examine both deployment significance and training-time sample efficiency:

  1. Paired Deployment Evidence & Statistical Significance (p=0.016p = 0.016 on RxR-CE): Evaluating the fixed clear-trained policy across all validation episodes (1,839 on R2R-CE; 3,669 on RxR-CE) by toggling test-time clearing yields consistent net gains on both benchmarks, reaching statistical significance on the larger, longer-horizon RxR-CE benchmark (p=0.016p = 0.016, two-sided exact McNemar test):
Benchmark / ScopeTotal NNClear-Only WinsKeep-Only WinsNet Episode Gain / Ratepp-value / Significance
RxR-CE (All Episodes)3,669158117+41 episodesp=0.016∗p = 0.016^* (Significant)
R2R-CE (All Episodes)1,83910184+17 episodesp=0.239p = 0.239 (Diluted by un-alarmed runs)
R2R-CE (Deviation-Onset States)110——No-Turn Error: 30.9% →\rightarrow 6.4%4.8×4.8\times error reduction (p<0.001p < 0.001)

Why is p=0.239p = 0.239 on full R2R-CE yet highly significant on RxR-CE and localized states? In full-dataset evaluation, the majority of episodes never trigger an alarm (where Clear and Keep behave identically), and stochastic downstream navigation after an alarm adds variance to binary episode outcomes. On R2R-CE (N=1,839N=1,839), this dilution masks statistical power, whereas on RxR-CE (N=3,669N=3,669, 2×2\times larger sample size and longer trajectories), Clear-only wins significantly outnumber Keep-only wins (158 vs. 117, p=0.016p = 0.016). Moreover, when isolating the exact deviation-onset states after full RL convergence (Iter 19), clearing slashes no-turn action locking from 30.9% to 6.4%—proving a profound, persistent behavioral effect that full-episode averaging dilutes.

  1. Explaining Table 11 & Matched-Convergence Results at Iter 30: In Table 11, why does a policy trained without clearing temporarily narrow the global SR gap at Iter 19 (68.1% vs. 67.8%)?
    • Training Catalyst vs. Gradient Starvation: At initialization (Iter 0), retaining history locks exploration, yielding only 22.7% pass@8\text{pass}@8 (meaning 77.3% of alarm states produce all-failure rollouts and zero GRPO gradients). Clearing history instantly doubles initial pass@8\text{pass}@8 to 50.8%, unlocking rich contrastive signals from the very first iteration.
    • Parameter Compensation Hits a Ceiling, While Clearing Keeps Improving (Iter 30): Over extended RL iterations, the Keep-trained policy slowly uses brute-force parameter updates to partially suppress attention to corrupted history, narrowing the SR gap around Iter 19. However, this compensation saturates and leaves severe residual inertia at critical turning states (30.9% vs. 6.4% post-training no-turn error). Extending the matched-iteration comparison of separately trained Clear vs. Keep policies to sufficient convergence (Iteration 30 on R2R-CE, N=1,839N = 1,839) shows that Keep plateaus at 67.48% SR / 60.45% SPL, whereas Clear continues improving to 70.47% SR / 63.20% SPL (+2.99 pp SR / +2.75 pp SPL), with highly significant paired episode counts:
Iter.BenchmarkTotal NNClear-Only WinsKeep-Only WinsΔ\DeltaSR (Clear vs. Keep)Exact McNemar pp
30R2R-CE1,83913883+2.99 pp (70.47% vs. 67.48%)p=2.6×10−4p = 2.6 \times 10^{-4}
  1. Stability Consideration: While running multiple full-scale multi-iteration VLM-RL training seeds is computationally prohibitive, the consistent advantage of clearing is independently corroborated across (i) two benchmarks totaling 5,508 paired episodes (p=0.016p=0.016 on RxR-CE; p=2.6×10−4p = 2.6 \times 10^{-4} on R2R-CE at Iter 30 convergence), (ii) four distinct frozen VLN architectures in Sec. 2, and (iii) multiple training checkpoints (where cleared exploration consistently maintains high pass@8\text{pass}@8 from 50.8% →\rightarrow 69.5% →\rightarrow 77.3% across Iterations 0, 9, and 19). We will include these exact paired McNemar tables and clarify the statistical dilution effect in the revised manuscript.
AuthorsOpenReview ↗

Response to Reviewer wVDj (Part 2/2): Checkpoint Selection, LOSO Alarm Selection, and NaVILA-8B Transfer (W2 / Q1, Q3, Q4)

Response to W2 / Q1: Checkpoint Selection and Aligned Comparisons

  1. Disjoint Checkpoint Selection (iter15): We selected iter15 as the checkpoint achieving the highest recovery success rate on a separately held-out validation set that is strictly disjoint from the R2R-CE and RxR-CE val-unseen splits used for final reporting. We will explicitly document this held-out validation split and checkpoint-selection criterion in the revised manuscript.
  2. Aligned Iteration-15 Comparison (Replacing Table 4): The aligned comparison below evaluates all Table 4 variants at the same iteration (iter15) across all 1,839 R2R-CE val-unseen episodes (GRPO_start trains from episode starts; GRPO_keep and MOON train from alarm states, retaining and clearing history, respectively):
MethodStart StateHistorySR (%) ↑\uparrowSPL (%) ↑\uparrow
SFTAlarmClear62.7557.90
GRPO_startStep 0—65.8559.33
GRPO_keepAlarmKeep67.3761.00
MOON (Ours)AlarmClear69.062.3

At the aligned iteration, MOON retains the highest SR and SPL—exceeding GRPO_keep by +1.63 pp SR and GRPO_start by +3.15 pp SR. We will replace Table 4 with this aligned comparison and report each variant's exact training budget.


Response to W2 / Q3: Held-Out Alarm-Level Selection

Under leave-one-scene-out (LOSO) alarm-level selection, MOON consistently outperforms the frozen base policy on both benchmarks, achieving gains of +4.02 pp SR on R2R-CE and +2.56 pp SR on RxR-CE without tuning the alarm level on the evaluated scene:

  1. LOSO Evaluation Protocol: Fixing the recovery checkpoint at iter15, each of the 11 folds selects the alarm level α\alpha maximizing SR on 10 val-unseen scenes and evaluates that α\alpha on the held-out 11th scene. Pooling across all 11 folds counts every episode exactly once (1,839 R2R-CE and 3,669 RxR-CE episodes):
MethodR2R-CE (SR / SPL %)RxR-CE (SR / SPL %)
Base (Frozen)63.78 / 59.0464.95 / 55.94
MOON (main, reported α\alpha)69.0 / 62.370.9 / 60.8
MOON (LOSO held-out)67.81 / 60.1767.51 / 56.52
  1. Cross-Scene Robustness: On both benchmarks, 10 of 11 folds independently select the reported α\alpha. Combined with the held-out gains above, this confirms that alarm-level selection generalizes reliably across unseen scenes. We will add the held-out SR/SPL results and full LOSO protocol to Appendix D.3.

Response to Q4: Transfer to a Second Base Policy

Yes—the complete MOON framework transfers effectively to a second VLN backbone, NaVILA-8B (Llama 3), raising SR from 50.50% to 53.50% (+3.00 pp SR / +2.16 pp SPL) while keeping all base policy weights frozen.

We adapted both lightweight modules (deviation judge and GRPO recovery policy) to NaVILA-8B under a small training budget and evaluated all variants on the same 600-episode R2R-CE val-unseen subset:

MethodSR (%) ↑\uparrowSPL (%) ↑\uparrow
NaVILA-8B (Frozen Base)50.5047.36
+ Judge and Clearing51.9848.02
+ Full MOON (Judge + Clearing + Recovery)53.5049.52

Judge-triggered clearing improves over the frozen base (+1.48 pp SR), and adding the GRPO recovery policy delivers a further +1.52 pp SR and +1.50 pp SPL improvement, confirming that both components transfer constructively to another architecture.

AuthorsOpenReview ↗

Summary of Reviewer Consensus, Key Strengths, and Manuscript Revisions

Dear Area Chair and Reviewers,

We sincerely thank all reviewers for their constructive feedback. We have uploaded the revised manuscript (updates in blue) to incorporate the clarifications and analyses discussed in our responses. Below, we summarize the core strengths recognized across reviews and provide a section-by-section guide to the manuscript revisions.

Part I: Consensus Strengths Across All Reviews

  1. Fresh Perspective & Nontrivial Discovery on VLN Compounding Errors (YsLR, 4WsE, wVDj): All reviewers highlighted our matched-state causal analysis across four VLN architectures (wVDj called it "the paper's most valuable contribution"; YsLR noted it "provides a fresh perspective of the fundamental compounding error problem in VLN"). In particular, reviewers valued the nontrivial finding that accumulated history induces behavioral inertia (locking deviated agents into forward movement when a turn is required), and that deleting only post-deviation frames is far less effective than a full history reset (YsLR, wVDj).
  2. Well-Designed, Lightweight (<1% Params) & Generalizable Framework (4WsE, wVDj, YsLR): Reviewers recognized that MOON effectively translates these insights into a modular framework on a frozen backbone (+0.81% params, +1.8% latency), achieving state-of-the-art R2R-CE SR (69.0%), strong gains over the frozen base (+5.2 / +6.0 pp SR on R2R-CE / RxR-CE), and zero-shot transfer to HM3D-OVON (+16.5 pp SR) and a physical robot (4WsE, wVDj).

Part II: Guide to Manuscript Revisions (Highlighted in Blue)

  1. Clarifying Training vs. Deployment History Clearing (4WsE W1/W2/Q1, wVDj W1/Q2):
    • Sec. 4.3 (pp. 9–10) & App. D.6 (Table 23, p. 30): Clarified that the 67.4% variant in Table 2 is trained with clearing and only disables clearing at deployment, whereas Table 4 (GRPO_keep) evaluates training without clearing. Added paired deployment counts showing statistical significance on the larger RxR-CE split (158 clear-only vs. 117 retain-only wins, exact McNemar p=0.016p=0.016) and a 5×5\times reduction in post-training no-turn errors (30.9% -> 6.4%).
    • Sec. 4.3 (p. 10) & App. D.7 (Table 24, p. 30): Documented how clearing prevents GRPO cold-start starvation (pass@8: 22.7% -> 50.8%) and included the matched Iteration-30 convergence comparison (70.47% SR / 63.20% SPL with clearing vs. 67.48% SR / 60.45% SPL without, p=2.6×10−4p=2.6\times 10^{-4}).
  2. Deductive Flow (Sec. 2 -> Sec. 3) & Self-Contained Main Text (YsLR W2/Q2/Q3, W3, 4WsE Q2):
    • Secs. 2.2–2.4 (pp. 4–5) & App. B.6 (Table 11, Fig. 8, p. 21): Added Design Implication I–III callouts at the end of each subsection in Sec. 2 to explicitly connect every empirical finding to MOON's design, supported by a diagnostic summary table and transition diagram in App. B.6.
    • Secs. 2.2, 3.2, 4.1, 4.2 (pp. 3–8) & App. H (p. 41): Consolidated state-population protocols, training start-state mining (5,510 alarm states), loss normalization weights, and the separation between threshold calibration and alarm-level selection directly into the main text. Defined pooled SR at first use and standardized explicit alarm denominators (375/666 on R2R-CE; 589/1,286 on RxR-CE).
    • App. A.4 (p. 17) & App. B.7 (Tables 12–13, Figs. 9–11, pp. 21–24): Included the post-training history diagnostics confirming that all Sec. 2 findings persist after RL, and expanded the discussion on behavioral inertia across VLN memory paradigms.
  3. Methodological Distinctions, Aligned Ablations & Generalization (YsLR W1/Q1/Q4, 4WsE W3, wVDj W2/Q1/Q3/Q4):
    • Sec. 1 (p. 2), Sec. 3.2 (p. 7), App. A.3 (Table 6, p. 17) & App. D.2 (Tables 17–18, p. 28): Contrasted context resetting against corrective supervision (ReflectVLN, DAgger, CorrectNav), detailed MOON's three recovery-specific MDP mechanisms in Sec. 3.2, and documented single- vs. multi-alarm execution and pass@8 progression (50.8% -> 69.5% -> 77.3%).
    • Sec. 4.1 (p. 7), Sec. 4.3 (Table 4, p. 9), App. C.9, D.1 (p. 27), D.3 (Table 19, p. 29) & D.8 (Table 25, p. 31): Documented disjoint checkpoint selection, aligned all Table 4 baselines at Iteration 15, included scene-held-out (LOSO) alarm selection results, and documented framework adaptation to frozen NaVILA-8B (50.50% -> 53.50% SR).

We warmly welcome any further feedback during the discussion period!

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.