acceptodds
Desk rejected at ICLR 2027

Learning Navigation Corrections from a Weak Recovery Policy

Desk rejected at ICLR 2027, so left out of the paper lists and the map.

Abstract

Learning to recover from navigation errors can disrupt actions that an agent already performs well. Yet a recovery policy that is unsuitable for deployment may still offer useful local supervision. We propose RCSR-D, a training procedure that separates the usefulness of a policy's action proposals from its ability to navigate complete trajectories. A frozen recovery policy supplies candidate turns. An offline rule selects proposals under empirical retention criteria, and a student initialized from the original policy learns the selected first actions while matching reference behavior on the remaining states. Parameter interpolation moderates the learned update before deployment. On the same reserved 128-episode cohort in the navigation-tuned 9B setting, the original and recovery policies achieve 33.59% and 6.25% success, respectively, while RCSR-D students achieve 42.97–51.56% across three training seeds. On the full 1,839-episode R2R-CE validation-unseen split, the primary student improves success from 37.25% to 46.93% and success weighted by path length from 29.36% to 36.28%. These results show that a recovery checkpoint can remain a useful training resource despite poor navigation performance. RCSR-D provides a way to convert its local proposals into improved navigation while deploying a single student.

Primary area:
applications to robotics, autonomy, planning
Keywords:
Vision-and-language navigation, Corrective post-training, Knowledge distillation, Reference-policy retention, Embodied AI, Vision-language models

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_8iFSOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes RCSR-D, a corrective post-training method for Vision-and-Language Navigatio designed to extract local supervision from a weak recovery policy without deploying it. When evaluated as a full navigator on a reserved 128-episode cohort, a recovery-tuned Qwen3.5-9B policy fails severely (achieving only 6.25% Success Rate vs. 33.59% for the original policy). RCSR-D bridges this gap by decoupling supervision utility from trajectory execution: an offline margin-based routing rule identifies confident corrective turn proposals, a student model initialized from the original navigator trains via cross-entropy on these turns while matching the original policy via top-k KL divergence on all other states, and parameter interpolation (α=0.5) moderates the final weight update. On the 1,839-episode R2R-CE val-unseen benchmark, RCSR-D improves the 9B model's SR from 37.25% to 46.93% (+9.68 pp) and SPL from 29.36% to 36.28% (+6.92 pp), deploying only a single student policy at runtime.

Critical strengths
  1. Valuable Conceptual Insight: Decouples the utility of a policy's action proposals from its closed-loop navigation competence, establishing that a checkpoint with catastrophic full-trajectory failure can still serve as an effective local teacher.

  2. Clean Single-Agent Deployment: Unlike runtime mixture-of-experts or multi-policy cascading schemes, the offline selection rule and recovery policy are discarded after distillation, resulting in a single deployed student model using the original inference interface without extra communication or model switching.

  3. Statistically Substantial Gains: On the complete 1,839-episode R2R-CE val-unseen split, the method yields statistically robust gains across all metrics (SR +9.68 pp, SPL +6.92 pp, Navigation Error -1.73 m), with 95% bootstrap intervals safely excluding zero and consistent improvements across all 11 evaluation scenes.

  4. Rigorously Controlled Baselines: Validates the distillation strategy against direct weight interpolation (θ_0+0.5(θ_r-θ_0 )), demonstrating that RCSR-D significantly outperforms direct parameter blending (+19.34 pp SR on a 512-episode cohort).

Critical weaknesses
  1. Inconclusive Ablations for Core Algorithmic Components: The margin-based routing rule (g_τ) fails to show a statistically distinguishable SR advantage over stratified random delegation (+0.78 pp difference, with a 97.5% CI of [-9.38,+10.94]pp).

    Soft KL retention shows no statistically significant SR improvement over standard hard-label CE preservation (+5.47 pp difference with a 97.5% CI of [-3.91,+14.84]pp on the 9B model, and slightly negative point estimates in the StreamVLN experiments).

  2. Heuristic and Fragile Calibration Pipeline: Thresholds (τ=2.25, α=0.5) and tolerances (e.g., 5-point preservation drops, 3% false Stop limits) are tuned via multi-step heuristics on training and development slices; as shown in Appendix D, identical thresholds failed to meet constraints when applied to StreamVLN.

  3. Inconsistent Generalization Across Backbones: Gains on native-action architectures are noticeably smaller or statistically ambiguous (e.g., anchored NaVIDA achieves only a +0.65 pp SR gain over the baseline, with a 95% CI crossing zero at [-1.36,+2.61]pp).

Questions for authors
  1. Given that margin-based delegation does not statistically outperform stratified random delegation on the 128-episode cohort (Δ"SR"=+0.78 pp, 97.5% CI includes zero), is the specific margin-thresholding mechanism strictly necessary, or does the benefit derive primarily from exposing the student to any balanced mix of corrective turns alongside reference retention?

  2. In both the 9B cohort (Table 4) and the StreamVLN experiments, the SR differences between soft KL and hard-label CE cross zero (with KL yielding lower point estimates in StreamVLN). In light of this, what is the empirical justification for preferring top-k KL divergence over standard, computationally simpler cross-entropy for reference preservation?

  3. Parameter interpolation is fixed at α=0.5 based on development checks. How sensitive is the final closed-loop navigation performance on R2R-CE to the choice of α(e.g., across a sweep of α∈{0.0,0.25,0.5,0.75,1.0})?

  4. Why does the anchored selective-distillation approach exhibit marginal and inconclusive improvements on NaVIDA (+0.65 pp SR, CI crossing zero) compared to the substantial +9.68 pp gain observed on the Qwen3.5-9B model?

2 comments

Thank you for the careful review and for highlighting the distinction between useful local proposals and poor navigation performance. We address your questions and the calibration concern below.

  1. Is margin selection necessary?

The current comparison does not show that it is necessary. On the matched 128 episodes, the primary student achieves 42.97% SR and stratified random delegation achieves 42.19%. The difference is +0.78 points, with a 97.5% paired interval of [-9.38, 10.94] (Table 4). The random control matches supervision counts, source roles, candidate actions, and semantic action-change status. This makes it a comparison of selection rules, but the interval is too wide to establish either superiority or equivalence.

The margin rule selects proposals offline according to empirical recovery and retention criteria (Section 3.2). Those checks do not by themselves prove a closed-loop SR advantage. We also cannot conclude from this comparison that any balanced mix of corrective turns would work equally well.

  1. Why use KL rather than hard-label CE?

The motivation in Section 3.3 is to match the reference distribution rather than keep sharpening a single selected target. Specifically, the KL approximation compares distributions renormalized on the reference's top-128 token support. This explains the objective, but is not evidence that KL gives better navigation performance or lower computational cost.

On the 9B cohort, KL minus hard preservation gives an SR difference of +5.47 points, with a 97.5% interval of [-3.91, 14.84], and an NE difference of -0.97 m [-1.91, -0.05] (Section 5.4). The evidence favors KL for NE in that comparison, not for SR. On the two matched StreamVLN seeds, the KL-minus-CE SR differences are -1.09 [-3.15, 0.92] and -0.22 [-2.18, 1.69] points, with 97.5% intervals (Section 5.6). We therefore do not claim that KL is generally a better preservation loss than CE.

  1. Sensitivity to the interpolation coefficient

As described in Section 3.4, alpha=0.5 was selected during development, fixed before the reported unseen evaluations, and reused across seeds. Those results show performance at that coefficient; they do not answer how closed-loop performance changes across the requested values {0, 0.25, 0.5, 0.75, 1.0}.

  1. Whether the calibration transfers across backbones

The retention constraints and development checks in Sections 3.2-3.4 test behavior on fixed states. They do not guarantee behavior on future trajectories or show that the same thresholds can be used unchanged on another backbone. We can therefore claim performance under the specified calibration settings, not robustness to changes in those settings.

  1. Why the NaVIDA gain is smaller

The anchored NaVIDA variant gains +0.65 SR points, with a 95% interval of [-1.36, 2.61] (Section 5.6). Since this interval includes zero, the experiment does not establish a gain. NaVIDA and the 9B model also use different native action interfaces and training recipes, so the comparison cannot isolate why their gains differ. An explanation based on saturation or another mechanism would be speculative.

The 9B full-split gain is +9.68 SR points, with a paired 95% interval of [7.40, 11.96], for one fixed primary training seed (Table 1). That result supports the procedure in the 9B setting, but does not imply a similar gain on NaVIDA.

Thanks for the rebuttal. I was expecting some numerical evidence, especially for Q1-4. Based on the current rebuttal. I choose to stay with my current rating.

Reviewer_AGHXOpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes RCSR-D, a selective distillation framework that extracts corrective supervision from a weak recovery policy to improve vision-and-language navigation. It combines margin-based action selection, CE/KL-based behavioral retention, and parameter interpolation to refine an existing navigation policy. Experiments on R2R-CE and supplementary evaluations with StreamVLN and NaVIDA demonstrate improvements over the corresponding baseline policies.

Critical strengths
  • The paper addresses a relevant problem by exploring how a weak recovery policy can provide useful corrective supervision.
  • The overall method is straightforward to implement.
Critical weaknesses
  • The technical novelty is limited, as the method primarily combines existing techniques.
  • Ablations do not establish clear benefits from the key components, while comparisons with existing methods, cross-dataset validation, and reproducibility details remain insufficient.
Questions for authors
  1. RCSR-D primarily combines logit-margin-based action selection, CE/KL-based distillation, and parameter interpolation, all of which have precedents in prior work. Its main novelty appears to lie in combining these techniques to extract local corrective supervision from a poorly performing Recovery Policy, rather than introducing a substantially new learning mechanism. The paper does not establish a clear technical advantage over existing selective distillation and navigation correction methods, making its contribution appear closer to an application of existing techniques than a sufficiently novel methodological advance.

  2. The evaluation primarily compares RCSR-D against the authors' own Natural checkpoint. Although SR improves from 37.25% to 46.93%, this does not establish its competitiveness with existing navigation methods. In particular, recent lightweight approaches can achieve competitive performance without relying on a 9B-parameter model and additional full-parameter training. Why is RCSR-D preferable to these alternatives, and what advantages does it offer beyond improving the authors' own baseline? Furthermore, the paper does not sufficiently report the overall training time and computational resources required by the full pipeline, making it difficult to assess its practical cost-effectiveness and reproducibility.

  3. Margin-based selection is a key component of RCSR-D, yet Table 4 shows that the full method improves SR by only 0.78 percentage points over matched random delegation, with a confidence interval spanning zero. Thus, the proposed selection rule has not demonstrated a clear advantage over a simpler random alternative. The independent benefit of KL-based retention over hard-label CE is also insufficiently established. Given these results, it remains unclear whether the overall improvement can be attributed to the proposed mechanisms rather than more general benefits of selective supervision and fine-tuning.

  4. The generalization evidence remains limited. Although supplementary experiments are conducted on StreamVLN and NaVIDA, the different training recipes make it difficult to isolate the contribution of RCSR-D's core mechanisms, and some variants show no statistically clear improvement. The main evaluation is also restricted to R2R-CE, without validation on other continuous navigation benchmarks such as RxR-CE. The current evidence is insufficient to establish RCSR-D as a general approach to extracting useful knowledge from weak policies; its effectiveness may remain specific to particular models and training settings.

  5. The key results rely on the specific Natural and Recovery checkpoints, yet the paper does not provide sufficient details to independently reconstruct these models. In particular, the training procedures and data configurations used to obtain these checkpoints are essential for reproducing the proposed method. What additional implementation details or resources can the authors provide to enable independent verification of the reported results?

2 comments

Thank you for raising these questions about novelty, comparisons, and reproducibility. We address them below, with the comparison and cost questions discussed separately.

  1. Contribution beyond the individual components

We agree that margin selection, CE/KL distillation, and parameter interpolation are established techniques. The contribution is how we use them to extract useful supervision from a recovery checkpoint that performs poorly as a navigator. We select its proposals offline, train a student initialized from the original navigator, retain reference behavior on other states, and deploy only the student (Sections 3.1-3.4).

On the same 128 episodes, Recovery achieves 6.25% SR and Natural achieves 33.59%, while the three students reach 42.97%, 46.88%, and 51.56% (Figure 1). On the fixed 512-episode subset, the student also outperforms direct recovery-weight interpolation at the same coefficient: 48.24% versus 28.91% SR (Table 3). These comparisons show why the checkpoint can be useful as a source of local supervision despite being a poor navigator. They are not comparisons against existing selective-distillation or navigation-correction methods.

  1. Comparison with lightweight navigation methods

On the full 1,839-episode split, SR improves from 37.25% to 46.93% and SPL from 29.36% to 36.28% relative to the fixed Natural checkpoint (Table 1). This measures improvement to that navigator. It does not establish a better accuracy-cost trade-off than lightweight systems, since the reported comparison does not match their training or deployment budgets.

  1. Training and inference cost

The main recipe uses 4,224 states, 48 updates, a global batch size of 128, and full-parameter training (Section 4). At deployment, only the student runs; the recovery policy is not needed. This does not mean the total inference workload is unchanged. Section 6 records 251,158 Natural forwards and 270,497 student forwards, with mean recorded episode times of 2,358.6 and 2,542.4 seconds under parallel evaluation. These figures describe that evaluation workload, not isolated model latency or the full pipeline's training cost.

  1. Attributing the gain to margin selection or KL

Table 4 gives a margin-minus-random SR difference of +0.78 points, with a 97.5% interval of [-9.38, 10.94]. The KL-minus-hard-preservation difference is +5.47 points, with a 97.5% interval of [-3.91, 14.84]. Neither establishes an SR advantage. KL does reduce NE in this comparison: the difference is -0.97 m, with an interval of [-1.91, -0.05] (Section 5.4). We therefore cannot attribute the overall SR gain independently to margin selection or KL.

  1. Generalization to other backbones and datasets

The StreamVLN and NaVIDA studies use different native interfaces and training recipes, rather than repeating the complete 9B pipeline under controlled conditions. NaVIDA's SR difference is +0.65 points, with a 95% interval of [-1.36, 2.61] (Section 5.6), so a gain is not established there. The full-split main result also uses one fixed training seed; its episode-bootstrap interval does not measure training-seed variability. These results do not establish transfer to RxR-CE or generality beyond the tested model and recipe settings.

  1. Reproducing the Natural and Recovery checkpoints

Sections 3-4 and Appendix A describe the selection rule, first-action masks, top-128 KL approximation, training configuration, evaluation protocol, and seeds. We agree that these details alone do not replace access to the locally navigation-tuned Natural initialization and Recovery specialist. The reproducibility statement identifies their anonymous release as a remaining requirement for independent reproduction. The procedural description should therefore not be read as sufficient to reconstruct the complete system from the paper alone.

Thank you for the response. I appreciate the authors' clarification of the results and their limitations.

My main concerns remain. While the improvement over the Natural checkpoint is clear, the contribution of the proposed components is still unclear. Margin-based selection performs similarly to random delegation, and KL preservation shows no consistent SR advantage over hard-label CE. I also agree with reviewer eN5P that a repair-only baseline would help clarify whether preservation is necessary.

Another concern is how RCSR-D compares with existing navigation approaches. I understand that achieving SOTA performance may not be the goal, but improving an internal baseline alone does not fully establish the value of the method. Given the use of a 9B model and additional full-parameter training, I would like to understand what RCSR-D offers over simpler or more lightweight alternatives, and at what computational cost. The paper provides some training configurations, but the total cost of the pipeline remains unclear.

I would appreciate further clarification on these points.

Reviewer_eN5POpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes RCSR-D, which extracts local action proposals from a weak recovery policy for navigation improvement. It combines margin-based proposal selection, CE distillation, KL-based reference preservation, and parameter interpolation. On R2R-CE validation-unseen, the method improves SR from 37.25% to 46.93% and SPL from 29.36% to 36.28%. Additional experiments examine component contributions and transfer to other navigation policies.

Critical strengths

1. Interesting finding with practical relevance. The paper shows that a recovery policy with only 6.25% SR can still provide useful local supervision. This is an interesting observation, suggesting that poor trajectory-level performance does not necessarily imply limited value as a source of training supervision.

2. Meaningful navigation improvements. The full 1,839-episode evaluation shows substantial gains with paired confidence intervals. The authors also report multiple training seeds on a smaller subset and provide detailed component comparisons, including results that do not show statistically significant differences.

Critical weaknesses

1. Limited evidence for the effectiveness of key components. In Table 4, margin-based selection improves SR by only 0.78 points over random delegation, while KL preservation shows no significant SR advantage over hard CE. The StreamVLN results also do not establish a clear benefit of KL, leaving the value of these mechanisms uncertain.

2. Limited attribution and generalization. The absence of a repair-only control makes it difficult to isolate the contribution of reference preservation. The main full-split result uses one training seed, and component comparisons cover only 128 episodes. The additional experiments use different training recipes, leaving the generality of the complete method unclear.

Questions for authors
  1. Given the similar performance of margin-based and random delegation, could the authors further clarify the benefits of the proposed selection rule? A larger-scale comparison on the full evaluation split would be helpful.

  2. Could the authors consider including a repair-only baseline without KL preservation and evaluating the complete RCSR-D procedure across additional seeds or backbones? Such results would help clarify the contributions of individual components and the generality of the method.

2 comments

Thank you for the detailed comments. We agree that the improvement from the full procedure needs to be distinguished from the contribution of each component.

  1. Evidence for margin selection and KL

On the matched 128 episodes, margin selection achieves 42.97% SR versus 42.19% for stratified random delegation. The difference is +0.78 percentage points, with a 97.5% paired interval of [-9.38, 10.94] (Table 4). This does not show an advantage for margin selection, but the wide interval does not establish equivalence either.

KL versus hard-label preservation gives an SR difference of +5.47 points, with a 97.5% interval of [-3.91, 14.84]. The NE difference is -0.97 m, with an interval of [-1.91, -0.05] (Section 5.4). Thus, this comparison supports lower navigation error, but not higher SR. The matched StreamVLN comparisons also do not show an SR advantage for KL (Section 5.6).

  1. Training seeds and cross-backbone results

The main 1,839-episode evaluation uses one fixed primary training seed, while the component comparisons cover 128 episodes. The full-split paired interval measures uncertainty across episodes, not variability across training seeds.

StreamVLN and NaVIDA use different native action interfaces and training recipes, so they are not controlled replications of the complete 9B procedure. NaVIDA's SR difference is +0.65 points, with a 95% interval of [-1.36, 2.61] (Section 5.6). We therefore cannot infer that the complete procedure works equally well across backbones.

  1. What margin selection adds

The rule in Section 3.2 selects proposals offline using empirical recovery and retention criteria. The random control matches supervision counts, source roles, candidate actions, and semantic action-change status. Table 4 therefore compares how supervision is selected, rather than how much supervision is provided. Its results leave the advantage of the margin rule unresolved.

On the full split, SR rises from 37.25% to 46.93%, a gain of +9.68 points with a paired 95% interval of [7.40, 11.96] (Table 1). This is evidence for the full procedure, not a substitute for the full-split margin-versus-random comparison you request.

  1. Repair-only training and additional seeds

The reported controls compare different preservation objectives; they do not compare training with preservation against repair-only training. They therefore cannot isolate the benefit of adding preservation.

On the shared 128 episodes, three student seeds achieve 42.97%, 46.88%, and 51.56% SR, compared with Natural's 33.59% (Figure 1; Table 2). These results show that the gain is not confined to one seed on that subset, but they do not provide full-split multi-seed validation. Together with the recipe differences discussed above, this limits our conclusion to extracting useful supervision from the tested weak recovery checkpoint.

Thanks for the rebuttal. I also read the other reviewers' comments. I appreciate the clarifications, but the benefit of margin-based selection remains unclear, and the repair-only control and full-split multi-seed evaluation are still missing. I tend to maintain my original rating.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.