acceptodds
Under review as a conference paper at ICLR 2027

Data-Centric Recursive Self-Improvement for Navigation via Policy-Synchronized Language-Action Dual Bootstrapping

Abstract

Vision-language navigation requires agents to follow natural-language instructions to reach target locations, making high-quality language supervision essential. Since human annotation is costly, prior work has explored scalable data synthesis methods. However, existing methods typically either use separate models for navigation and instruction generation or couple the two tasks through task-specific components; most also annotate the full trajectory corpus with a fixed model before updating, which can cause generated supervision to lag behind the evolving policy. Therefore, we introduce **RSI-Nav**, a data-centric recursive self-improvement framework for navigation via policy-synchronized language-action dual bootstrapping, which unifies vision-language-to-action instruction following and vision-action-to-language instruction generation within a shared model. Specifically, after supervised fine-tuning establishes fundamental capabilities, RSI-Nav progressively processes unlabeled trajectories in small shards. The current model generates instructions for these trajectories, executes them to evaluate goal completion and path efficiency, retains qualified samples, and immediately updates itself before processing the next shard. This shard-wise update keeps newly generated supervision aligned with the evolving policy. We also incorporate spatial understanding data, since stronger spatial reasoning benefits navigation. Extensive experiments show that RSI-Nav reaches 68.5% SR on R2R-CE and 72.2% SR on RxR-CE, while providing stronger instruction-generation utility and retaining strong spatial understanding after recursive evolution.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Vision-Language-Navigation, Data-Centric Recursive Self-Improvement

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_yNNNOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

RSI-Nav tries to reduce the cost of language supervision for continuous vision-language navigation. A single shared Qwen3.5-9B handles instruction following, instruction generation, and spatial understanding. After supervised initialization, it processes offline trajectories in shards: generate instructions, execute them, filter by success and SPL, then train in both directions on accepted instructions paired with the original trajectories, with spatial-data replay mixed in. The main claimed contribution is that supervision is synchronized frequently with the evolving policy. The paper reports 68.5% and 72.2% success on R2R-CE and RxR-CE validation-unseen, and generated instructions are more useful downstream. Spatial performance stays strong overall but drops on three of four benchmarks after evolution (Tables 1 and 4).

Critical strengths
  1. The navigation gains over the same initialization are substantial. Table 1 shows gains of 15.6 and 17.5 percentage points in success rate on R2R-CE and RxR-CE, with improvements in SPL and RxR-CE nDTW as well. The final success rates are 3.1 and 2.9 points above the strongest listed results. These numbers support the practical value of the overall training pipeline with single-view RGB observations.
  2. Generated instructions are evaluated by their downstream training utility, which is the right test. Table 4 trains identical navigation models on the same trajectories annotated by different generators. RSI-Nav supervision gives 41.3% SR and 35.1% SPL, versus 36.8% and 30.4% for SRDF. That directly asks whether the generated language helps navigation. The paper also appropriately qualifies NLP-score comparisons across different observation settings.
Critical weaknesses

The evidence does not yet establish the specific synchronization/co-evolution contribution. SRDF already iterates instruction generation, navigation-based filtering, and generator/navigator refinement; LANA already jointly learns following and generation with shared encoders. RSI-Nav’s clearest distinctions are complete parameter sharing and finer-grained updates between disjoint trajectory shards. Table 3 is useful: on the same 64K candidate trajectories, the preferred shard size gets 57.2% SR versus 55.1% for the full-subset offline setting. But supervision freshness and the training schedule change together, and the paper does not report accepted-sample counts or optimizer/replay exposure. So the proposed explanation is not cleanly separated from other effects of the update schedule. Also, Tables 4–5 do not compare the generator before and after recursive evolution. Better utility than SRDF does not show that RSI-Nav’s own supervision improves progressively. The system is effective overall, but the evidence for the specific advance over existing data-flywheel methods is still limited.

Questions for authors
  1. Does an evolving generator beat a frozen cold-start generator when using the same trajectory shards, current-policy filtering, and comparable optimization/replay budgets? What are the accepted-sample counts and training budgets for the settings in Table 3?
  2. Which checkpoint produces the instructions evaluated in Table 4? How do the cold-start and evolved generators compare on a fixed trajectory set, especially in downstream training utility?
  3. Please specify the recursive trajectory pool’s sources, size, and scene splits, the numerical value of $ \tau_{\mathrm{SPL}} $, and the per-shard optimization and spatial-replay schedule.
5 comments

Q1. Isolating the Contribution of Policy-Synchronized Instruction Generation

We sincerely thank the reviewer for the careful examination of our method and experiments. We appreciate your recognition of the navigation gains and the downstream utility of the generated instructions. Your observation about the interaction between supervision freshness and the training schedule is particularly helpful, and your suggestion of a frozen cold-start generator gave us a concrete way to investigate this question. We agree that the original Table 3 alone cannot fully distinguish these effects.

Training Schedule of Our Recursive Self-Improvement

Controlling the Training Budget. In Table 3, candidates/GPU denotes trajectories processed per generation and verification batch. Accepted VL2A/VA2L training samples accumulate across batches in a global queue. Each SFT update consumes 24,576 training samples plus 8,192 spatial-replay samples. Unused samples are retained. The 32,768-sample budget and optimizer configuration per update are fixed, while candidates needed to fill the queue depend on filtering outcomes. We will clarify this in the revised paper.

With 64 GPUs, a global batch size of 128, and one training epoch, each update corresponds to 256 optimizer steps. The same optimization configuration and replay allocation apply to Frozen and Evolving.

Frozen vs. Evolving Generator

Following the reviewer's suggestion, we compare Frozen and Evolving under the same 256-candidate-per-GPU setting as Table 3. Both use the same 64K trajectory pool, cold-start initialization, filtering criteria (SR = 1 and SPL ≥ 0.75), and queue-triggered SFT mechanism.

Navigation execution and filtering always use the current policy. The only difference is instruction generation. Frozen retains the cold-start generator throughout the experiment. Evolving uses the latest updated model to generate instructions for subsequent candidates.

SettingNE ↓OSR ↑SR ↑SPL ↑
Cold Start5.5466.452.945.5
256 Evolving (Ours)4.9168.757.248.7
256 Frozen4.9968.355.646.7
1024 (Offline)5.0368.155.146.3

Under the same 256-candidate configuration, Evolving outperforms Frozen by 1.6 SR points and 2.0 SPL points. Frozen provides only modest improvements over the 1024-candidate offline setting.

This comparison suggests that more frequent updates alone do not fully explain the improvement in Table 3. Refreshing instruction generation with the evolving policy provides an additional benefit, supporting the policy-synchronized bootstrapping mechanism described in Sec. 3.2.

Accepted-Sample Statistics

To address the reviewer's question about accepted-sample counts, we also recover the acceptance statistics for the original Table 3 settings. Each setting processes 65,536 candidate trajectories.

Candidates/GPUAccepted Trajectories (SR = 1, SPL ≥ 0.75)Acceptance Rate
6432,63649.80%
25635,36353.96%
102433,75151.50%

These settings yield relatively similar acceptance rates, suggesting that the observed performance differences cannot be simply explained by modest differences in the amount of accepted supervision.

We are grateful for the specificity of the reviewer's questions, which helped us present the comparison and training procedure more clearly. We will incorporate the frozen-generator comparison and the corresponding acceptance and training-budget statistics into the revised manuscript. We hope these results and clarifications address the concern about the contribution of policy synchronization.

Q2. Instruction-Generation Checkpoints and Downstream Training Utility

We appreciate the reviewer’s close reading of Tables 4–5 and the question about which checkpoint produced the evaluated instructions. We also thank you for recognizing the value of evaluating generated language through downstream navigation training. Your request to compare our own generator before and after evolution helped us examine the progression of supervision quality more directly.

Clarification of Table 4

The instructions evaluated in Table 4 are generated by the same checkpoint used in the third row of Table 2 and the last row of Table 5. This checkpoint is obtained by fine-tuning Qwen3.5-9B jointly on VL2A and VA2L data, without spatial-understanding supervision or recursive evolution.

Controlled Comparison Across Checkpoints

Following the reviewer’s suggestion, we compare four saved generators—Cold Start, 64K Evolving, 512K Evolving, and Final—under consistent evaluation protocols.

For instruction-generation quality, NLP metrics are evaluated on the same trajectory set as Table 4.

To assess downstream training utility, each generator independently annotates the same randomly sampled 512K unlabeled trajectories, followed by execution-based filtering with identical criteria (SR = 1 and SPL ≥ 0.75). Each resulting dataset contains sufficient accepted samples to construct a matched training set of 512K navigation samples. We then fine-tune Qwen3.5-9B from the same initialization, using identical data volumes and optimization configurations.

GeneratorBLEU-4 ↑CIDEr ↑SPICE ↑Downstream SR ↑Downstream SPL ↑
Cold Start28.450.220.536.830.5
64K Evolving28.650.121.039.232.9
512K Evolving29.350.821.941.535.2
Final29.948.922.843.837.6

Evolution of Supervision Quality

The results show that conventional NLP metrics improve modestly and are not strictly monotonic. These metrics primarily measure similarity to human instructions and do not necessarily reflect their effectiveness as navigation supervision.

Downstream training utility improves consistently across all four checkpoints under the same trajectory pool, filtering criterion, training-data volume, and navigator initialization. Downstream SR increases from 36.8% to 43.8%, and SPL from 30.5% to 37.6%.

These results demonstrate that instructions generated by successive RSI-Nav checkpoints become increasingly useful for downstream navigation learning, providing additional evidence for the language–action co-evolution proposed in Sec. 3.2.

The reviewer’s question was helpful in directing our attention to this within-model comparison. Thank you for identifying where the original tables left the progression of supervision quality unclear. We will include these results in the revised manuscript.

Q3. Trajectory Pool, Filtering Threshold, and Training Schedule

Thank you for identifying the implementation details that needed a clearer explanation. We appreciate the care with which the reviewer examined the trajectory sources, filtering rule, and training schedule. These questions helped us recognize where the manuscript relied on details that should have been stated explicitly. We provide the requested information below.

Trajectory Pool and Scene Splits

Our recursive evolution uses approximately 2.37M unlabeled continuous shortest-path trajectories derived from HM3D training scenes.

Instruction-generation NLP evaluation and R2R-CE Val-Unseen navigation evaluation use MP3D unseen scenes, with no scene overlap between recursive training and evaluation. The recursive trajectory pool is separate from the multi-source supervised data used for cold-start initialization, described in Sec. 3.3.

SPL Filtering Threshold

We use a fixed τ_SPL = 0.75 throughout recursive evolution. Accepted samples satisfy SR = 1 and SPL ≥ 0.75.

The numerical threshold was omitted from Eq. (4) in the original manuscript. We also provide a controlled sensitivity comparison of 0.1, 0.75, and 1.0 in our response to reviewer ywxj.

Optimization and Spatial Replay

As explained in our response to Q1, each candidate batch processes 256 trajectories per GPU, and accepted VL2A/VA2L training samples accumulate across candidate batches in a persistent global queue.

Once the queue contains at least 24,576 training samples, we take 24,576 samples in total across VL2A and VA2L and combine them with 8,192 spatial-replay samples for one SFT update. Unused samples remain in the queue for subsequent updates.

Each SFT update uses:

  • 32,768 training samples in total.
  • One training epoch.
  • A global batch size of 128.
  • A constant learning rate.
  • 256 optimizer steps.
  • 8,192 spatial-replay samples drawn using a fixed seed.

The updated checkpoint is then used for subsequent instruction generation and execution-based verification.

We appreciate the reviewer’s effort to examine the details behind the reported results. Your questions helped us make the connection between the trajectory pool, acceptance rule, and actual training schedule more explicit. We will clarify these details in Secs. 3.2–3.3 of the revised manuscript.

Reviewer_ywxjOpenReview ↗
Rating
4
Confidence
2
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper presents RSI-Nav, a data-centric recursive self-improvement framework for vision-language navigation. The proposed approach unifies instruction following, instruction generation, and spatial understanding within a single model. Through policy-synchronized language-action dual bootstrapping, the model generates navigation instructions, evaluates them through execution-based filtering, and progressively improves itself using the selected training data.

The experimental results demonstrate strong performance on R2R-CE and RxR-CE, achieving 68.5% and 72.2% success rates, respectively. The paper also provides extensive ablation studies and real-world demonstrations to validate the effectiveness of the proposed framework.

Critical strengths

Well-motivated framework. The paper addresses an important challenge in vision-language navigation: how to efficiently leverage unlabeled trajectories for improving navigation policies. The proposed recursive self-improvement framework is intuitive and practical.

Unified language-action learning. Integrating instruction following and instruction generation into a shared model is an appealing design. The policy-synchronized training strategy provides an effective way to leverage complementary supervision.

Strong experimental performance. The method achieves impressive results on both R2R-CE and RxR-CE, with substantial improvements over the supervised initialization. The ablation studies further support the effectiveness of the proposed components.

Practical relevance. The real-world navigation demonstrations and evaluations on spatial understanding benchmarks further strengthen the practical significance of this work.

Critical weaknesses

Filtering strategy: How sensitive is the performance to the SPL threshold used for filtering generated instructions? It would be helpful to provide the threshold value and a brief discussion.

Recursive improvement: How does the acceptance rate of generated instructions change across different training iterations? A simple analysis could provide additional insights into the self-improvement process.

Computational efficiency: Since the framework involves repeated instruction generation and navigation execution, could the authors briefly discuss the computational overhead compared with conventional offline data generation?

Questions for authors

Please follow the above questions.

3 comments

Q1. Sensitivity to the SPL Filtering Threshold

We sincerely thank the reviewer for the thoughtful review and the encouraging assessment of our framework, experiments, and real-world demonstrations. We particularly appreciate your suggestion to clarify the filtering threshold and examine its sensitivity. This is an important implementation detail, and your comment helped us make the choice and its effect on navigation performance more explicit.

Training Schedule of Our Recursive Self-Improvement

Controlling the Training Budget. In Table 3, candidates/GPU denotes trajectories processed per generation and verification batch. Accepted VL2A/VA2L training samples accumulate across batches in a global queue. Each SFT update consumes 24,576 training samples plus 8,192 spatial-replay samples. Unused samples are retained. The 32,768-sample budget and optimizer configuration per update are fixed, while candidates needed to fill the queue depend on filtering outcomes. We will clarify this in the revised paper.

Filtering Threshold and Sensitivity Analysis

The original paper did not state the numerical value in Eq. (4). Our full-scale recursive evolution uses a fixed τ_SPL = 0.75 throughout.

Following the 256-candidate-per-GPU setting of Table 3, we vary only the SPL filtering threshold while keeping the same 64K trajectory pool, cold-start initialization, optimizer settings, and spatial-replay scheme. The queue-triggered training mechanism described above keeps the data budget of each SFT update fixed.

SPL ThresholdNE ↓OSR ↑SR ↑SPL ↑
Cold Start (Baseline)5.5466.452.945.5
0.15.7962.349.943.5
0.75 (Ours; Table 3)4.9168.757.248.7
1.05.3465.553.546.1

The threshold balances the reliability and diversity of accepted instructions. A permissive threshold of 0.1 admits inefficient executions, yielding low-quality data where instructions may mismatch trajectories. This vision-language misalignment injects erroneous supervision, causing SR to drop below the baseline.

At the other extreme, 1.0 selects only near-optimal executions, potentially excluding more challenging trajectories that could support further learning. SR improves only to 53.5%.

The intermediate threshold of 0.75 achieves the best values across all four navigation metrics. These observations motivate the fixed threshold used in our experiments.

We are grateful to the reviewer for drawing our attention to this detail and suggesting the sensitivity analysis. We will report the threshold and these results in the revised manuscript.

Q2. Recursive Improvement: Acceptance Rate Across Iterations

We sincerely thank the reviewer for suggesting this analysis. We appreciate your interest in how the recursive improvement process develops over time. Your question encouraged us to examine the intermediate checkpoints and provide a clearer account of how the acceptance rate changes as the model evolves.

Evaluation Across Saved Checkpoints

Detailed per-update logs were not retained, so we conduct a retrospective evaluation using four saved checkpoints—Cold Start, 64K Evolving, 512K Evolving, and Final.

We evaluate each checkpoint on the same 2,048 randomly selected unlabeled shortest-path trajectories from HM3D training scenes and a fixed set of 320 randomly selected R2R-CE Val-Unseen reference trajectories from MP3D unseen scenes. Each checkpoint generates and executes its own instructions.

For a consistent comparison, the acceptance criterion is SR = 1 and SPL ≥ 0.75 for every checkpoint, matching our training threshold. We also report execution success on the unseen set without imposing an additional SPL criterion.

CheckpointHM3D SR (2,048) ↑[email protected] (2,048) ↑Val-Unseen Execution SR (320) ↑
Cold Start62.35%44.34%57.19%
64K Evolving66.16%49.61%61.25%
512K Evolving70.07%54.93%65.31%
Final RSI-Nav74.22%60.16%69.38%

Acceptance and Execution Success

The fixed-threshold acceptance rate increases from 44.34% to 60.16%, while successful execution on the same HM3D trajectories rises from 62.35% to 74.22%. On the held-out Val-Unseen subset, execution SR also improves from 57.19% to 69.38%.

These results show progressively better self-generated and self-executed instructions, including in unseen environments. Because each checkpoint uses its own navigator, these rates reflect the whole evolving system and the joint behavior of instruction generation and navigation.

We thank the reviewer again for this constructive suggestion. Examining the same trajectories across saved checkpoints makes the evolution of acceptance and execution success clearer, and we will include this checkpoint-level analysis in the revised manuscript.

Q3. Computational Overhead and Efficiency

We sincerely thank the reviewer for raising this practical question. We appreciate your recognition of the real-world relevance of our work and agree that the computational overhead of repeated instruction generation and execution-based verification deserves a clearer explanation. Your comment helped us make both the resource requirements and the implementation costs more explicit.

Computational Cost

All stages use 64 GPUs. The data volumes, wall-clock times, and allocated GPU-hours are summarized below.

StageData VolumeWall-Clock TimeAllocated GPU-hours
Cold-start SFT1.4M VL2A + 0.4M VA2L + 1.14M spatial~2.5 days~3,840
Recursive evolution2.5M VL2A + 1.1M VA2L + ~1.2M spatial replay~7 days~10,752
Total—~9.5 days~14,592

During recursive evolution, instruction generation, HM3D rendering, and closed-loop navigation verification account for more runtime than SFT itself. The multimodal nature of the tasks further increases computational demands. VA2L uses up to 32 trajectory images in a single turn. For VL2A, we provide up to eight history images at the start of the dialogue and one current observation image in each of eight turns, for a total of up to 16 image inputs across the dialogue.

Comparison with Previous Methods

Large-scale data generation and embodied VLM training are computationally demanding in related work as well.

  • SRDF [1] involves iterative instruction generation, navigation verification, and model refinement, with estimated computation on the order of thousands of GPU-hours, although exact total costs are not reported.
  • SID-VLN [2] reports 5,430 A800 GPU-hours across data generation, trajectory sampling, iterative training, and final pretraining.
  • NavFoM [3] reports 4,032 H100 GPU-hours for fine-tuning on approximately 12.7M navigation and QA samples.
  • RoboRefer [4] reports nearly one week on 80 A100 GPUs for 8B spatial-understanding SFT, corresponding to roughly 13K GPU-hours.

Compared with conventional offline annotation, RSI-Nav offers two methodological advances.

  1. A unified Qwen3.5-9B model performs instruction following, instruction generation, and spatial understanding with fully shared parameters.
  2. Policy-synchronized language–action dual bootstrapping allows updated policies to generate and verify subsequent supervision.

Beyond improving navigation performance, our framework produces approximately 2.5M VL2A and 1.1M VA2L training samples through execution-verified bootstrapping. The computational investment therefore yields both a multi-capability embodied model and a reusable large-scale supervision corpus.

Implementation Limitations and Optimization Opportunities

Our current implementation still has substantial room for acceleration. Compatibility constraints between Habitat and the Qwen3.5 inference environment have limited straightforward integration with inference acceleration frameworks such as vLLM/LMDeploy. HM3D rendering and sequential navigation execution also introduce significant overhead.

Furthermore, CPU/GPU utilization has not yet been fully saturated, leaving room for more concurrent rollouts per GPU. Similar inference-framework compatibility challenges have been discussed in RoboRefer [4].

We therefore regard the current runtime as an implementation-dependent cost. We thank the reviewer again for encouraging us to make these practical requirements clearer. We will clarify the computational requirements, methodological benefits, and potential optimizations in the revised manuscript.

References

[1] SRDF: Bootstrapping Language-Guided Navigation Learning with Self-Refining Data Flywheel.

[2] SID-VLN: Learning Goal-Oriented Language-Guided Navigation with Self-Improving Demonstrations at Scale.

[3] NavFoM: Embodied Navigation Foundation Model.

[4] RoboRefer: Towards Spatial Referring with Reasoning in Vision-Language Models for Robotics.

Reviewer_CTqdOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes RSI-Nav, a data-centric recursive self-improvement framework for vision-language navigation. It unifies instruction following, instruction generation, and spatial understanding within a single model. The proposed policy-synchronized dual bootstrapping framework progressively generates, verifies, and filters instructions from unlabeled trajectories, followed by shard-wise model updates. Experiments on R2R-CE, RxR-CE, and spatial understanding benchmarks demonstrate strong navigation performance.

Critical strengths
  1. The policy-synchronized dual bootstrapping framework provides a practical approach to leveraging unlabeled trajectory for iterative navigation policy improvement.
  2. The proposed method achieves strong results on R2R-CE and RxR-CE, with substantial improvements over the supervised initialization and competitive performance against recent VLN methods.
  3. The paper provides meaningful ablation studies on dual learning, action-conditioned instruction generation, and update granularity. The downstream training utility evaluation further supports the usefulness of the generated instructions.
Critical weaknesses
  1. The update-granularity ablation changes both the update frequency and the amount of data processed between updates. Without a fixed-generator baseline under matched training budgets, it is difficult to determine whether the improvements primarily come from policy synchronization or more frequent training updates.
  2. The filtering criterion only considers goal completion and path efficiency, while the original trajectory is retained for supervision. Consequently, an instruction may be accepted even when the executed route differs substantially from the original trajectory, potentially introducing noisy supervision.
  3. Although navigation improves significantly, the paper does not report how instruction-generation quality evolves across recursive updates. Moreover, Table 5 shows that joint training slightly decreases most instruction-generation metrics compared with VA2L-only training.
  4. RSI-Nav employs a 9B backbone, extensive multi-source supervision, and large-scale recursive training, whereas competing methods use different architectures and data budgets. Several improvements over the strongest baselines are also marginal, particularly for SPL and nDTW, without reported variance across runs.
Questions for authors
  1. Can the authors compare policy-synchronized bootstrapping with a fixed-generator approach under matched data volumes, filtering criteria, and optimization budgets? This would help isolate the contribution of the proposed synchronization mechanism.
  2. How do the authors ensure that an accepted instruction is consistent with the original trajectory rather than merely leading to the same destination? Have trajectory alignment or instruction-level semantic consistency been evaluated for the accepted samples?
  3. Can the authors report instruction-generation quality, data acceptance rates, and navigation performance across successive recursive updates? Such results would help verify whether the proposed framework improves both complementary capabilities over time.
  4. Can the authors clarify the specific advantages of their framework over SRDF and other iterative self-improvement approaches under comparable model and data settings?
4 comments

Q1. Controlled Comparison with a Frozen Generator

We sincerely thank the reviewer for this careful and constructive suggestion. We appreciate your recognition of the framework and the ablation studies, and agree that the fixed-generator comparison is important for understanding the contribution of policy synchronization. Your question helped us identify an aspect of the training schedule that was insufficiently explained and motivated the additional comparison below.

Training Schedule of Our Recursive Self-Improvement

Controlling the Training Budget. In Table 3, candidates/GPU denotes trajectories processed per generation and verification batch. Accepted VL2A/VA2L training samples accumulate across batches in a global queue. Each SFT update consumes 24,576 training samples plus 8,192 spatial-replay samples. Unused samples are retained. The 32,768-sample budget and optimizer configuration per update are fixed, while candidates needed to fill the queue depend on filtering outcomes. We will clarify this in the revised paper.

With 64 GPUs, a global batch size of 128, and one training epoch, each SFT update corresponds to 256 optimizer steps. The same optimization configuration and replay allocation apply to Frozen and Evolving.

Frozen vs. Evolving Generator

Following the reviewer's suggestion, we compare Frozen and Evolving under the same 256-candidate-per-GPU setting on the same 64K trajectory pool, with identical Qwen3.5-9B initialization, candidate ordering, filtering criteria (SR = 1 and SPL ≥ 0.75), and queue-triggered SFT configuration.

Both settings execute and filter generated instructions with the current navigator. The only intentionally changed component is instruction generation. Frozen retains the cold-start weights for annotation throughout the experiment. Evolving uses the latest updated shared model to generate instructions for subsequent candidates.

SettingNE ↓OSR ↑SR ↑SPL ↑
Cold Start5.5466.452.945.5
256 Evolving (Ours)4.9168.757.248.7
256 Frozen4.9968.355.646.7
1024 (Offline)5.0368.155.146.3

Under the matched 256-candidate setting, Evolving improves SR by 1.6 points and SPL by 2.0 points over Frozen, whereas Frozen only marginally outperforms the 1024-candidate offline setting. These results suggest that policy synchronization contributes substantially more to the observed gains than increased update frequency alone, highlighting the importance of keeping instruction generation aligned with the evolving policy.

We are grateful to the reviewer for recommending this comparison and drawing our attention to the need for a clearer description of the training budget. We will add this comparison to the revised manuscript.

Q2. Trajectory Consistency of Accepted Instructions

We sincerely thank the reviewer for raising this important question. We appreciate the careful observation that successful navigation to the intended destination does not by itself establish consistency with the original trajectory. Since our framework retains the source trajectory for supervision, this question directly concerns the quality of the accepted training pairs. We agree that goal success and SPL alone do not guarantee alignment with every step of the source route.

Route-Alignment Evaluation

Following the reviewer's suggestion, we evaluate the final checkpoint on 320 fixed R2R-CE Val-Unseen trajectories from MP3D unseen scenes. These are the same trajectories used in our checkpoint analysis. The model generates an instruction from each original trajectory and executes it from the corresponding start. We then measure nDTW between the executed route and the source trajectory.

MetricResult
Evaluated trajectories320
Successful executions (SR = 1)222 (69.38%)
Accepted executions (SR = 1 and SPL ≥ 0.75)180 (56.25%)
Mean nDTW of accepted executions (0–100) ↑90.85

The 180 accepted executions achieve a mean nDTW of 90.85, indicating that their executed routes are generally close to the source trajectories. The use of MP3D Val-Unseen scenes also provides evidence of route-level consistency beyond the HM3D environments used for recursive training.

Although high nDTW cannot verify every landmark description or spatial relation in the generated text, it directly addresses the route-alignment concern by comparing the executed and source trajectories.

We are grateful to the reviewer for drawing attention to this aspect of the filtering procedure. Your suggestion prompted us to examine route agreement explicitly, and we will include this evaluation in the revised manuscript.

Q3. Evolution of Instruction Generation and Navigation Performance

We sincerely thank the reviewer for this thoughtful suggestion. We appreciate your attention to how instruction generation and navigation change throughout recursive training. Examining intermediate checkpoints helps make the evidence for joint improvement clearer, and your question motivated us to provide the following checkpoint-level analysis.

Evaluation Across Recursive Checkpoints

As detailed per-update acceptance logs were not retained, we retrospectively evaluate four saved checkpoints—Cold Start, 64K Evolving, 512K Evolving, and Final—under the same evaluation protocol.

Navigation SR and SPL are measured on the full R2R-CE Val-Unseen split using its reference instructions. Instruction-generation metrics are evaluated on the same trajectory set across all four checkpoints.

Each checkpoint also generates and executes its own instructions on the same 2,048 randomly sampled HM3D training-scene trajectories, using the acceptance criterion SR = 1 and SPL ≥ 0.75. We additionally measure execution success on the 320 fixed R2R-CE Val-Unseen trajectories described in our response to Q2. For this unseen set, we report SR without imposing an additional SPL criterion.

CheckpointNav SR ↑Nav SPL ↑BLEU-4 ↑CIDEr ↑SPICE ↑HM3D [email protected] (%) ↑Unseen Execution SR (%) ↑
Cold Start52.945.528.450.220.544.3457.19
64K Evolving57.248.728.650.121.049.6161.25
512K Evolving62.255.529.350.821.954.9365.31
Final68.558.629.948.922.860.1669.38

Navigation, Acceptance, and Supervision Quality

Navigation SR increases from 52.9% to 68.5%, and SPL from 45.5% to 58.6%. NLP metrics change more modestly. The fixed-threshold acceptance rate increases from 44.34% to 60.16%, while execution success on the fixed unseen trajectories rises from 57.19% to 69.38%.

Because each checkpoint generates and follows its own instructions, these acceptance and execution rates reflect the joint evolution of language generation and navigation. Together with the separate generator-utility comparison in our Q2 reply to reviewer yNNN, these results provide complementary evidence for our dual-learning framework.

We thank the reviewer again for encouraging us to examine the intermediate behavior of the system. This analysis helps make the recursive improvement process more explicit, and we will include these checkpoint-level results in the revised manuscript.

Q4. Advantages over Existing Iterative Self-Improvement Methods

We sincerely thank the reviewer for asking us to clarify the contribution in relation to existing iterative self-improvement methods. We appreciate your recognition of the navigation results and downstream training-utility evaluation, as well as your careful attention to differences in model architectures and data budgets. Your question helps us explain the methodological distinctions and the context of the comparisons more clearly.

Methodological Differences

SRDF [1] iteratively refines a separate instruction generator and navigator. CorrectNav [2] focuses on learning self-correction from failed navigation trajectories. SID-VLN [3] improves navigation through self-generated exploration demonstrations.

RSI-Nav differs in two main aspects.

  1. Unified language–action learning. One Qwen3.5-9B model jointly performs instruction following (VL2A), instruction generation (VA2L), and spatial understanding with fully shared parameters.

  2. Policy-synchronized language–action dual bootstrapping. The updated model generates and verifies instructions for subsequent candidate trajectories, keeping newly generated supervision aligned with the evolving policy. The Frozen–Evolving comparison in our response to Q1 examines the benefit of refreshing instruction generation within this framework.

Beyond improving navigation performance, our framework produces approximately 2.5M VL2A and 1.1M VA2L training samples, yielding both a unified multi-capability model and a reusable execution-verified training corpus.

Data Scale and Supervision Settings

For context, we summarize the data scales and supervision settings of related approaches.

MethodData ScaleTrajectory SettingData Type
SRDF [1]20.42MDiscreteSynthetic navigation instructions
CorrectNav [2]2.1M+ContinuousInitial step-wise navigation samples
SID-VLN [3]46.53MDiscreteSelf-improving navigation demonstrations
NavFoM [4]12.7MMixed tasks8.02M navigation + 4.76M QA
RoboRefer [5]20MN/ASpatial-understanding QA
RSI-Nav (Ours)3.6MContinuous2.5M VL2A + 1.1M VA2L

SRDF [1] and SID-VLN [3] primarily construct supervision from discrete navigation trajectories. RSI-Nav generates execution-verified language–action supervision in continuous environments.

These datasets differ in sample definitions and supervision objectives, so their sizes are not directly comparable. In particular, CorrectNav's 2.1M samples belong to its initial DAgger training [2]. Our contribution lies in a framework that jointly improves language–action capabilities while generating verified supervision.

We are grateful to the reviewer for encouraging a more precise account of how RSI-Nav relates to previous methods. We will include these comparisons and clarifications in the revised manuscript.

References

[1] SRDF: Bootstrapping Language-Guided Navigation Learning with Self-Refining Data Flywheel.

[2] CorrectNav: Self-Correction Flywheel Empowers Vision-Language-Action Navigation Model.

[3] SID-VLN: Learning Goal-Oriented Language-Guided Navigation with Self-Improving Demonstrations at Scale.

[4] NavFoM: Embodied Navigation Foundation Model.

[5] RoboRefer: Towards Spatial Referring with Reasoning in Vision-Language Models for Robotics.

Comments on the paper

2 comments
AuthorsOpenReview ↗

Rebuttal Summary and Clarification of Contributions (1/2)

We sincerely thank Area Chairs and all three Reviewers for your careful evaluation and constructive feedback. As the discussion period concludes, we would like to briefly summarize the contributions of RSI-Nav, the additional evidence provided during rebuttal, and our response to the remaining novelty concern.

1. Contributions and Reviewer Feedback

RSI-Nav presents a data-centric recursive self-improvement framework with three closely related contributions:

  1. Policy Synchronization: Queue-triggered updates allow newly generated and execution-verified supervision to be incorporated into the evolving policy, ensuring that subsequent instruction generation benefits from the updated model.
  2. Language–Action Dual Bootstrapping: Instruction generation (VA2L) and navigation (VL2A) provide complementary supervision, enabling recursive improvement through closed-loop execution verification on unlabeled trajectories.
  3. Unified Embodied Capabilities: A single Qwen3.5-9B model performs navigation, instruction generation, and spatial understanding with fully shared parameters, without separate generators or task-specific modules.

Beyond these methodological contributions, RSI-Nav produces approximately 2.5M VL2A and 1.1M VA2L training samples, achieves strong performance on R2R-CE and RxR-CE, provides extensive ablation and spatial-understanding evaluations, and demonstrates practical capabilities through real-world robotic navigation.

We appreciate the reviewers' recognition of our strong navigation performance, meaningful ablations, unified learning framework, downstream instruction utility, and real-world relevance.

2. Additional Evidence from Rebuttal

We conducted additional experiments and provided detailed clarifications addressing the reviewers' questions:

  1. Evolving vs. Frozen Generator (CTqd Q1, yNNN Q1): Under matched 64K-trajectory settings, Evolving improves navigation by +1.6 SR / +2.0 SPL points, supporting the benefit of policy synchronization.
  2. Instruction-Generation Evolution and Utility (CTqd Q3, yNNN Q2): Across four checkpoints, downstream training utility improves from 36.8% to 43.8% SR and 30.5% to 37.6% SPL, with matched training settings.
  3. Trajectory Consistency (CTqd Q2): Execution-based route alignment evaluation yields 90.85 mean nDTW across 180 accepted trajectories in unseen environments, supporting consistency with the original trajectories.
  4. Acceptance Rates During Recursive Evolution (CTqd Q3, ywxj Q2): Retrospective evaluation across four checkpoints shows the fixed-threshold acceptance rate increasing from 44.34% to 60.16%, alongside improved execution success.
  5. SPL Filtering Sensitivity (ywxj Q1, yNNN Q3): Controlled experiments with thresholds 0.1, 0.75, and 1.0 support our fixed choice of 0.75.
  6. Data and Training Protocol (yNNN Q1/Q3, CTqd Q1): We clarified the 2.37M HM3D trajectory pool, scene splits, accepted-sample counts, queue-triggered updates, and fixed per-update optimization and spatial replay.
  7. Prior-Method Comparisons and Computational Cost (CTqd Q4, ywxj Q3): We compared prior methods' data scales and supervision settings, reported 9.5 allocated GPU-days, and discussed implementation overhead and optimization opportunities.

Together, these additions address the principal technical questions raised across all three reviews and strengthen the empirical evidence for RSI-Nav.

AuthorsOpenReview ↗

Rebuttal Summary and Clarification of Contributions (2/2)

3. Clarification of the Remaining Novelty Concern

We particularly appreciate Reviewer yNNN's follow-up, which recognizes our paper as "empirically solid", acknowledges that the new experiments "definitely strengthen the revision", and indicates that our clarifications resolved earlier questions about the experimental setup. Regarding the remaining novelty concern, we would like to emphasize three points:

  1. Significance relative to data scale. Policy synchronization achieves +1.6 SR / +2.0 SPL over Frozen using only 64K unlabeled trajectories. For context, cold-start dual learning uses 400K+ VA2L samples for +4.1 SR / +2.4 SPL, while spatial supervision uses 1.14M+ samples for +4.1 SR / +4.5 SPL. Although these are different-stage ablations, they highlight the significance of the synchronization gains at a relatively small data scale.
  2. Inherent relationship with update frequency. Policy synchronization requires newly verified supervision to update the model before subsequent data generation. Our queue-triggered training implements this mechanism; update cadence is therefore an integral part of synchronization rather than an independent design choice.
  3. Learning efficiency over conventional offline data flywheels. With a constant learning rate and fixed random seed, Frozen performs similarly to the 1024-candidate offline setting, consistent with their reliance on non-refreshing instruction generation. Under the same 64K-trajectory budget, Evolving achieves approximately 1.6× the SR gain and 2.7× the SPL gain of Frozen relative to Cold Start. This controlled comparison demonstrates that policy-synchronized bootstrapping makes more effective use of the same unlabeled data than the conventional fixed-annotator offline data-flywheel paradigm.

Our contribution is not the novelty of individual operations, but their integration into a unified, policy-synchronized language–action dual bootstrapping framework. Comparisons with prior methods, computational costs, and potential optimizations are further discussed in our responses to CTqd and ywxj.

We sincerely appreciate your time and consideration, and respectfully hope that the overall methodological contribution and comprehensive rebuttal evidence will be considered in the final assessment.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.