acceptodds
Under review as a conference paper at ICLR 2027

Credit-Guided Policy Improvement for Test-time Adaptive Vision-Language Navigation

Abstract

To advance the development of embodied general navigation, Test-time Adaptation for Vision-Language Navigation (TTA-VLN) has attracted increasing attention, aiming to adapt pretrained policies online to previously unseen environments using only test-time observations and interaction history. However, distribution shifts in unseen environments can distort the pretrained policy's local action preferences and lead to off-course decisions. Existing methods seek to correct such deviations using test-time signals, such as predictive uncertainty, trajectory-level feedback, or accumulated adaptation experience. Yet these signals do not directly establish whether an executed behavior actually contributes to instruction-guided progress toward the goal. Moreover, even when a test-time signal suggests a plausible corrective direction, the resulting policy change may still be unreliable and should not necessarily persist in subsequent decisions. The central challenge is therefore twofold: how to identify whether an interaction supports goal-directed improvement, and how to determine whether the resulting policy update is worth retaining. We observe that every executed action induces an immediate observation transition that exposes evidence of its local consequence. Based on this observation, we propose Credit-Guided Policy Improvement (CGPI), which uses action-induced observation transitions to recover signed, reference-relative decision credit without external outcome feedback. The recovered credit proposes a lightweight policy update, which is verified against prior credit-supported interactions and retained only when supported; otherwise, it is rolled back, while the pretrained navigation policy remains frozen. CGPI achieves consistent gains across the evaluated VLN benchmarks and navigation backbones, while qualitative robot trials further illustrate zero-shot sim-to-real feasibility.

Primary area:
applications to robotics, autonomy, planning
Keywords:
Vision-Language Navigation, Test-Time Adaptation

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_XZUMOpenReview ↗
Rating
3(+1)
Confidence
4
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes Credit-Guided Policy Improvement (CGPI), a feedback-free test-time adaptation framework for Vision-Language Navigation. It derives reference-relative decision credits from action-induced observation transitions to guide lightweight policy updates, which are selectively retained or rolled back through historical credit-based verification.

Critical strengths
  1. The identify–propose–verify–retain framework provides a clear and intuitive approach to test-time policy adaptation.
  2. The experiments cover multiple VLN benchmarks and navigation backbones.
Critical weaknesses
  1. The decision credit relies on fixed semantic-similarity measures and handcrafted evidence heuristics, which appear somewhat tailored to the evaluated benchmarks (e.g., goal-oriented REVERIE versus fine-grained instruction-following R2R/R2R-CE). Moreover, the paper lacks individual ablations of the four evidence components and quantitative real-world evaluations (e.g., SR) under more diverse scenes and instructions. These limitations raise concerns about the reliability of the proposed credit assignment and weaken the evidence for its generalizability beyond the evaluated benchmarks.

  2. The proposed Verification Mechanism relies on historical decision credits derived from the same heuristic evidence functions used for policy updates. Although the authors introduce a warm-up stage to alleviate memory sparsity, this does not ensure the reliability of the accumulated credits. Incorrect credits from early interactions may contaminate the Verification Memory, potentially reinforcing harmful updates or rejecting beneficial ones. Despite the robustness analysis under random credit noise, the paper does not investigate this potential self-reinforcing failure mode, raising concerns about the reliability of the Verify-and-Retain mechanism.

Questions for authors
  1. Could the authors provide individual ablations of the four evidence components (Goal, Progress, Grounding, and Risk), particularly removing Goal/Grounding on REVERIE and Progress on R2R/R2R-CE, to clarify their respective contributions?

  2. Could the authors provide quantitative real-world evaluations (e.g., SR) against the frozen backbone, particularly in scenarios involving more abstract or complex instructions and challenging instruction–observation semantic alignment, to better demonstrate generalizability?

  3. How robust is the Verification Mechanism when the initial memory contains systematically incorrect credits? Could the authors evaluate this scenario, analyze how frequently such errors occur on real benchmark trajectories, and investigate whether the resulting impact depends on the pretrained backbone's capability?

3 comments

Thank you for recognizing the clarity of the identify–propose–verify–retain framework. We added evidence-design controls, component ablations, paired robot trials and historical-credit audits. References below use the revised PDF.

W1. Evidence design and benchmark dependence

CGPI uses frozen learned representations with shared evidence, reference-relative credit and update-verification rules. The task-dependent Goal context is explicit: REVERIE uses the full goal description, while R2R/R2R-CE use the final instruction segment (App. D).

We directly test dependence on this choice by using the full instruction as Goal context across all benchmarks, including its STOP computations. REVERIE already uses this context. On R2R/DUET and R2R-CE/ETPNav, the shared-context variant reaches 77.6/66.5 and 59.8/50.9 SR/SPL, compared with frozen 72.0/60.0 and 57.0/49.0 (B.9, Table 19). Deduplicating the resulting Goal/Grounding signal while keeping denominator four gives 77.8/66.7 and 60.0/51.1. The table also includes denominator-three and fixed-STOP controls; normalization uses source data and other hyperparameters remain fixed. These controls retain adaptation gains without benchmark-dependent Goal context.

W2. Historical-credit contamination and self-reinforcement

We added natural-error audits, controlled initial-memory corruption and tracking after FIFO eviction. These measure both harmful-update acceptance and beneficial-update rejection, together with later memory errors and navigation outcomes (G.7, Table 32; Q3 below).

A complementary test applies persistent semantic distractor bias to both update evidence and verification-memory writes. At bias strength ρ=0.25, CGPI achieves 56.8 SR versus 55.4 for Direct Commit. At ρ=0.50, the results are 53.4 versus 53.8 (H.3, Table 38). The retention benefit persists at the moderate setting and is absent at the stronger setting.

Q1. Contributions of Goal, Progress, Grounding and Risk

Table 4 now contains individual removals across all three benchmarks. Component removal retains denominator four and fixes the complete STOP branch. On REVERIE/DUET, Goal and Grounding coincide by construction: removing either gives 58.1 SR, removing both 56.9, versus 58.76 for Full CGPI. The three-term deduplicated mean gives 58.8, distinguishing semantic removal from aggregation changes.

Removing Progress reduces R2R/DUET SR 78.9→76.8 and R2R-CE/ETPNav 61.2→59.0. Progress measures local semantic change over fixed instruction segments, without completion tracking. We also added reference-relative counterfactual diagnostics on these instruction-following benchmarks: Spearman 0.589/0.551, sign errors 20.4%/22.7% (B.13, Table 27).

Q2. Quantitative real-world evaluation

We compare the same R2R-CE-pretrained BEVBert with and without CGPI on Unitree Go2, sharing starts, candidates, controller and execution budgets. Four environments and 30 instructions × four repetitions give 120 attempts per method, in ten independently initialized paired blocks. All failures and interventions count.

CGPI improves SR 21.7%→30.8% (26/120→37/120, +9.2 points). Abstract/ambiguous tasks improve 8/40→12/40; ordered multi-stage tasks improve 4/40→8/40, with success requiring the specified stage order. H.1 reports all 94/83 failures by mutually exclusive cause (Tables 35–36). H.2 stratifies the same paired trials by visual conditions: reduced illumination 5/24→8/24, occlusion 4/24→6/24, and both 3/24→5/24 (Table 37).

Q3. Error frequency, initial corruption and backbone dependence

Across 20 REVERIE task orders, initial-memory movement sign errors against the reference-relative oracle, excluding neutral records, are 24.8% for HAMT and 15.2% for DUET. Subsequent insertions have error rates of 20.1%/12.1%. We then reverse 0/2/4 initially oracle-consistent credits, preserving magnitudes and other states. This paired experiment uses the 16 orders eligible for both backbones, with the original warm-up of eight and FIFO capacity of 32.

SR for 0/2/4 reversals is 36.9/35.9/34.7 for HAMT and 58.6/58.0/56.9 for DUET. Four reversals therefore cost 2.2 versus 1.7 SR points. For DUET, harmful-update acceptance rises 11.8%→31.2% and beneficial-update rejection 18.3%→35.7% (G.7, Table 32).

After the injected records are evicted, new-credit errors remain 25.7%/15.9%, versus unmodified runs' 20.1%/12.1%, with SR deficits of 1.6/1.0 points. An oracle-assisted diagnostic corrects subsequent memory-write signs in both unmodified and corrupted runs, leaving proposal credits unchanged; the SR gaps shrink to 0.5/0.3 points. This supports error propagation through later memory writes after the original contaminated records are evicted (G.7).

I thank the authors for the detailed rebuttal and revision. The new component ablations (Table 4), shared-context controls (Table 19), real-world evaluations, and memory-corruption analyses (Apps. G.7 and H.3) have adequately addressed my main concerns. I also appreciate the transparent reporting of negative results.

Some limitations remain regarding verification reliability under strong systematic credit errors and the limited scale of real-world evaluations. I encourage the authors to discuss these limitations more explicitly in the main text and Appendix K, and to report confidence intervals for the real-world results.

Overall, the original submission did not provide sufficient experimental evidence to fully support its claims. However, the substantial additional experiments have addressed my main concerns and significantly strengthened the paper. I therefore raise my rating accordingly.

AuthorsOpenReview ↗

Thank You for Your Constructive Feedback and Reassessment

Dear Reviewer XZUM,

We sincerely appreciate your careful reassessment of our revised manuscript and the time you have devoted to reviewing our additional experiments.

We are particularly grateful that you acknowledge the new evidence addressing your main concerns. Your detailed feedback has been invaluable in helping us strengthen the experimental validation and clarify the scope of our contributions.

We also appreciate your remaining suggestions regarding verification reliability under systematic credit errors and the statistical uncertainty of real-world results. We agree that these limitations deserve clearer discussion and will explicitly address them in the main text and Appendix K, together with confidence intervals for the real-world evaluation.

Thank you again for your constructive comments, thoughtful engagement, and willingness to reconsider your assessment in light of the new evidence.

Best regards,
The Authors

Reviewer_SrteOpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

This paper proposes Credit-Guided Policy Improvement (CGPI), a test-time adaptation framework for Vision-Language Navigation (VLN). The method leverages action-induced observation transitions to estimate decision credit and introduces a Verify-and-Retain mechanism to selectively preserve beneficial policy updates. Experiments across three VLN benchmarks and four navigation backbones demonstrate consistent performance improvements.

Critical strengths

Clear Motivation. The paper clearly distinguishes between policy change and policy improvement, addressing an important challenge in online adaptation: how to identify beneficial updates and reliably retain them.

Effective and Practical Framework. CGPI combines reference-relative credit assignment with a Verify-and-Retain mechanism. The framework operates without external task-level feedback and only updates a lightweight action reranker, making it practical and computationally efficient.

Comprehensive Experiments. The authors evaluate the proposed method across multiple benchmarks and navigation backbones. The extensive ablation studies, credit quality analysis, and counterfactual evaluations provide convincing evidence of the method's effectiveness.

Critical weaknesses
  1. Credit Validity: Can the authors directly evaluate the correlation between the proposed decision credit and an oracle reference-relative advantage computed from simulator counterfactuals, rather than absolute one-step geodesic progress?

  2. Verification Effectiveness: How does the proposed verifier compare against random retention, KL-only gating, or replay-based selection when all methods are matched to the same update acceptance rate and computational budget?

  3. Generalization and Robustness: How robust is CGPI to systematic errors in its semantic evidence functions? Can the authors provide quantitative real-world evaluations or results under stronger visual and instruction distribution shifts?

Questions for authors

Please follow the above question.

1 comment

Thank you for recognizing the motivation and practicality of CGPI. We added direct counterfactual credit evaluation, controlled verifier comparisons, systematic-bias tests, and paired robot experiments. All references below point to the revised PDF.

1. Credit validity against a reference-relative oracle

For each movement decision, we restore the same simulator and agent state and execute every movement alternative. The one-step oracle is the executed action's geodesic progress minus the frozen policy's expected progress over those alternatives. This matches the requested reference-relative comparison. All credit variants use the same transitions before clipping or update filtering; STOP is excluded. Sign error uses oracle-nonneutral records; correlations retain neutral cases. Simulator oracles are used for diagnostics.

On REVERIE/DUET, replacing pre-action hypotheses with observed consequences improves Spearman correlation 0.503→0.626 and reduces credit-sign error 24.0%→18.5%. Full CGPI's Pearson correlation is 0.648. Grounded evidence without reference subtraction gives Spearman 0.394 and sign error 28.7%, supporting both grounding and reference subtraction (G.7, Table 31). HAMT also improves: Spearman 0.421→0.542, sign error 29.1%→24.2%. The added R2R/R2R-CE diagnostics yield Spearman 0.589/0.551 and sign errors 20.4%/22.7% (B.13, Table 27).

2. Verification at matched retention rates and controlled budgets

Fixed candidates. All selectors evaluate the same KL-feasible candidate bank and memories, retaining exactly 25%, 50%, or 75%. At 50%, AIR is 54.3% for Random, 57.1% for KL-only, 61.8% for Unweighted Memory Scoring, 72.1% for Sign-only Memory Scoring, and 76.2% for CGPI. AIG is 0.64 m for CGPI, versus 0.12 m for Random and 0.36 m for unweighted scoring (G.7, Table 33).

The common additional budget cap is 64 reranker state evaluations per candidate, with no extra backbone passes. The three memory scorers each use all 64 evaluations; Random and KL-only reuse proposal computations. Thus, the memory-scorer comparison isolates the value of credit weighting at equal evaluation counts.

Closed-loop navigation. Source-calibrated thresholds produce retention rates of 49.7%–50.3% and 80.9–82.0 accepted updates per 1,000 steps. Random/KL-only/Unweighted/CGPI achieve SR/SPL of 54.9/37.0, 55.8/37.8, 56.8/38.6, and 58.3/39.8, respectively (G.7, Table 34). CGPI exceeds unweighted scoring by 1.5 SR and 1.2 SPL points at similar verification latency: 1.23 versus 1.20 ms per proposal. This connects better candidate selection to navigation gains while controlling update frequency and memory-scoring cost.

Replay optimization. A separate control uses the same reranker, credits and memory, with historical losses entering gradients. At approximately 50% retention, Replay+Random achieves 57.1/38.8 SR/SPL versus 58.3/39.8 for CGPI (B.10, Table 21). Memory scoring and replay training are therefore evaluated as distinct uses of history.

3. Systematic errors and real-world generalization

Persistent semantic bias. H.3 applies a fixed distractor-instruction bias to both pre-action and post-action semantic evidence, affecting update proposals and subsequent verification-memory writes. The distractor is shared across time and semantic views. At bias strength ρ=0.25, CGPI achieves 56.8/38.3 SR/SPL, versus 55.4/37.1 for Direct Commit. At ρ=0.50, the results are 53.4/35.8 and 53.8/36.1, respectively (Table 38). The retention advantage persists at the moderate setting and is absent at the stronger setting. G.7 additionally measures initial-memory corruption and propagation after FIFO eviction (Table 32).

Paired physical-robot evaluation. Frozen R2R-CE-pretrained BEVBert and BEVBert+CGPI share task starts, candidates, controller and execution budgets on Unitree Go2. Four environments and 30 instructions repeated four times give 120 attempts per method, organized into ten separately initialized paired blocks. Success rises 26/120→37/120 (21.7%→30.8%, +9.2 points); all failures and interventions are included (H.1, Tables 35–36).

Abstract/ambiguous instructions improve 8/40→12/40, and ordered multi-stage tasks 4/40→8/40, requiring completion of the specified stages in order. The same paired trials are stratified by visual conditions: reduced illumination improves 5/24→8/24, occlusion 4/24→6/24, and their combination 3/24→5/24 (H.2, Table 37). These directly quantify adaptation benefits under more difficult instruction and visual conditions.

Reviewer_v5JhOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes CGPI, a test-time adaptation method for VLN (vision language navigation) that needs no external feedback. The pretrained navigator stays frozen, and only a small residual "reranker" on its candidate features is trained online. After each move, CGPI scores the actual next observation with a few fixed similarity measures in the backbone's feature space, subtracts the same scores predicted from the candidates, weighted by the frozen policy, and treats the difference as a signed credit. A large enough credit triggers one optimizer step on a credit-weighted log-likelihood with KL and L2 terms. The step is kept only if it also raises the credit-weighted log-likelihood on a memory of earlier interactions and keeps the KL to the frozen policy small; otherwise the parameters and optimizer state are rolled back. Experiments cover REVERIE, R2R and R2R-CE with several VLN backbones. Results show performance gainsand ablations, post-hoc diagnostics and qualitative robot runs on a quadruped.

Critical strengths
  1. The method respects the test-time setting and is cheap to "bolt on" to a wide range of existing methods. The credit uses only the instruction, the observations seen so far in the episode and frozen features; normalization statistics come from the training split, and geodesic distance shows up only in post-hoc diagnostics. I found no privileged information used during adaptation. The reranker starts at zero with a bounded correction, so adaptation begins exactly at the pretrained policy, and a rejected step also restores the optimizer state. The extra cost on DUET is small, so it is easy to try on top of other navigators.

  2. I like the keep-or-rollback rule: each self-generated update gets checked against earlier evidence and undone if the evidence doesn't support it. The gains over the strongest feedback-free baseline are small but consistent. CGPI beats IDEA on every reported metric across six benchmark and backbone pairs, and in the one setting with run-to-run variation, five shuffled episode orders on REVERIE Val Unseen, the gap holds up.

Critical weaknesses
  1. The contribution is incremental and the measured gains are thin. The credit, a frozen-feature similarity score of the next observation minus a policy-weighted prediction, is close to RLCF (ICLR 2024, cited) and to VLM-RMs (ICLR 2024, not cited). The keep-or-rollback rule is essentially TRPO's accept test run on held-out past samples, much as GEM (NeurIPS 2017) and high-confidence policy improvement (ICML 2015) check updates against stored or held-out samples; none of these three is cited. Within VLN, FSTTA already reverts to historical states, and EAM (IEEE TMM 2025, not cited) keeps the VLN model frozen and adapts an auxiliary decision model online with replay. In the paper's own ablation, grounding on the next observation adds only a couple of SR points. Over IDEA, the margins run from +0.3 to +3.0 points, run-to-run spread is reported for one setting only, and R2R is already at human level, so gains of this size are hard to interpret. Two results in Table 7 are also hard to square with the method's own settings (Q1, Q2).

  2. It isn't clear what CGPI adds on top of strong, current navigators, and the real-world evidence is anecdotal. All four backbones are 2021 to 2024 specialist models, and no VLM-based navigator appears as a backbone or baseline. Even with CGPI, the results sit at or below current numbers on the same benchmarks, for example SRDF (ICLR 2025) on REVERIE and OmniNav (ICLR 2026), an RGB-only VLM navigator the paper itself cites, on R2R-CE. VLM navigators trained at scale, such as NaVILA (RSS 2025), are what people put on robots today, and I expect they would handle the short landmark-following tasks in the robot figures well. The robot evidence is six successful runs, which the paper itself calls qualitative, with no trial count, success rate, failure case or comparison to the frozen policy, and the deployed policy is not named. So it is left open whether CGPI still helps once the base policy is strong, or only for small, older backbones. Furthermore, it would be important to draw comparisons to "frontier" VLMs that are increasingly becoming capable of solving language navigation tasks "zero shot".

Questions for authors
  1. (Clarification) In the episode-reset run of Table 7, the reranker, optimizer and memory are reset before every episode, and a candidate can only be proposed once 8 eligible interactions from that same episode are in memory, so an episode needs nine credit-eligible decisions before its first update, while HAMT's REVERIE trajectories average only about six moves. How, then, does episode reset still improve SR, and was N_min changed for this run? How many episodes got at least one retained update, and what was the largest change in action probabilities?
  2. (Clarification) In the left half of Table 7, phi_t and phi_t^+ differ by one AdamW step. What fraction of candidates give identical greedy rollouts, and how are those counted in AIR, AIG and RP? The reported values imply that most retained and most rejected candidates change the rollout after one small step, so a histogram of per-candidate changes in action probabilities would settle it.
  3. (Suggestion) Three controls would show whether the adaptation and the verifier matter: add the four evidence scores to the frozen logits with no learning at all; replace the verifier with random acceptance at the same rate; and swap it for plain replay or an A-GEM-style memory constraint (ICLR 2019) using the same reranker, credit and memory. If the first control recovers most of the gain, the contribution is the heuristic rather than test-time learning.
  4. (Suggestion) Does CGPI still help a strong or VLM-based base policy, for example SRDF or ScaleVLN on R2R and REVERIE, NaVILA or OmniNav on R2R-CE, or a zero-shot navigator that already chooses among candidates, such as MapGPT or Open-Nav? For policies that output chunks of low-level actions, how would you define the candidate-level credit and reranker? A short discussion of concurrent work such as TANGO (arXiv 2609.09158), which trains navigation policies entirely in simulation and deploys them zero-shot, would also help.
  5. (Clarification) Which policy ran on the Go2 in the robot figures, and how was CGPI instantiated for it? The overlays show up to four actions per step; how do these map to the candidate set and the reranker? How many runs per instruction did you make, and what were the success rates with and without CGPI, failures included?
  6. (Clarification) Every baseline row in Tables 1 to 3 matches IDEA's tables exactly. Did you re-run any baselines under your seeds and episode order? Please report the standard deviation over your three seeds, and R2R and R2R-CE results to one decimal.
  7. (Clarification) The grounded evidence uses the pooled next observation, while the reference uses candidate representations. The shared mean cancels in the credit, so any systematic gap between the two representation types shifts every credit. On the test stream, what are the mean credit and the fraction of positive credits for each view?
  8. (Clarification) The appendix says Goal and Grounding coincide by construction on REVERIE, and Progress takes the best match over a fixed set of instruction segments without tracking which ones are done. Could you add a per-view ablation, and report the credit diagnostics on R2R or R2R-CE, where instruction order matters?
5 comments

Thank you for the detailed review. References are to the revised PDF (original Table 7 is now Table 8). CGPI tackles a fundamental TTA-VLN challenge: policy change does not necessarily imply policy improvement. Without labels or outcome feedback, interaction-driven updates may reinforce errors. The key is not merely how to update, but how to identify beneficial changes and prevent harmful ones from persisting. CGPI addresses this through a lightweight, frozen-backbone identify–propose–verify–retain loop: observed consequences guide what to change, while past interactions determine what to retain .

W1. Novelty and limited gains.

App. A now distinguishes CGPI from RLCF/VLM-RMs, TRPO, GEM, HCPI, FSTTA and EAM. Beyond semantic scoring, CGPI evaluates actual next-observation evidence relative to frozen-policy-weighted pre-action alternatives. TRPO optimizes return under KL; HCPI tests return confidence; GEM constrains past gradients; EAM trains with replay; FSTTA restores historical states. CGPI tests each completed update against past signed credits, using forward passes, and restores model/optimizer states on rejection.

Grounding credit in actual outcomes raises REVERIE/DUET SR 56.82→58.76 (Tab 5) and oracle-relative Spearman 0.503→0.626, reducing sign errors 24.0%→18.5% (G.7, Tab 31). Full Verify-and-Retain, including the KL gate, improves on direct commit: SR 57.41→58.76, AIR 0.548→0.741, AIG 0.19→0.63 m, RP 0.716 (Tables 7–8). Static scoring reaches 54.20 versus 58.76 SR (B.10, Table 20).

CGPI is applied directly to the original frozen HAMT, DUET, BEVBert and ETPNav—not on top of IDEA. Its gains over these backbones reach 4.02–11.78 SR points across six Val Unseen pairs. The 1.84–3.20-point margins over IDEA compare two adaptation methods.

W2. Stronger policies and robots.

Matched Frozen→CGPI R2R SR is 81.0→82.7 for ScaleVLN, 85.8→87.0 for SRDF (B.11, Table 22). A Qwen3-VL robot navigator improves 39/120→45/120 successes; GPT-5.5/Gemini 2.5 Pro provide zero-shot comparisons (H.4, Table 39). Paired Go2/BEVBert trials improve 26/120→37/120, including ambiguous/multi-stage tasks and failures (H.1).

Q1. Episode reset.

N_min remains 8; the ninth eligible interaction first permits a proposal. Despite ~6 mean moves, 18.6% reach nine eligible interactions, 15.2% retain updates, 13.8% retain a movement update before another decision, and 10.4% change greedy sequences (SR 30.40→32.62). Maximum probability deviations are 0.0036 per update and 0.0084 cumulatively from frozen (Sec. 3.7; F.1, Table 29).

Q2. One-step rollouts.

Identical five-step greedy rollouts occur for 22%/23% of retained/rejected candidates; zero gains remain in AIR/AIG/RP denominators. Median first-divergence action margins are 0.00062/0.00068. Table 30 reports signed gains; Fig. 5 shows probability-change histograms (F.1).

Q3. Controls.

We added static scoring, random/KL-only retention and replay. Replay uses the same reranker, credits and memory, with historical losses entering gradients. At ~50% retention and similar updates/1k steps, Random/KL-only/Replay+Random/CGPI yield SR/SPL 54.9/37.0, 55.8/37.8, 57.1/38.8, 58.3/39.8 (B.10, Table 21; G.7, Table 34). At 50% quota on a common candidate bank, Random/KL-only/CGPI AIR is 54.3/57.1/76.2% (G.7, Tables 33–34).

Q4. Strong/VLM/chunks.

Tables 22/39 report paired tests; H.4 specifies VLM interfaces. App. K defines matched-horizon chunk credit (observed minus frozen-expected evidence), residual reranking of chunk candidates, and boundary verification. App. A discusses TANGO.

Q5. Go2.

R2R-CE-pretrained BEVBert supplies waypoint/STOP candidates for CGPI reranking; a fixed controller executes the selected waypoint's commands. Arrows show execution order. Across four environments, 30 instructions × 4 repetitions give 120 attempts/method; all 94/83 failures, including interventions, are reported (H.1, Tables 35–36).

Q6. Provenance and variance.

Tables 1–3 quote IDEA's published baseline entries; these rows were not rerun. FSTTA and IDEA were separately rerun for the five-order evaluation (B.5, Table 15). B.12 reports CGPI three-seed mean±SD (Tables 23–25; DUET/REVERIE SR 58.76±0.25). CGPI R2R/R2R-CE results use one decimal.

Q7. Representation gap.

Table 26 (B.13) reports per-view means/SDs, positive rates and observed–candidate gaps. REVERIE/DUET Goal/Progress/Grounding/Risk means are 0.164/0.102/0.164/−0.028; positive rates are 59.8/56.4/59.8/46.2%. Table 27 evaluates source-offset correction.

Q8. Overlap and ordering.

Table 4 ablates all four components, including REVERIE's identical Goal/Grounding terms: joint removal gives 56.9 SR, full 58.76, deduplicated 58.8. Removing Progress reduces R2R/R2R-CE SR by 2.1/2.2. Progress uses fixed segments without completion tracking. B.13, Table 27 extends oracle-relative diagnostics to R2R/R2R-CE.

AuthorsOpenReview ↗

Follow-up on Critical Weakness 1: Novelty and the source of performance gains

CGPI addresses a core problem in feedback-free TTA-VLN: policy change does not imply improvement. In unseen environments, adaptation lacks labels or outcome feedback, while actions shape later observations. Existing uncertainty or semantic signals may prompt updates without showing whether an action helped or the update should persist; errors can compound. We therefore couple two decisions: observed consequences guide what to change, while past credits test what to retain. Our contribution is this evidence-grounded coupling beyond isolated similarity scoring or parameter rollback.

What differs from the cited methods? For an executed movement action, CGPI compares evidence from the actual next observation against the frozen navigator's probability-weighted pre-action candidate hypotheses over valid movement alternatives. The resulting signed credit is local and reference-relative, with STOP handled via a dedicated completion-versus-continuation formulation, requiring no simulator reward, target location, or external outcome label during adaptation. Whereas RLCF centers CLIP rewards over sampled outputs and VLM-RMs use pretrained vision-language models as reward functions, CGPI uniquely couples a frozen-navigator action reference with post-execution observation grounding and historical verification of completed updates.

The retention rule likewise serves a distinct informational and computational role compared to prior work. TRPO and HCPI evaluate return-based policy improvement; GEM constrains updates using historical gradients; EAM trains an auxiliary model using replayed pseudo-labeled samples; and FSTTA restores historical states. In contrast, CGPI performs a single current-interaction optimizer step and uses past (state, action, signed credit) tuples exclusively as a forward-only acceptance test over cached reranker representations. A candidate update is committed only if it improves historical credit agreement while satisfying the proposal-state KL constraint relative to the frozen policy; otherwise, both model and optimizer states are rolled back. We have incorporated all suggested citations and detailed these distinctions in Appendix A.

Which part of the gain comes from which mechanism? Our controlled ablations decompose the total improvement across each stage of the framework:

  • Improvement identification (Table 5; G.7, Table 31): Absolute Hypothesis / Grounded Evidence / Relative Hypothesis / Full CGPI achieve 54.32 / 56.14 / 56.82 / 58.76 SR on REVERIE Val Unseen with DUET. Grounding executed actions in their actual post-action observations adds +1.94 SR points on top of reference centering, increases correlation with the reference-relative counterfactual oracle from 0.503 to 0.626 (Spearman), and reduces credit-sign error from 24.0% to 18.5%. This directly isolates the signal-validity benefit of transition grounding.
  • Online adaptation versus static scoring (B.10, Table 20): Frozen DUET / static evidence reranking / Full CGPI achieve 46.98 / 54.20 / 58.76 SR. While static pre-action evidence scoring improves SR from 46.98 to 54.20 (+7.22 points), online credit-guided adaptation with verify-and-retain contributes an additional +4.56 SR and +3.55 SPL points.
  • Retention quality under matched acceptance and compute budgets (Tables 7–8; B.10, Table 21; G.7, Tables 33–34): Moving from Direct Commit to Full CGPI raises SR from 57.41 to 58.76 and Accepted Improvement Rate (AIR) from 0.548 to 0.741. To control for acceptance frequency, we further matched retention rates at ~50%: CGPI achieves 58.3/39.8 SR/SPL, outperforming random retention (54.9/37.0), unweighted memory scoring (56.8/38.6), and credit-weighted replay with random retention (57.1/38.8). On the same candidate bank at a fixed 50% quota, AIR progresses from 61.8% (unweighted memory) to 72.1% (sign-only memory) and 76.2% (CGPI), with all three selectors using the exact same budget of 64 cached reranker evaluations per candidate. This confirms that the gain stems from credit-weighted selection quality under matched acceptance rates and identical evaluation budgets.

Finally, CGPI is applied directly to the original frozen HAMT, DUET, BEVBert, and ETPNav backbones as a standalone module rather than stacked on IDEA. Across the six Val Unseen benchmark–backbone settings, CGPI improves SR by 4.02–11.78 percentage points over the frozen backbones and by 1.84–3.20 points over IDEA's published results (Sec. 3.1; Tables 1–3), with three-seed statistics reported across all six configurations in Appendix B.12 (Tables 23–25).

We hope this decomposition separates the underlying primitives, the coupled identify–propose–verify–retain mechanism, and the controlled quantitative contribution of each stage to clarify both our technical novelty and the source of CGPI's empirical gains.

AuthorsOpenReview ↗

Follow-up on Critical Weakness 2: Stronger policies and quantitative robot evidence

To establish that CGPI scales beyond earlier specialist backbones and qualitative demonstrations, we added matched-backbone evaluations on top-performing R2R policies, an open-weights VLM navigation interface, and 120-trial paired physical-robot evaluations across diverse instruction types and visual perturbations.

1. Does adaptation still help a stronger specialist navigator? In Appendix B.11 (Table 22), we evaluate ScaleVLN and SRDF on R2R Val Unseen. Each Frozen/CGPI pair shares the exact same pretrained checkpoint, native visual features, candidate construction, action masks, and execution budget, adapting only CGPI's lightweight residual reranker. ScaleVLN improves from 81.0/70.0 to 82.7±0.2/72.1±0.3 SR/SPL (+1.7/+2.1 points), and SRDF improves from 85.8/78.6 to 87.0±0.2/79.9±0.2 (+1.2/+1.3 points). Even in this high-performance regime (>81–85% SR), CGPI delivers consistent gains in both success and path efficiency without backbone retraining.

2. Does CGPI help a VLM-based candidate-action navigator? We implemented a zero-shot VLM navigation interface using Qwen3-VL-8B-Instruct (Appendix H.4, Table 39). Valid waypoint and STOP candidates are supplied to the frozen VLM; its candidate-identifier logits define the frozen reference policy, while its frozen hidden representations drive CGPI's bounded residual reranker. Under the same checkpoint, prompt, candidates, controller, and evaluation suite, Frozen Qwen succeeds on 39/120 (32.5%) attempts versus 45/120 (37.5%) with CGPI (+5.0 percentage points). Alongside this paired evaluation, we report GPT-5.5 at 47/120 (39.2%) and Gemini 2.5 Pro at 49/120 (40.8%) as fixed-policy, zero-shot comparators. Applying CGPI to the open-weights 8B VLM substantially narrows the gap to proprietary frontier models through online test-time adaptation alone.

3. Does CGPI yield quantitative real-robot gains across diverse instructions and visual conditions? In Appendix H.1–H.2 (Tables 35–37), we evaluate R2R-CE-pretrained BEVBert against BEVBert+CGPI on a physical Unitree Go2 quadruped across four indoor environments (30 instructions × 4 repetitions = 120 attempts per method). Ten independently initialized paired blocks match starting poses, candidate generation, low-level control, and execution budgets. Success increases from 26/120 (21.7%) to 37/120 (30.8%), a net gain of +11/120 (+9.2 percentage points). Across the 120 paired attempts, 17 succeed exclusively with CGPI, 6 exclusively with Frozen BEVBert, 20 with both, and 77 with neither; all 94 (Frozen) and 83 (CGPI) failed or intervened attempts remain in the denominator and are categorized by failure mode in Table 36.

Gains hold consistently across instruction categories—clear landmarks (14/40→17/40), abstract or ambiguous language (8/40→12/40), and ordered multi-stage instructions requiring sequential completion (4/40→8/40)—as well as under controlled visual perturbations including reduced illumination (5/24→8/24), partial occlusion (4/24→6/24), and combined lighting plus occlusion (3/24→5/24) in Table 37.

4. What precisely runs on Go2? On the physical robot, BEVBert proposes discrete waypoint and STOP candidates, CGPI adapts only their selection logits, and a fixed low-level controller executes the selected waypoint. As clarified in Appendix H.1, the multiple arrows illustrated in Figs. 3 and 6 depict successive low-level motion commands along the executed path rather than simultaneous reranker candidates. Appendix K further formulates a prospective extension showing how matched-horizon chunk credit, residual chunk-candidate reranking, and boundary verification extend to action-chunk policies.

While adapting end-to-end continuous chunk policies such as NaVILA or OmniNav on their native benchmarks remains a promising direction for future work, our controlled evaluations across ScaleVLN, SRDF, Qwen3-VL-8B-Instruct, and the 120-trial Unitree Go2 suite directly establish that CGPI provides consistent quantitative gains across stronger specialist backbones, VLM-based navigators, and physical robot deployment.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.