acceptodds
Under review as a conference paper at ICLR 2027

Beyond the Endpoint: Rethinking Termination in Vision-Language Navigation

Abstract

Vision-language navigation requires agents to follow language instructions, reach the intended destination, and stop at the appropriate termination. However, standard benchmarks define success by proximity to a single demonstrated endpoint, even though an instruction may permit multiple valid stopping positions. This criterion therefore credits instruction-inconsistent stops and rejects valid alternatives, conflating endpoint proximity with instruction satisfaction. We address this mismatch by representing each instruction with an instruction-conditioned goal set, comprising the navigable positions that satisfy its terminal spatial requirements. This formulation induces two evaluation metrics: Goal-Set Success Rate (GSR), which measures whether the agent stops within the goal set, and Goal-Set Oracle Success Rate (GOSR), which measures whether its trajectory reaches the set at any point. To operationalize this, we construct goal sets by extracting these requirements from language, grounding the referenced rooms and objects in the scene, and compiling the resulting constraints into navigable regions. Applying this framework to R2R and RxR produces goal-set annotations for 65,253 training instructions and manually verified evaluation assets for 12,845 validation cases. Re-evaluating 17 open-source VLN methods substantially changes measured success and method rankings, revealing both endpoint successes that never reach an instruction-consistent destination and trajectories that reach a valid destination but fail to stop there. We further introduce NavJev, a lightweight plug-in for termination control that learns from goal-set supervision while keeping the navigation policy frozen. Across six R2R policies, NavJev improves mean GSR from 41.05% to 42.97%. These results establish instruction-conditioned goal sets as a common foundation for evaluating destination arrival and improving termination decisions.

Primary area:
applications to robotics, autonomy, planning
Keywords:
vision language navigation

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_kyezOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

The paper questions whether proximity to a demonstrated endpoint adequately measures satisfaction of the instruction’s terminal spatial requirements in vision-language navigation. It constructs instruction-conditioned goal regions for R2R and RxR and introduces Goal-Set Success Rate (GSR) and Goal-Set Oracle Success Rate (GOSR). Re-evaluating 17 navigation methods changes success labels and rankings. The paper also proposes NavJev, a vision-language-model-based stopping controller for frozen navigators, reporting an increase in mean GSR from 41.05% to 42.97% across six R2R policies.

Critical strengths

The paper identifies a valid problem with endpoint-based evaluation: a nearby position can violate the instruction, while a more distant position can satisfy it. This observation exposes a limitation that cannot generally be resolved by adjusting the distance threshold and motivates examining the semantic requirements of termination.

Critical weaknesses
  1. The construction of the evaluation target is insufficiently established to support the proposed metrics. GSR and GOSR are straightforward membership statistics; their validity rests on how the goal sets are constructed. Although Appendix A describes geometric operations, important semantic decisions remain difficult to reproduce, including the parsing prompts, resolution of ambiguous referents, and treatment of multiple valid interpretations. The nominal definition 𝐺(𝐼,𝐸) also conceals dependencies on the demonstrated endpoint and reference route, which determine the search area, entry opening, and approach direction. The paper does not establish whether these choices exclude valid alternatives. Fixed geometric interpretations, such as treating both "near" and "beside" through a 3,m threshold, are not accompanied by quantitative robustness analysis. The independent audit covers only R2R, leaving the multilingual RxR annotations without comparable verification. Consequently, the ranking changes do not sufficiently establish that the proposed criterion is a generally more faithful evaluation standard. 2, NavJev is neither specified sufficiently for reproduction nor evaluated in a way that establishes its claimed contribution. The model and main optimization settings are given, but "terminal stage" is not operationally defined, and the construction of hard-positive and hard-negative states and their corresponding visual histories is unclear. The complete completion prompt and backbone-specific STOP-masking interfaces are also missing. More fundamentally, the experiments introduce a 4B vision-language model, goal-set supervision, hard-example sampling, and forced continuation together, then compare the complete system only with native stopping. Without an otherwise identical controller trained on endpoint-based labels, the gains cannot be attributed to the proposed supervision. Changes in aggregate GSR and GOSR also cannot establish whether improvements arise from correcting termination decisions after the trajectory has already reached a valid goal region, or from the effect of suppressing the navigator’s native STOP action and allowing the frozen policy to continue executing additional actions before termination. These omissions leave both the method's reproducibility and the explanation for its improvement unresolved.
Questions for authors
  1. How are ambiguous referents and multiple valid interpretations converted into a final goal set? When does reference-route context merely resolve ambiguity, and when does it exclude alternative stopping regions?
  2. Do the principal ranking reversals persist under reasonable alternative spatial thresholds? What independent evidence supports the semantic accuracy of the multilingual RxR goal sets?
  3. What precisely defines a terminal-stage training state? How are hard-positive and hard-negative positions sampled, and how are their visual histories constructed from the demonstration data?
  4. How does NavJev compare with an otherwise identical controller trained on endpoint-based labels? Among its recovered successes, how many had already reached the goal set in the original run, and how many reached it only after the original STOP was suppressed?
4 comments
AuthorsOpenReview ↗

Part I. Details on goal-set construction

Thank you for recognizing the limitation of endpoint-based evaluation and for asking how we turn terminal semantics into a reproducible target.

How are ambiguities resolved, and when does the reference affect the region? (W1, Q1)

We specify parsing, grounding, and geometric construction, explicitly including the demonstrated endpoint and route in G(I,E,Dref)\mathcal{G}(I,E,D_{\mathrm{ref}}). The endpoint localizes candidate entities; the route supplies entry and approach context (Section 3.2, Eq. 1).

(a) Parsing inputs, outputs, and validation (Appendix A.1). Qwen3.8-Flash receives a fixed prompt and JSON containing only the case identifier and full instruction. Each case in the output results contains entities (IDs, object/room categories, primary/reference roles, and attributes), constraints (operators, entity IDs, stop/identity scope, and supporting source_text), and the verbatim terminal_text and orientation_text. For example, “enter the bedroom and stop beside the chair” yields room-membership and chair-proximity conditions. Validation checks JSON parsing, a nonempty terminal clause, and complete entity references. Decoding uses temperature 0, extended thinking disabled, and an 8,192-token cap with prompt shown in the appendix A.1.

(b) Resolving ambiguity and selecting interpretations (Appendices A.1 and A.3). Candidates are searched on the endpoint's floor, in its source room and rooms connected through adjacent openings: the Matterport3D inventory supplies candidate objects/rooms, overhead views fix connectivity and extent, and RGB-D panoramas and object crops resolve appearance and size, checking categories and attributes, inter-entity relations, spatial extent, and floor.

Selection follows “identity first, then region, effective only on confirmation.” A descriptive relation in the instruction first fixes entity identity (“the chair beside a fireplace” identifies the target through that relation, which serves identity only); the stopping region is then compiled around the confirmed entity, with the side/orientation frame set by the terminal predicate type (later in d). Competing readings are kept as separate hypotheses and compiled into separate candidate regions, never merged prematurely. A reading becomes the effective region only when the instruction and scene evidence jointly confirm it (confirm-only): the reference endpoint/route supplies search context and frame only. Within a confirmed reading all terminal predicates are intersected, and an empty region triggers inspection; any entity, relation, or orientation that cannot be uniquely resolved is disclosed together with all competing candidates and sent to human review. All 12,845 validation cases are additionally reviewed individually against the instruction, route, and scene views.

(c) Coordinate alignment and reachable regions (Appendix A.2). Matterport3D house coordinates are transformed into Habitat coordinates:

(xH,yH,zH)=(xM,zM,−yM).(x_H,y_H,z_H)=(x_M,z_M,-y_M).

Here, yHy_H is vertical and xHx_H–zHz_H is horizontal; objects, routes, regions, and stops share this convention. A 0.1 m grid represents reachable surfaces, retaining height to distinguish overlapping floors. Object proximity propagates from cells with an unobstructed connection to the surface, using the surface gap plus the shortest traversable grid path and retaining positions within 3 m that satisfy the other terminal constraints.

(d) Directional projection and route dependence (Section 3.2; Appendix A.2). Taking the object center as the local origin, we project the candidate stop's horizontal offset onto a forward axis and its perpendicular leftward axis. The projection signs select the front/back and left/right half-planes. The source of the forward axis determines route dependence:

FrameConstruction and route dependence
Object-relative, e.g., “in front of the sofa”Scene views identify the object's front. With the referent and other constraints fixed, changing the approach preserves this directional condition.
Travel-relative, e.g., “left of the bed as you enter”Use the final nonzero reference-approach direction. This also supplies the frame when object orientation is insufficient; ambiguous orientations receive human review. Changing the approach can rotate the accepted half-plane.
Entry-relative, e.g., “just inside the doorway”Select the reference route's final entry opening and retain room-interior cells within 3 m traversable distance. Changing the opening can change the region; an unqualified “enter the room” retains the room.

Among side predicates with recorded frames, 60.8% on R2R and 47.9% on RxR use approach directions; the remainder use object axes. Intersecting these conditions with reachability, room membership, and proximity gives the goal set for that reference context.

AuthorsOpenReview ↗

Part II. Additional results on alternative spatial thresholds and multilingual semantic accuracy

Alternative spatial threshold? (W1, Q2a)

We added a 1m comparison: NaVIDA and StreamVLN remain the leaders under both settings, while some intermediate ranks change.** We rescore the same six policies and all 1,839 R2R val_unseen episodes, holding trajectories, referents, and other spatial constraints fixed. GSR measures explicit STOP inside the goal set; GOSR measures whether the trajectory ever enters it.

MethodGSR@3 m (%)GSR@1 m (%)GOSR@3 m (%)GOSR@1 m (%)
NaVIDA50.3536.7057.1549.92
StreamVLN46.8232.5752.4743.83
ETP-R144.0528.3849.9238.88
Uni-NaVid43.0129.5847.0938.66
AwareVLN42.5229.7450.7941.11
NaVILA39.3727.6245.5136.32

NaVIDA and StreamVLN remain ahead of ETP-R1 and AwareVLN, and Uni-NaVid remains ahead of NaVILA under both GSR settings. ETP-R1 and AwareVLN exchange third and fifth place. GSR falls by 11.75–15.67 percentage points and GOSR by 7.23–11.04 points, quantifying sensitivity in absolute scores and some relative orderings. These results and region comparisons are included in Appendix B.4.

multilingual semantic accuracy? (W1, Q2b)

We added a blinded audit of 500 RxR positions; goal sets reduce errors by 15–20 percentage points relative to the 3 m endpoint rule. The sample comprises 100 actual policy stops per language and 200 diagnostic positions across region interiors, boundaries, and exteriors. Two annotators judge independently, with a third resolving disagreements; masks, policy names, automatic decisions, and highlighted reference endpoints are hidden. Instructions are read in their original language, with bilingual adjudication for Hindi and Telugu.

RxR samplePositionsEndpoint errorsGoal-set errorsGoal-set precision / recall (%)Error reduction (pp)
English policy stops10024895.6 / 92.916.0
Hindi policy stops100251093.9 / 91.215.0
Telugu policy stops10024994.1 / 92.815.0
Diagnostic positions200622289.0 / 89.020.0

Within each language, both criteria are compared against human labels at the same positions. Policy-stop precision of 93.9–95.6% and recall of 91.2–92.9% provide direct evidence of semantic accuracy. The protocol and results are included in Appendix A.4 and Table 5.

AuthorsOpenReview ↗

Part III. Additional details and ablations on NavJev

How are training states, histories, and STOP interfaces specified? (W2, Q3)

We added the sampling rules, complete completion prompt, and shared control procedure in Appendix.

  • States and hard examples. The terminal stage starts at the demonstration's first goal-set entry, or comprises its final KK steps if it never enters; the sampler records KK and the trigger step. Hard positives lie inside the goal set and outside the 3 m endpoint neighborhood; hard negatives satisfy the reverse membership. Both are sampled with ordinary examples from the same suffix state pool.
  • Visual history. Each state receives the instruction and 16 uniformly sampled frames from the trajectory start through that state, repeating frames for short prefixes. Training uses the corresponding demonstration prefix; deployment uses the executed prefix.
  • Score and control. Qwen3-VL-4B-Instruct predicts stop or not stop, with score P(stop)/[P(stop)+P(not)]P(\mathrm{stop})/[P(\mathrm{stop})+P(\mathrm{not})] from first-token probabilities. Queries begin after 20 low-level actions; two consecutive scores of at least 0.5 trigger STOP. Full NavJev masks native STOP throughout the episode and requests a movement at the same step whenever continuation is required. All six backbones share the checkpoint and control rule.

Training uses 168,726 R2R decisions: 32,290 stop and 136,436 not stop. The checkpoint after one training epoch serves every policy. These details, the full prompt, and the deployment procedure appear in Section 5.1 and Appendices C.1–C.3.

Can the gains be attributed to goal-set supervision? (W2, Q4)

We added a crossed comparison of supervision and native STOP handling: goal-set labels outperform endpoint labels under both STOP rules. Controllers share the 4B model, training states including hard examples, instructions, and 16-frame histories; only labels differ. All configurations use the same 1,839 episodes and action budget. “Honored” lets either navigator or controller stop; “Masked” gives the controller stopping authority. GSR (%) follows.

SupervisionNative STOPETP-R1AwareVLNStreamVLN
Native baselineHonored44.0542.5246.82
EndpointHonored44.1042.6346.87
Goal setHonored45.1343.8847.15
EndpointMasked44.2142.7446.98
Goal setMasked46.1745.1947.42

With masking fixed, goal-set labels add 1.96, 2.45, and 0.44 points, respectively. With goal-set labels fixed, masking adds 1.04, 1.31, and 0.27 points. Every endpoint-supervised configuration stays within 0.25 points of the native baseline. This positively shows that the endpoint region is an inappropriate stop-supervision target: under matched DAgger-style training (same model, visited states, and demonstration prefixes, differing only in the label), DAgger expands the visited-state distribution, but if the STOP label is still defined by the reference endpoint, more states do not redefine which positions satisfy the instruction's terminal condition, so endpoint supervision barely changes the stop decision; goal-set supervision instead materially improves the actual stop decision, confirming the need for stop supervision. The six-policy mean of 41.05%→42.97% measures the full system and this additional experiment appears in Section 5.2 and Table 3.

How many recoveries follow prior arrival or continued execution? (W2, Q4) We added an episode-paired analysis: most new successes for each policy had already visited the goal set in the native run, and continuation produces additional arrivals. For first arrivals after STOP suppression, we verify that both runs share the action history through the native stopping state.

MethodNew successesGoal set visited in native runFirst reached after STOP maskingSuccesses lost
ETP-R151331812
AwareVLN63531014
StreamVLN2422213

After subtracting successes lost, the three policies net 39, 49, and 11 additional successful episodes. Most new successes are cases where the native trajectory had already reached the goal set but stopped at the wrong position, so the gain mainly reflects more accurate stopping; continued execution after STOP masking adds further arrivals. This is consistent with Q5: goal-set stop supervision improves the actual stop decision, and continuation expands arrival. Results are included in Appendix C.4 and Table 10.

AuthorsOpenReview ↗

Sincerely looking forward to your more feedbacks

Dear Reviewer kyez,

We sincerely appreciate your efforts put into the reviewing process, which has significantly contributed to the refinement of our manuscript. We've carefully examined all the constructive questions and have accordingly revised our work. To make the best use of the discussion period and to improve our work, we are eager to know whether our answers adequately address your concerns, as it is crucial for us to have a candid and thorough discussion to continuously strengthen our method. Please share your thoughts on viewing our reply. We hope to resolve your doubts with our best efforts.

We are ready to respond to any further issues raised. Please let us know.

Sincerely,

The Authors

Reviewer_qsMoOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper revisits termination evaluation in VLN, arguing that proximity to a single demonstrated endpoint does not always reflect whether the stopping position satisfies the instruction. It constructs instruction-conditioned goal sets for R2R and RxR, introduces goal-set evaluation metrics, and uses them to reassess 17 existing methods. The paper also proposes NavJev, a stopping module trained with goal-set supervision that can be attached to frozen navigation policies.

Critical strengths

The paper identifies a meaningful limitation of endpoint-based evaluation: a fixed-radius region may accept positions that violate the terminal instruction while rejecting other reasonable stopping positions. The large-scale annotations, human verification, and re-evaluation of existing methods provide useful evidence that this choice can affect both success labels and method rankings.

Critical weaknesses

See Questions for Authors below.

Questions for authors

I agree that the proposed goal sets are useful for filtering out stops that are close to the endpoint but do not satisfy the terminal semantics, and the formulation also seems reasonable for goal-reaching benchmarks.

My concern is that R2R is also an instruction-following task, where the endpoint of the expert trajectory usually reflects the intended route and final approach. Is it always appropriate to expand the success region beyond this endpoint?

For example, suppose a bed can be reached through doors on both sides, but the instruction asks the agent to enter through the right door and the reference endpoint is naturally located on the right side of the bed. Counting the left side as an equally valid stopping region may include an agent that entered through the other door or took an unnecessary detour around the bed. In such cases, the final position may satisfy the goal semantics, while the trajectory may be less consistent with the instruction.

I think this distinction between terminal validity and route consistency is worth further discussion.

2 comments
AuthorsOpenReview ↗

Part I. Discussion on success region and two door cases

Thank you for recognizing this endpoint-evaluation limitation and valuing our human-verified annotations and re-evaluation evidence on success labels and rankings.

  1. When should a stop beyond the endpoint neighborhood?

We believe termination should be judged beyond the single demonstrated endpoint. A point cannot reliably capture termination accuracy, and the 3m-radius traversable round is a coarse proxy in the past: it may cross a doorway into adjacent space the instruction never required, or miss admissible stops. And this coarseness motivated our goal set. Acceptance condition is that stop must satisfy every stated terminal spatial constraint. “Stop by the bed” allows several reachable positions meeting room and proximity conditions; “stop on the right side” adds a side constraint; “stop just inside the right doorway” restricts the region to that entry. The endpoint records one valid execution; the goal set preserves other positions the instruction permits.

Quantitatively, the goal set is usually smaller than the endpoint neighborhood. Comparing G with the endpoint 3 m neighborhood B₃ on the same navigable grid over R2R val_unseen (1,819 comparable episodes): 84.7% differ substantially (IoU < 0.90; shown in Figure 8) and only 15.3% closely match (IoU ≥ 0.90; Figure 7); 80.8% are smaller in area (median 5.92 vs 10.13 m²). Yet 45.3% retain admissible area beyond the neighborhood, so this is a semantic re-partition. Details and examples are in Section 3.2 and Appendices A.2–A.3 and B.2.

  1. How are the wrong door, wrong side, and detour judged?

GSR judges the terminal position, not the route. Which doorway was crossed, detours, and landmark order are route events checked against the trajectory; whether the stop meets the instruction's terminal constraints is judged by the goal set. Per case:

SituationVerdictCan GSR judge it?
“Enter through the right door and stop by the bed,” but entry through the left door and a valid bed-side stopTerminal valid, route violated → full instruction following failsTerminal judged correctly (accepts the valid bed-side stop); the wrong door is not detected
“Stop on the right side of the bed,” but a left-side stopTerminal failureJudged correctly (rejected; side is a terminal constraint)
“Stop by the bed just inside the right doorway,” but just inside the left doorwayTerminal failureJudged correctly (rejected; the entry constrains the terminal region)
Enter through the right door, circle the bed, and stop validlyTerminal valid; if only entry and stop are specified, the detour costs efficiencyTerminal judged correctly; detour efficiency needs SPL/nDTW

Actually, any position-only measure cannot judge the route. For the side frame, object-axis left/right stays constant, while “left/right as you enter” varies with the approach and is set from the reference route during construction. So, GSR is recommanded to be combined with path metrics: nDTW describe geometric similarity or coverage relative to the reference route, SPL/GSPL capture efficiency, and explicitly required route events (e.g., crossing the right doorway from the corridor into the bedroom before stopping) need a per-trajectory check.

AuthorsOpenReview ↗

Part II. Discussion on terminal validity and route consistency

Terminal validity and route consistency are two independent dimensions and Part I have covered the division of labor between GSR and nDTW/SPL. Here, we would like to share our perspective on the evaluation and training of process semantics.

Your example exposes a related issue along the route: a trajectory can deviate substantially from the reference geometry while fully satisfying the instruction. If the instruction only says “proceed down the corridor,” the reference may follow the left side and the agent the right. Both can comply, although lateral separation lowers nDTW. Checking instruction consistency therefore requires distinguishing required route events from permissible movement choices, rather than judging by geometric similarity alone.

However, fine-grained human annotation of individual routes presents two related challenges:

  1. Valid routes are non-unique. Instructions usually constrain selected landmarks, events, and their order, leaving freedom between them. Segment-level labels require judging whether each deviation preserves those constraints, making them difficult to derive directly from reference coordinates or actions.
  2. Process judgments require extended temporal and scene context. Interpreting a turn or detour often requires earlier and later observations, the doorway actually crossed, and subsequent landmark order. Annotators must replay trajectories, inspect the scene, and workload grows with route length, while ambiguous instructions increase disagreement.

So, using a teacher model to score the visual trajectory against the instruction, then using those scores as rewards, is a promising direction. Existing work explores dense step-level supervision from imperfect trajectories (Li et al., 2026, Let's Reward Step-by-Step). Following this approach, a teacher may assess whether actions advance the instruction, complete required events, or recover compliance after a deviation.

Reviewer_fdTrOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper revisits termination evaluation in vision-language navigation by introducing instruction-conditioned goal sets instead of relying on proximity to a single demonstrated endpoint. It proposes GSR and GOSR to evaluate instruction-consistent stopping and goal region arrival, and constructs goal-set annotations for R2R and RxR. Re-evaluating 17 navigation methods reveals substantial changes in success labels and model rankings. The paper further introduces NavJev, a goal-set-supervised stopping module that improves the performance of six frozen navigation policies.

Critical strengths

(1) The paper addresses an important and insightful limitation of existing VLN evaluation protocols.

(2) The experimental evaluation is extensive and convincing. Re-evaluating 17 VLN methods reveals substantial differences between endpoint-based and goal-set-based success.

(3) The paper provides large-scale goal-set annotations with comprehensive human verification and an independent position-level audit.

(4) The improvements of NavJev across six frozen navigation policies demonstrate the practical utility of the proposed supervision.

Critical weaknesses

(1) Equation (1) defines the goal set as G(I, E), but the actual construction uses the demonstrated endpoint to restrict the search region and the reference path to determine entry and directional constraints. This raises questions about whether different valid demonstrations would produce consistent goal sets.

(2) Although all validation annotations undergo human review, the independent position-level audit covers only 700 R2R positions, with policy stops sampled from three models. The reliability of goal-set membership and ranking changes on multilingual RxR remains less thoroughly validated.

(3) On R2R, 93–99% of changed labels correspond to rejected endpoint successes, while only 3–14 previously failed episodes per policy are recovered. Thus, the experiments primarily demonstrate stricter success filtering, while the practical impact of recognizing multiple valid stopping positions remains less extensively established.

(4) The method replaces native STOP decisions with an additional 4B completion model, but lacks a matched endpoint-supervised stopping baseline. Consequently, the average 1.92-point GSR improvement cannot be attributed specifically to goal-set supervision rather than the additional predictor and altered stopping policy.

Questions for authors

(1) How sensitive are GSR, model rankings, and success reclassification to the fixed 3 m threshold and alternative geometric constraint definitions?

(2) Can the authors provide independent annotation validation on RxR and further analyze why most success-label changes are rejections rather than recoveries?

(3) How does NavJev compare with an identical endpoint-supervised completion model? Can the effects of goal-set labels and native STOP suppression be evaluated separately?

(4) How robust is NavJev to policy-generated trajectories and different stopping thresholds? Can the authors report paired significance tests and end-to-end inference latency for its improvements?

2 comments
AuthorsOpenReview ↗

Part I. Additional details and analysis on goal set

Thank you for recognizing our contribution to a key VLN evaluation limitation and for valuing our re-evaluation of 17 methods, audited goal-set annotations, and NavJev across six frozen policies.

When do different demonstrations yield the same or different regions? (W1)

We have made reference dependence explicit in Equation (1), G(I,E,Dref)\mathcal{G}(I,E,D_{\mathrm{ref}}), and specified its two roles. The endpoint localizes candidate objects, rooms, and floors; the route supplies spatial context for entry- and approach-relative terminal expressions. The grounded terminal constraints jointly determine the region.

Consistency depends on referent identity and the constraint frame. For the same grounded entity, room membership, object proximity, and object-axis side constraints remain fixed. “Just inside this door” or “left as you approach” varies with the entry or approach, so different valid routes can yield different regions. Human review checks referent identity, and each directional predicate records its frame: 60.8% on R2R and 47.9% on RxR use approach directions; the remainder use object axes. These details are revised and added in Section 3.2 and Appendices A.1–A.3.

What independently validates RxR semantic accuracy? (W2, Q2)

We added a blinded audit of 500 RxR positions; goal sets reduce policy-stop errors by 15–16 percentage points in each of English, Hindi, and Telugu. The sample comprises 100 actual stops per language and 200 diagnostic positions across region interiors, boundaries, and exteriors. Two annotators judge independently; a third adjudicates disagreements. Masks, policy names, automatic decisions, and highlighted endpoints are hidden. Instructions are read in their original language, with bilingual adjudication for Hindi and Telugu.

RxR samplePositionsEndpoint errorsGoal-set errorsGoal-set precision / recall (%)Error reduction (pp)
English policy stops10024895.6 / 92.916.0
Hindi policy stops100251093.9 / 91.215.0
Telugu policy stops10024994.1 / 92.815.0
Diagnostic positions200622289.0 / 89.020.0

Each row compares both criteria with human labels at identical positions; the endpoint rule uses the demonstrated endpoint's 3 m neighborhood. Policy-stop precision reaches 93.9–95.6% and recall 91.2–92.9%; diagnostic errors fall by 20 points. The protocol and results appear in Appendix A.4 and Table 5.

What is the practical value of alternative stops, and why do rejections dominate? (W3, Q2)

We added examples of both relabeling directions and analyzed their arrival histories. Each R2R policy recovers 3–14 previously failed episodes with admissible alternative stops and loses 159–430 endpoint successes whose stops violate terminal conditions. Rejections account for 93–99% of changed labels.

The imbalance reflects where policies travel: 74.3–95.1% of rejected successes on R2R and 64.4–88.8% on RxR never visit the goal set. They approach the demonstrated endpoint without reaching the required region. Recovery requires an actual stop inside the set and beyond the endpoint neighborhood. Alternative stops therefore have a concrete role, with their frequency determined by policy trajectories. Added examples show valid endpoint-distant stops and goal regions of similar or smaller area. Counts, arrival analysis, and examples appear in Sections 4.1–4.2 and Appendices A.3 and B.2.

How sensitive are scores and rankings to spatial thresholds? (Q1)

We added paired rescoring at 3m and 1m (similar pipeline and manual-reviewed, cases shown in fig10): the leading relationships persist, while absolute scores and some intermediate ranks change. The same six policies and 1,839 R2R val_unseen episodes are evaluated with trajectories, referents, and other constraints fixed. GSR measures explicit STOP inside the goal set; GOSR measures any visit to it.

MethodGSR@3 m (%)GSR@1 m (%)GOSR@3 m (%)GOSR@1 m (%)
NaVIDA50.3536.7057.1549.92
StreamVLN46.8232.5752.4743.83
ETP-R144.0528.3849.9238.88
Uni-NaVid43.0129.5847.0938.66
AwareVLN42.5229.7450.7941.11
NaVILA39.3727.6245.5136.32

GSR falls by 11.75–15.67 points and GOSR by 7.23–11.04 points. NaVIDA and StreamVLN remain first and second; ETP-R1 and AwareVLN exchange third and fifth. The 1 m set is contained in the 3 m set, so tightening proximity only removes accepted positions.

We also specify the remaining geometry: room conditions follow the functional space; “just inside” retains reachable interior positions within 3 m of the selected opening; side conditions use a half-plane defined by the object axis or reference approach. These constraints stay fixed in the threshold comparison. Definitions and results appear in Appendices A.2 and B.4.

AuthorsOpenReview ↗

Part II. Additional ablation and analysis on NavJev

What do goal-set labels and native STOP masking each contribute? (W4, Q3)

We added an otherwise matched endpoint-supervised controller and crossed supervision with STOP handling. Controllers share the 4B model, training states including hard examples, instructions, and 16-frame histories; labels use either the 3 m endpoint neighborhood or goal-set membership. Episodes and action budgets are fixed. “Honored” lets either navigator or controller stop; “Masked” gives the controller stopping authority and requests movement at the same step when native STOP is proposed. GSR (%) follows.

SupervisionNative STOPETP-R1AwareVLNStreamVLN
Native baselineHonored44.0542.5246.82
EndpointHonored44.1042.6346.87
Goal setHonored45.1343.8847.15
EndpointMasked44.2142.7446.98
Goal setMasked46.1745.1947.42

Goal-set labels improve all three policies under both STOP rules. With masking fixed, the gains are 1.96, 2.45, and 0.44 points; with goal-set labels fixed, masking adds 1.04, 1.31, and 0.27 points. Every endpoint-supervised configuration stays within 0.25 points of the native baseline. These results separately support supervision and continuation contributions; the six-policy mean gain of 1.92 points measures the full system. The experiment appears in Section 5.2 and Table 3.

The rollout results, stopping rule, significance, and latency (Q4)

We evaluate one NavJev checkpoint on trajectories generated by six frozen policies and add paired significance tests and inference timing. Every policy improves in GSR, with the mean rising from 41.05% to 42.97%. All use queries after 20 low-level actions and STOP after two consecutive scores of at least 0.5; the results below use this fixed rule.

New successes are split by whether the native trajectory already visited the goal set or first arrived after STOP suppression. For the latter, both runs share the action history through the native stopping state.

MethodNew successesGoal set visited in native runFirst reached after STOP maskingSuccesses lostMcNemar pp
ETP-R151331812<0.001
AwareVLN63531014<0.001
StreamVLN24222130.099

Net gains are 39, 49, and 11 successful episodes. Exact two-sided McNemar tests support significant improvements for ETP-R1 and AwareVLN (both p<0.001p<0.001); StreamVLN yields p=0.099p=0.099.

Additionally, one NavJev query takes approximately 674 ms, mostly encoding the instruction and frames. Encoding overlaps the navigator's forward pass on the same step; additional waiting depends on the portion extending beyond that pass. Our deployment shows no noticeable action pauses. Rollout results, paired tests, and timing appear in Table 2, Appendices C.2/C.4, and Tables 8/10.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary of Revision

Dear Chairs and Reviewers,

We thank all reviewers for their careful and constructive feedback. Our goal is to evaluate instruction-consistent termination in VLN and improve stopping through goal-set supervision. Following your suggestions, we have strengthened multilingual validation, added controlled NavJev experiments, and clarified construction. Revisions are highlighted in blue in the manuscript.

The major changes are summarized below.

Main Text

  • Section 3.2 — Goal-set definition and reference dependence. We explicitly include the reference demonstration in mathcalG(I,E,Dref)mathcal{G}(I,E,D_{\mathrm{ref}}) and clarify how the endpoint, entry opening, and approach direction affect construction, including when alternative routes yield different regions.
  • Section 5.2 — Matched supervision and STOP controls. We add an identical endpoint-supervised controller and separately compare supervision labels and native STOP handling. Goal-set supervision improves GSR under both STOP rules; with STOP masked, gains over endpoint supervision are 1.96, 2.45, and 0.44 percentage points for ETP-R1, AwareVLN, and StreamVLN.

Appendices

  • Appendices A.1–A.4 — Construction details and independent validation. We provide the parsing prompt, ambiguity-resolution procedure, and annotation-review visualizations. We add a blinded audit of 500 RxR positions covering English, Hindi, and Telugu policy stops and spatial diagnostics. Goal sets reduce errors relative to the endpoint rule by 15–20 percentage points across these samples.
  • Appendix A.5 — Terminal validity and route consistency. We add the two-door discussion to distinguish admissible stops, required doorway crossings, and detour efficiency, explaining how GSR complements nDTW and efficiency metrics.
  • Appendix B.4 — Proximity sensitivity. We compare 3m and 1m settings for six policies on the same 1,839 R2R episodes. NaVIDA and StreamVLN retain the top two positions, while ETP-R1 and AwareVLN exchange third and fifth.
  • Appendices C.1–C.4 — NavJev reproducibility and gain analysis. We specify training-state sampling, hard examples, visual histories, the completion prompt, and STOP handling. We also add episode-level recovery analysis and paired significance tests.

Thank you again for helping us improve the clarity and validation of our work.

Yours sincerely, The Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.