acceptodds
Under review as a conference paper at ICLR 2027

The Goal Is Not Enough: Vision-and-Language Navigation under Negative Spatial Constraints

Abstract

Vision-and-Language Navigation in Continuous Environments (VLN-CE) is typically evaluated by whether an agent reaches the goal, rather than whether it obeys what the instruction forbids. Consequently, an agent may reach the target while entering a restricted region or approaching a fragile object—an unsafe success that standard VLN metrics fail to expose. We introduce , a benchmark for constraint-compliant navigation under language-specified negative spatial constraints. NS-VLN grounds these constraints as forbidden regions in continuous environments and provides constraint-compliant reference trajectories, a three-level taxonomy of separated, linguistically integrated, and semantically entangled constraints, and safety-specific metrics that distinguish goal success from safe success. It further includes forbidden–allowed counterfactual pairs for diagnosing whether an agent understands how a referent is used in the instruction or merely associates particular object categories with danger. Our evaluation reveals substantial unsafe success across representative VLN-CE agents and shows that naive training on constraint-augmented data still leaves a substantial gap in reliable constraint compliance. As a complementary method for graph-based VLN agents, we introduce (Constraint-Conditioned Navigator), which learns linguistic-role-sensitive transition risks, composes them into route-level risk, and performs risk-aware waypoint selection. Together, NS-VLN and C2Nav establish negative spatial constraints as a first-class objective for continuous VLN evaluation and learning.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Vision-and-Language Navigation, Negative Spatial Constraints, Safe Embodied Navigation

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_u8qjOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The submission introduces NS-VLN, a continuous VLN benchmark for negative spatial constraints, and C²Nav, a constraint-conditioned navigator. NS-VLN is built from R2R-CE and English RxR-CE in Matterport3D and organises prohibitions into three levels: L1 separated, L2 linguistically integrated, and L3 semantically entangled. Each episode pairs a goal-directed instruction with a forbidden region defined by a dilated horizontal object footprint; success requires both reaching the goal and avoiding all forbidden regions over the full trajectory. C²Nav augments ETP-R1 with instruction-conditioned transition costs, counterfactual role calibration, and risk-aware graph planning. Experiments on Validation Unseen show that goal-only GSR masks unsafe success and that C²Nav improves SSR over ETP-R1+.

Critical strengths

The problem is timely and well motivated. The GSR/SSR decomposition is a clean diagnostic, and the L1–L3 taxonomy is a useful operationalisation of linguistic and semantic difficulty. The counterfactual forbidden-allowed pairing is a sensible supervision signal for role sensitivity, and the metric suite—SSR, VR, Unsafe, NDTW, SDTW, MSM—is appropriately safety-aware. The core empirical observation, that goal-reaching metrics can conceal constraint violations, is valuable for the embodied navigation community.

Critical weaknesses
  1. C²Nav is not yet empirically established. NaVIDA reports higher SSR (36.9 vs. 36.1), and C²Nav regresses on L2 (25.8 vs. 35.6 for ETP-R1 and 31.2 for ETP-R1†), directly challenging the claimed advantage on linguistically integrated constraints.
  2. The contribution of each component remains unclear. No ablations isolate role-sensitive transition costs, counterfactual calibration, risk-aware planning, or the two-stage schedule; sensitivity to λ, m, β, and noisy-OR is absent.
  3. Benchmark construction lacks validation. There is no inter-annotator agreement for L2/L3, the LLM rewriting/manual check is unquantified, and L1 uses a permissive similarity threshold (>0.2) with 42.3% retention. The main protocol checks only the agent centerline, ignores y, and fixes δ = 0.3 m; a stricter footprint-based check would strengthen the safety claims.
  4. Evaluation is narrow and statistically weak: only Validation Unseen, with no test split, seeds, confidence intervals, or significance tests. Baselines differ in training regime, and zero-shot LLM adaptation is under-specified.
  5. Generalization is limited to English, 72 Matterport3D scenes, and static indoor settings. L3 claims about coreference and memory are not directly probed.
  6. Minor clarity/reproducibility issues include notation inconsistencies (e.g., f_flue vs. f_true), a caption typo, and missing compute/code/hyperparameter details.
Questions for authors
  1. Table 4 reports C²Nav L2 SSR of 25.8, below ETP-R1 (35.6) and ETP-R1† (31.2). Since L2 is precisely where linguistic integration should matter, what explains this regression—over-avoidance, degraded instruction parsing after fusion, or optimization instability?
  2. In Table 3, NaVIDA achieves higher SSR than C²Nav (36.9 vs. 36.1). Is this comparison under identical observation, action, and training protocols? If not, could you report C²Nav with the same backbone and training budget, or clarify why this gap does not undermine the primary safety claim?
  3. Could you report performance without role-sensitive transition costs, without counterfactual role calibration, without risk-aware graph planning, and with a single-stage schedule? Sensitivity to λ, margin m, β, and the noisy-OR aggregation would also help isolate what drives the gains.
  4. For L2/L3 human annotations, what is the inter-annotator agreement? How many annotators and items were involved, and how were disagreements resolved? For L1, what is the path-similarity distribution and rejection rate by source corpus?
  5. Why does the main evaluation check only the agent centerline and ignore the vertical coordinate? Would the ranking in Table 3 change under an agent-footprint sweep or a full 3D violation check? Appendix B reports one controlled pair; extending this to all methods would strengthen the safety claims.
  6. How were the LLM-based baselines prompted and adapted to NS-VLN? Were they given the full L1–L3 instructions, and were they purely zero-shot or tuned? Details on ETP-R1† training data, schedule, and compute would also clarify comparability.
2 comments

Dear Reviewer u8qj,

Thank you for your review. The authors have submitted their rebuttal. Given your initial rating of , could you please check whether their response has addressed your concerns? Your prompt feedback would be greatly appreciated, as it would allow the authors to provide further clarification if needed.

Best,

Your AC

Thank you for the clarifications and additional results. I acknowledge that I misread the L1 violation rate as the L2 safe success rate, and I withdraw my concern about L2 regression. I also acknowledge the component ablations in Table 5. The additional experiments and annotation details substantially address my main concerns. Some limitations remain regarding the geometric assumptions and component attribution. Overall, I view the benchmark contribution positively and will raise my rating from 2 (weak reject) to 3 (weak accept), keeping my confidence at 2.

Reviewer_YmsFOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces NS-VLN, a vision-language navigation benchmark built around negative spatial constraints It separates goal success from safe success and spans three levels of constraint difficulty, from explicit prohibitions to context-dependent referents. The authors also propose C2 Nav, which predicts instruction-conditioned transition risks and uses them in route planning. Existing VLN agents show a large gap between goal and safe success, and C2 Nav improves SSR over a fine-tuned ETP-R1 baseline.

Critical strengths

The paper shows that standard goal success can substantially overstate instruction compliance; for example, ETP-R1 reaches 53.0% GSR but only 26.0% SSR. Standard VLN-CE evaluation can systematically miss this failure mode, and the proposed metrics measure it concretely. C2 Nav assigns risks to local transitions, composes them along graph routes with a noisy-OR rule, and applies the resulting route risk before waypoint selection. Relative to the retrained ETP-R1 backbone, it improves SSR by 6.5 points and reduces VR by 5.1 points. Ablations indicate that transition-risk learning and counterfactual calibration improve grounding and role separation, while risk-aware planning drives the main navigation gains. Good diagnostic analysis. Counterfactual pairs, ablations, and the constraint-masking experiment suggest C2 Nav responds to the linguistic role of the forbidden object rather than learning generic object avoidance.

Critical weaknesses
  1. Forbidden regions are fixed-margin (δ = 0.3 m) dilations of 2D bounding boxes, and many instructions are produced or rewritten by an LLM before two-annotator review. The paper reports filtering and geometry sensitivity, but offers no inter-annotator agreement, no audit of accepted versus rejected cases, and no human check that these regions capture the intended meaning of “avoid,” “do not approach,” or “keep distance.” As NS-VLN is the core contribution, the lack of validation for instruction–region–trajectory alignment and for the naturalness of LLM-rewritten prohibitions weakens confidence in the benchmark.

  2. C2 Nav is compared mainly on one graph-based backbone (ETP-R1), so it is unclear whether the gains carry over to other architectures or depend on this specific model. Variance across training runs is not reported. Performance on L3 also stays low (30.3% SSR, 37.2% VR), indicating the method does not yet reliably handle the semantically entangled cases it targets. Together, these points reduce the significance of an otherwise clear improvement over the controlled baseline.

Questions for authors
  1. Are C2 Nav’s gains consistent across different random seeds, and on the R2R-CE-based and RxR-CE-based subsets separately? These two sources differ in how detailed their instructions are and how they were built.
  2. How much did the annotators agree with each other at each review stage, separately for L1, L2, and L3? Also, how often did the verifier reject the first annotator’s work, and can you share a short summary of the common errors in the final dataset?
1 comment

Thanks to the authors for the detailed response. The additional experiments address most of my concerns.

The three-seed results show that the C²Nav gains are stable, and the separate R2R-CE / RxR-CE results are useful. The agreement and verifier pass rates give me more confidence in the benchmark quality-control process.

I still have a minor concern about the semantic meaning of the fixed forbidden-region geometry. The human checks for ambiguity, trajectory consistency, and unnatural detours alleviate this concern.

Overall, the rebuttal reduces my concerns about dataset quality. I will keep my rating at 3, but I am more positive after the rebuttal.

Reviewer_La9POpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces NS-VLN, a benchmark for evaluating whether agents obey language-specified negative spatial constraints while reaching a goal in continuous environments. Built on R2R-CE and English RxR-CE, NS-VLN contains 22,569 episodes across 72 Matterport3D scenes and organizes negative spatial constraints into three levels: separated, linguistically integrated, and semantically entangled. A forbidden region is defined by dilating the horizontal footprint of the forbidden object instance's annotated Matterport3D axis-aligned bounding box by 0.3 m, and a trajectory violates the constraint if any segment of the agent-center trajectory intersects that region. The benchmark provides constraint-compliant reference trajectories, safety-specific metrics including Safe Success Rate (SSR) and Violation Rate (VR), and forbidden–allowed counterfactual pairs. The authors also propose C2Nav (Constraint-Conditioned Navigator), which augments ETP-R1 with role-sensitive transition costs and risk-aware graph planning. On Validation Unseen, C2Nav improves SSR from 29.6 to 36.1 and reduces VR from 35.8 to 30.7 relative to the NS-VLN-trained ETP-R1 baseline.

Critical strengths

The strongest aspect of the paper is that it identifies and operationalizes a failure mode that standard VLN metrics can overlook: reaching the goal does not imply compliance with a language-specified forbidden region. By combining full-trajectory violation checks with GSR, SSR, and VR, NS-VLN turns this form of unsafe success into a measurable evaluation target. Table 3 shows substantial GSR–SSR gaps across multiple representative agents, demonstrating that the benchmark exposes failures that conventional goal-success evaluation does not capture.

Critical weaknesses
  1. The problem itself is compelling, but the paper does not yet delineate NS-VLN's distinct contribution clearly enough relative to prior work on physically safe navigation and language-constrained planning. Safe-VLN already addresses collision avoidance in VLN-CE; SAGE-3D and related recent benchmarks emphasize explicit collision geometry and physically executable trajectories; and work on language-to-temporal-logic grounding, such as Lang2LTL, has studied natural-language specifications with constraints such as always avoid. Against this background, the clearest contribution of NS-VLN appears to be the systematic formulation of instruction-dependent, object-level forbidden regions as a continuous VLN benchmark, together with its construction and evaluation protocols, rather than the first introduction of physical safety or negative language constraints. Without a sharper distinction, readers may interpret NS-VLN as a replacement for physically safe navigation or general negative-constraint planning and overstate its scope. I encourage the authors to narrow and clarify the novelty claim, provide a substantive comparison with Safe-VLN, SAGE-3D, and Lang2LTL-style approaches, and explain how these directions complement NS-VLN.
  2. The current experiments show that NS-VLN can be constructed and evaluated and that its constraints affect model behavior, but they do not fully establish either that negative-language understanding is a capability distinct from ordinary language grounding and conditional planning, or that L1-L3 measure progressively deeper reasoning. The forbidden–allowed counterfactual pairs are used mainly to train and evaluate the transition-cost estimator through PairAcc and ΔRisk. The constraint-masking intervention demonstrates dependence on the annotated constraint span, but it is not equivalent to switching the allowed/forbidden role of the same referent in end-to-end navigation. Table 5 also shows that the substantial SSR improvement appears only when risk-aware planning is enabled. Moreover, L1 is automatically constructed, L2 mixes automatic and human construction, and L3 is manually constructed, while the levels use different episode subsets and retention rates. The monotonic performance degradation in Table 4 may therefore also reflect differences in construction source, object category, scene geometry, or route difficulty. The current evidence convincingly shows that the benchmark exposes unsafe success, but it does not by itself isolate a distinct negative-language capability or justify interpreting the cross-level decline as evidence of increasing reasoning depth. I suggest emphasizing the benchmark contribution while moderating these stronger claims, unless more discriminative end-to-end evidence or controlled comparisons can be provided.
Questions for authors
  1. Does the paper evaluate the forbidden–allowed counterfactual pairs in end-to-end navigation, rather than only through transition-cost diagnostics?
  2. Does C2Nav introduce additional detours or reduce performance on ordinary VLN instructions that contain no negative spatial constraint?
  3. What was the rejection rate of the second annotator's binary verification pass for each constraint level?
  4. In Validation Unseen, can the forbidden referent be localized before the agent must commit to a detour, and does the split also contain forbidden-object categories not observed during training?
  5. How do the authors determine whether the main performance bottleneck lies in understanding and grounding the negative spatial constraint or in subsequent detour planning? If feasible, an oracle diagnostic that plans with ground-truth forbidden-region geometry could help separate these error sources.
  6. How sensitive are C2Nav’s GSR, SSR, and VR to the planning coefficient λ, and how was the reported value selected?
2 comments

Dear Reviewer La9P,

Thank you for your review. The authors have submitted their rebuttal. Given your initial rating of , could you please check whether their response has addressed your concerns? Your prompt feedback would be greatly appreciated, as it would allow the authors to provide further clarification if needed.

Best, Your AC

Thank you for the detailed rebuttal and additional results. I have carefully reviewed the authors’ response and consider my major concerns to be substantially addressed.

Regarding W1, the response more clearly distinguishes NS-VLN from collision-avoidance benchmarks and language-to-formal-specification approaches. I agree that instruction-dependent, object-grounded forbidden regions constitute a distinct benchmark setting for continuous VLN, provided that the revised paper presents this contribution boundary explicitly and treats C2Nav as a complementary method rather than the primary source of novelty.

Regarding W2, I appreciate the authors’ acknowledgment that negative-language understanding is part of general language grounding rather than an entirely independent linguistic capability. The clarification that L1–L3 form an operational taxonomy of increasing linguistic and semantic complexity, rather than a formally validated hierarchy of reasoning depth, directly addresses my concern about the strength of the original claims.

The additional experiments are also helpful. The end-to-end forbidden–allowed evaluation provides behavioral evidence beyond PairAcc and ΔRisk, while the ground-truth forbidden-region oracle meaningfully distinguishes limitations in constraint grounding and risk estimation from those in subsequent planning and execution. The results on ordinary VLN instructions, the human-verification statistics, and the clarification of the Validation Unseen split also address my corresponding questions.

Overall, I consider my major concerns substantially addressed and will update my rating from 2 (Weak Reject) to 3 (Weak Accept). My confidence remains 3.

Comments on the paper

1 comment
Area_Chair_VaUjOpenReview ↗

discussion starts

Dear Reviewers,

The authors have submitted their rebuttal. Please review it and provide further comments as appropriate.

Best, Your AC

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.