Beyond the Endpoint: Rethinking Termination in Vision-Language Navigation

Vision-language navigation requires agents to follow language instructions, reach the intended destination, and stop at the appropriate termination. However, standard benchmarks define success by proximity to a single demonstrated endpoint, even though an instruction may permit multiple valid stopping positions. This criterion therefore credits instruction-inconsistent stops and rejects valid alternatives, conflating endpoint proximity with instruction satisfaction. We address this mismatch by representing each instruction with an instruction-conditioned goal set, comprising the navigable positions that satisfy its terminal spatial requirements. This formulation induces two evaluation metrics: Goal-Set Success Rate (GSR), which measures whether the agent stops within the goal set, and Goal-Set Oracle Success Rate (GOSR), which measures whether its trajectory reaches the set at any point. To operationalize this, we construct goal sets by extracting these requirements from language, grounding the referenced rooms and objects in the scene, and compiling the resulting constraints into navigable regions. Applying this framework to R2R and RxR produces goal-set annotations for 65,253 training instructions and manually verified evaluation assets for 12,845 validation cases. Re-evaluating 17 open-source VLN methods substantially changes measured success and method rankings, revealing both endpoint successes that never reach an instruction-consistent destination and trajectories that reach a valid destination but fail to stop there. We further introduce NavJev, a lightweight plug-in for termination control that learns from goal-set supervision while keeping the navigation policy frozen. Across six R2R policies, NavJev improves mean GSR from 41.05% to 42.97%. These results establish instruction-conditioned goal sets as a common foundation for evaluating destination arrival and improving termination decisions.