Beyond Success: Assessing Human-Plausible Execution in Vision-and-Language Navigation with HumanPathScore

Vision-and-language navigation (VLN) benchmarks measure whether an agent reaches its goal, how efficiently, and how closely its route follows a reference. These essential measures do not fully characterize execution. A successful trajectory may still intersect occupied space, turn abruptly or oscillate, undo progress, or lack visual evidence for an upcoming action. We introduce HumanPathScore (HPS), a reference-trajectory-free, scene-evidence-driven metric that assesses a single trajectory along four observable diagnostic dimensions of human-plausible execution: Physical Walkability, Motion Naturalness, Behavioral Coherence, and View for Action. Together, these constructs operationalize human-plausible execution by measuring whether a human-sized body could traverse the route, movement remains controlled, progress is preserved, and scene evidence supports action. Each construct and its evidence remain visible beside an aggregate that excludes task success. A common trajectory-evidence formulation handles graph-based and continuous trajectories and supports calibrated observations from both mesh-based and 3D Gaussian Splatting reconstructions. We validate HPS in discrete and continuous navigation using blinded human video judgments and matched metric comparisons. HPS shows the strongest overall alignment with human preference among the compared metrics. A separate multi-benchmark analysis shows that HPS produces method rankings different from those produced by SR, SPL, and nDTW. These results support HPS as a diagnostic complement to task-performance and reference-fidelity metrics: existing metrics assess whether an agent succeeds, whereas HPS directly assesses how the execution unfolds.