- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 3
Read review
Paper summary
The paper finds the execution gap and information gap in VLA navigation, and addresses these two gaps with the proposed PNav, which executes a metric or return program from an external origin register and otherwise leaves control with the frozen policy. It also improves the results on NavSpace, showing the value of the method.
Critical strengths
- No additional training is needed for this result, and the improvement is substantial.
- The method is tested on multiple benchmarks, such as SNav, StreamVLN and LLaVA-Video. The consistent improvements support the effectiveness of external execution.
Critical weaknesses
- PNav mainly works for instructions with explicit distances and angles, or for returning to the starting point. However, it is hard to get accurate structured commands in the real world. The proposed method is hard to generalize to other scenarios.
- The paper does not provide enough comparison with other VLM/VLA based approaches. For example, the VLM can also generate structured motion commands and then let the same deterministic controller execute them. In this way, it can still keep the language and visual understanding ability of the VLM while having the benefit of precise execution. Without this comparison, it is not clear why using a rule-based parser is a better choice, especially for more complex navigation instructions.
- In Table 4, PNav achieves almost the same Precise navigation performance across SNav-7B (54.2%), StreamVLN (54.2%), and LLaVA-Video (53.2%), even though their original performance is quite different. Does this mean that the improvement is mostly coming from the deterministic executor rather than the VLM itself? It would be better to further investigate how much the underlying VLM actually contributes to the final performance.
Questions for authors
- Why does the paper use a rule-based parser instead of letting the VLM generate structured motion commands for the deterministic executor? Have the authors considered comparing these two approaches?
- How well can PNav handle more complex navigation instructions that require both visual understanding and precise execution, especially when obstacles block the planned path?
Thank you for the positive assessment and helpful suggestions
Thank you for the constructive review. The suggested comparison between a rule parser and a VLM generated structured program was especially useful. We have now performed this comparison while holding the executor fixed. The results also clarify why different backbones approach similar Precise performance after adding PNav.
1. Rule parser versus VLM generated program.
We evaluated all 231 NavSpace instructions independently labeled by humans as fully executable.
Frozen VLA without executor: 13.4% SR.
Rule parser: 85.3% valid, 76.2% exact, 54.5% SR.
Frozen VLM program generator: 94.8% valid, 68.4% exact, 50.6% SR.
Human gold program from instruction only: 100% exact, 71.0% SR.
The VLM can therefore generate a structured program and use the same executor successfully. The fixed parser is one implementation of the program interface rather than a requirement of PNav. The VLM has higher coverage, while the rule parser has higher program accuracy. The human program reaches 71.0%, showing that better program extraction can further improve the system.
There are 143 instructions where the rule parser and VLM produce the same correct program. Their trajectories are identical in 141 cases, and success is 101/143 versus 100/143. Once the correct program is supplied, the source of that program has almost no effect on the resulting behavior.
2. Why use a conservative rule parser?
We evaluated an independent set of 300 instructions written by people who did not know the parser grammar.
On 100 fully executable instructions, exact program accuracy is 74% for the rule parser and 67% for the VLM. On 100 mixed instructions, the VLM recovers the executable prefix slightly more often, 64% versus 59%, but incorrectly treats the complete instruction as executable in 18% of cases, compared with 7% for the rule parser. On 100 grounding required instructions, erroneous complete takeover occurs in 10% versus 2%.
We therefore do not argue that rule parsing is inherently better. Its current advantage is conservative and auditable delegation. A learned program generator could replace it if its routing precision improves.
3. Does the executor explain most of the gain?
For fully executable instructions, yes. We believe this supports rather than weakens the execution gap hypothesis.
Across all 1,228 NavSpace instructions, two independent annotators labeled semantic executability using only the instruction. Raw agreement was 90.8% and Cohen's κ was 0.82.
Fully executable E: 13.4% to 54.5%, a gain of 41.1 points.
Mixed M: 28.3% to 26.1%.
Grounding required G: 31.6% to 32.5%.
Thus the large gain appears exactly where language already determines the motion program. This also explains why SNav, StreamVLN, and LLaVA Video all approach about 53% to 54% on Precise after PNav, despite very different frozen performance. Once the program is specified, persistent execution becomes the dominant missing capability. The visual model remains necessary for the much larger set of mixed and grounding tasks.
The existing OctoNav intervention supports the same interpretation. Coordinate parsing alone leaves SR at 5.7%, while parsing plus execution raises it to 19.7%. A structured representation alone is not sufficient.
4. Complex instructions that require both precise motion and visual reasoning.
We implemented residual handoff. The executor completes only the metric prefix, then returns control to the frozen visual policy with the remaining instruction and preserved RGB history.
On 100 independently written mixed instructions:
Frozen: 29%.
Recognition routing: 25%.
Semantic routing: 30%.
Residual handoff: 36%.
Human program and handoff: 41%.
We also evaluated 90 eligible Precise episodes. Frozen succeeds on 10/90, or 11.1%, while residual handoff succeeds on 26/90, or 28.9%. The exact McNemar test gives p = 0.00372. Among the 83 cases with a nonempty residual, success improves from 10.8% to 22.9%.
This shows that precise execution and visual reasoning can be composed within one episode rather than competing for control over the whole trajectory.
5. Obstacles.
We agree that PNav is not a general obstacle planner. It does consume collision feedback, so blocked forward actions are not integrated as successful motion. In the 90 episode experiment, 27 trajectories encounter at least one blocked action and nine encounter two consecutive blocked actions. We will make clear that this is feedback corrected execution, not visual route replanning.
We especially appreciate the suggestion to compare VLM generated programs with the fixed parser under the same executor. This experiment helped us separate program extraction from execution much more cleanly, and we believe it strengthens both the analysis and the scope of the paper.