- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes CTPNav, which evaluates the stability of navigation decisions by observing how candidate actions respond to controlled interventions on action competition, graph geometry, and instruction grounding. These responses are used to regulate graph reasoning, adjust map construction, provide an auxiliary training target, and slightly refine action scores at inference. Experiments on R2R-CE and RxR-CE show improvements over the underlying navigation baselines.
Critical strengths
The paper presents an interesting attempt to look beyond a policy’s raw confidence by examining how its decisions respond to controlled internal changes. The method is integrated into graph reasoning, mapping, training, and inference, and is evaluated on two standard VLN benchmarks with detailed ablations, paired comparisons, and runtime analysis.
Critical weaknesses
My main concern is whether the proposed interventions actually measure decision stability in a meaningful way.
A central challenge in VLN is to align language with visual observations and scene geometry, yet the method deliberately weakens one of these sources and treats changes in the resulting action scores as instability. I am not convinced that this interpretation is always valid. For example, if the instruction requires the agent to turn right, masking the most relevant instruction token may cause it to go straight, but this could simply show that the original decision was correctly grounded in the instruction rather than that it was fragile. Conversely, an intervention may accidentally change an incorrect prediction into the correct one, which also does not provide clear evidence of reliability. Since these interventions can remove information genuinely needed for the action, the meaning of the measured response dispersion remains unclear.
In addition, the evaluation mainly shows improvements on standard VLN benchmarks rather than robustness under meaningful perturbations, while the gain from online verification itself is small and not statistically significant.
Questions for authors
-
Could the authors explain more clearly why sensitivity to the proposed interventions should indicate an unreliable decision? In particular, how does the method distinguish brittle dependence from valid dependence on instruction or geometric evidence?
-
Why does the competition intervention equalize only the top two actions rather than the top-k actions or all candidates in the shortlist?
-
Why does the grounding intervention mask only the single most aligned token, and how sensitive is the method to the number and selection of masked tokens?