- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper proposes LVP (Likelihood-Based Visual Prompting), a training-free test-time scaling framework for VLA models. Candidate actions are projected onto the observation image as translation trajectories and rotation axes, then ranked by the expected utility of multiple-choice VQA answers computed from conditional token probabilities of the language-modeling head. The authors also propose a visual token pruning method that combines attention from the action generation stage with feature differences before and after visual overlay, reducing verification latency. The framework is evaluated on both autoregressive and flow-matching policies.
Critical strengths
-
The method turns action candidates into visual prompts and scores them with the model's existing language-modeling capability, without training a dedicated verifier. This converts action evaluation into a visual-semantic judgment and supports both autoregressive and flow-matching action decoders within the same framework, which gives it broad applicability.
-
The experiments are solid: evaluation covers multiple backbones, benchmarks, and real-robot tests, with a range of ablations.
Critical weaknesses
-
The visual action representation omits gripper open/close. Sec. 3.2 and Appendix A.2 define candidate actions with translation, rotation, and gripper state, yet the visual prompt shows only the translation trajectory and goal orientation. Candidates with the same end-effector trajectory and orientation but different gripper commands thus receive identical visual representations, even though gripper timing directly affects grasping and releasing success. The paper should clarify which action dimensions the score covers and how gripper differences participate in candidate selection.
-
No direct quantitative validation links candidate scores to execution quality. While Tab. 5 reports final success rates, it does not verify whether higher-scoring candidates succeed more often than lower-scoring ones within the same pool. This gap relates to the performance drop as the candidate pool grows: in Tab. 10a, GR00T's full-token average success rate on WidowX drops from 60.0 at N=4 to 56.0 at N=16, and the pruned version drops from 58.4 to 54.0, below the base policy's 55.0. Appendix A.7 attributes this drop to verification noise without identifying its cause. A candidate-level analysis would clarify in which states the score is reliable and why larger pools select actions with lower execution quality.
Questions for authors
Is the gripper open/close command included in the verifier input? When candidates differ in grasp or release timing, how do the visual prompt and scoring function distinguish them?
On a small set of representative simulation states, could you execute the candidate action chunks from the same pool with a fixed follow-up policy and compare LVP scores with downstream success? Showing the gap between LVP's selection and the oracle selection in each pool would help explain why performance drops at large N.
2 comments
Gentle follow-up regarding our rebuttal
Dear Reviewer wUmF,
Thank you again for your time and constructive feedback. Please let us know if our responses and additional experiments have addressed your concerns, or if you have any further questions. We look forward to any further feedback you might have.
Best regards, The Authors
We thank you for the constructive feedback and insightful suggestions. The reviewer highlights the strength of our training-free, decoder-agnostic paradigm, solid cross-backbone/benchmark evaluations, and real-robot validations. Below, we address each weakness and question in detail.
Reply: Thank you for pointing out this important detail. We do not use the gripper open/close command as the verifier input, and mainly focus on the geometric alignment of action candidates. We defined action candidates with translation, rotation, and gripper states in Sec. 3.2 and Appendix A.2, because the VLA policies jointly generate these candidates. We clarify this detail in Appendix A.2 of the revised manuscript.
Reply: We appreciate this valuable experimental suggestion. To directly validate whether higher candidate scores correlate with downstream execution success and to analyze the performance gap at large pool sizes (N=16), we conduct an empirical candidate-level execution experiment.
Experimental Setup
Experimental Results
Analysis