- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposed VaSeR, which is a method that equips a VLA with an action-conditioned value token. The value token is integrated into the policy's backbone, allowing for a single forward pass computation. The role of the value token is to score demonstrations produced by the VLA. Importantly, the value is trained from demonstration data alone. The demonstration actions receive a discounted time-to-go target, while the negatives receive a lower 'pseudo-label' based on their distance to the demonstration. At inference, the value selects the best sample, and is able to trigger a retreat when its values drop / stagnate. VaSeR is evaluated on ten RLBench tasks, showing improvement over pi_0.5 and a diffusion-only HybridVLA. The authors also demonstrate the performance improvements on two real-robot tasks.
Critical strengths
-
The idea is quite simple and cheap, requiring no additional models or data. This is quite a neat idea, as the lack of failure data is a major bottleneck for robot training. The approach is also shown to be feasible in both diffusion-based and flow-matching VLAs.
-
The writing is clear and easy-to-follow.
-
The paper contains useful ablations regarding the value objective (e.g., negatives / gradient detachment).
Critical weaknesses
-
I notice that the gains seem to be concentrated around the tasks where the baseline is unusually weak. In table 1, pi_0.5 succeeds on Close Box in only 4/60 episodes, while HybridVLA reaches 70% on the same task. II believe this single task accounts for a large portion of the percentage points gained on pi_0.5. The main pi_0.5 checkpoint also seems much weaker than the earlier four-task models in Appendix B (. Could you comment on why the main baseline underperforms, and on whether VaSeR still helps a well-tuned actor?
-
I understand you have removed the HybridVLA baseline's autoregressive branch and gated fusion, but it is arguably what defines HybridVLA. Would it be possible to evaluate the original model under your protocol?
-
I believe it would be quite informative and important to add a few baselines. For instance, for selection, these could be RoboMonkey, V-GPS or IBC-style energy. For failure detection, they could be SAFE, Sentinel or FAIL-Detect, which you already cite. Could the authors please address why none of the relevant approaches have been baselined against your method?
-
The paper does not seem to clarify where the 'recovery' gain comes from. As far as I can see, it could be the timing of the alarm, the physical retreat, or simply from drawing fresh samples? I could suggest (i) retreat triggered at random times with a matched alarm rate, and (ii) withholding the action on alarm and resampling without moving.
-
I would also appreciate some intuition for why a value trained only at demonstration observations should drop in unfamiliar states, given that the negatives perturb actions rather than states. I think this would be important to clarify.
Questions for authors
Please see "Weaknesses".