- Rating
- 3
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
This paper proposes DIVER, an importance-weighted verifier training framework based on decision-critical states to improve the effectiveness of Vision-Language-Action (VLA) models during test-time scaling (TTS). The authors observe that not all states along a trajectory are equally important during execution. Notably, they make the clever observation that at routine states (such as free-space motion), candidate actions exhibit lower variance in their action-expert hidden representations, whereas at decision-critical states, this variance is significantly higher. Leveraging this variance for importance weighting, DIVER encourages the verifier to focus more on decision-critical states during training. Extensive experiments demonstrate that DIVER achieves superior performance, and ablation studies clearly validate the effectiveness of the proposed importance weighting mechanism.
Critical strengths
1.The motivation of this paper is clear, and the proposed method is simple yet effective. The experimental evaluation is comprehensive, and the ablation studies clearly demonstrate the contribution of each component.
2.The idea of leveraging internal action-expert representations to estimate state criticality and discriminate candidate actions is clever and well-motivated.
Critical weaknesses
1.Trajectory-level supervision appears overly coarse. For instance, in a multi-stage manipulation task, if the robot successfully completes the first stage but fails during the second stage, the current training paradigm assigns a terminal label of (failure) to the entire trajectory. This penalizes the successful transitions in the first stage and leads to inefficient data utilization and noisy supervision.
2.While the distinction in action-representation variance between routine states and decision-critical states is intuitive for in-distribution scenarios, it remains unclear whether this property still holds in out-of-distribution (OOD) scenarios. Intuitively, under OOD conditions, the variance of sampled actions might be generally elevated across the trajectory due to epistemic uncertainty, even at routine states. However, the paper lacks an analysis of this aspect.
Questions for authors
1.Have the authors considered adopting more fine-grained supervision or training schemes—for example, leveraging expert demonstrations [1, 2] or incorporating sub-task decomposition [3]?
2.The intuition behind decision-critical states shares conceptual similarities with [4]. Since candidate actions at routine states exhibit low variance and barely affect task success, have the authors explored identifying routine states on the fly and adaptively disabling TTS during those phases to accelerate inference?
3.Does the distinction in action-representation variance between routine and decision-critical states still hold in OOD scenarios (where action variance might be generally elevated)? Could the authors provide some analysis or discussion on this?
Reference
[1]RoboMonkey: Scaling Test-Time Sampling and Verification for Vision-Language-Action Models
[2]V-GPS: Value-Guided Policy Search / RL with Value-Guided Verification
[3]ReinboT: Amplifying Robot Visual-Language Manipulation with Reinforcement Learning
[4]SkiP: When to Skip and When to Refine for Efficient Robot Manipulation
Thank you for the detailed responses and additional clarifications. They have fully addressed my concerns, and I am happy to raise my score.
I have a few follow-up questions.
Regarding W1 and Q1, is there a specific reason why only the N=16 results are reported for LIBERO, while the RoboCasa results are averaged across all settings? Looking at the full results in the paper, the gains still do not seem consistent. It also looks as though the higher aggregated performance comes mainly from a few tasks that flip between zero and non-zero success rates. Could you identify, for each task and each N, the cases where DiVeR does not improve performance, and explain why it fails in those cases? It would also help to see how the aggregated performance changes when these extreme cases are excluded.
Data construction takes about 5 additional hours, which adds compute time and GPU usage on top of inference. MG-Select, in contrast, only uses the GPU at inference time. Are there ways to reduce this data construction cost? For example, could the data be constructed once and shared across multiple tasks, so the cost is amortized?