- Rating
- 4
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
The core idea is that video generation models provide a better prioir for value estimation than VLMs because they are trained to predict future states instead of just semantic understanding of the current state. The authors propose training a video generation model into a value model (ViVa) to test this. They then compare with VLM baselines (both using ViVa as a stand-alone model and as supervision for RECAP on 3 tasks) and ablate over several of the design decision in ViVa.
Critical strengths
- Clear/easy to read.
- The core idea/intuition makes sense and it is clear how that intuition is embodied in the proposed method.
- The design/choice of experiments does a mostly solid job to validate their method.
Overall, this paper was a fun and interesting read: the reviewer is quite positive on acceptance.
Critical weaknesses
None of these are particularly "critical" weaknesses but the reviewer will mention them regardless:
-
Using a video understanding model in the VLM (instead of conditioning on static state) is an interesting (and missing) baseline---which may already solve the static discriminative representations problem. Overall, it seems best motivated to condition on video regardless of using a VLM or video diffusion backbone.
-
The proposed MDR and ErrDr metrics are very sensitive as they have no threshold. It would be useful to see versions of the main results with various thresholds (>1%, >5%, >10%, etc. percent change). This would provide stronger evidence for the claim that ViVa provides clearer / more finegrained supervision than baselines (a more quantified version of the claim in figure 4).
-
The ablation studies in 4.5 suffer from being on the shirt environment, which does not seem to provide very finegrained signal (especially for error detection). In several cases, it seems like a setting/method either gets a near perfect score or fails completely. This makes the results somewhat difficult to interpret and leaves some question about how much true improvement comes from each design choice. Especially when there seem to be cases where certain choices trade-off between milestone and error recognition.
-
The method seems quite sensitive to the settings of K and λ (although this could also be an artifact of the shirt environment). This also leaves some questions about how "well-tuned" ViVa is compared to the Pi_{0.6} baseline.
Minor:
- Equation 2 being a reward that is minimized (by an any agent using ViVa) is somewhat unintuitive. The reviewer would prefer calling this a cost or flipping the sign in equation 2 and training ViVa to predict the negative of the reward. But this is a minor presentation detail up to the authors.
Questions for authors
- How many milestone / error events exist (on average) per epidode for each environment?
- Will the code, data, and model weights be released to reproduce the core (non-real robot) results?
3 comments
Second Response to Reviewer 1yPn
W4: Sensitivity to K and λ.
A4: We thank the reviewer for raising this concern. We clarify our baseline configuration, hyperparameter selection, and cross-task generalization.
- Fair Baseline Configuration: The baseline follows its default training configuration without additional task-specific tuning. It also outperforms ViVa on Box Packaging (Table 1), demonstrating that the baseline remains competitive.
- Task-Agnostic Hyperparameter Selection: ViVa is jointly trained on all three tasks using shared and , without task-specific tuning. The choice of was informed by the advantage horizon used in .
- Generalization Across Temporal Scales: Despite differences in temporal scales, the shared configuration performs well across all three tasks. ViVa also generalizes to the unseen Pants Folding task without adjusting either hyperparameter.
W5: Reward formulation issue.
A5: We thank the reviewer for this professional suggestion. We agree that Eq. (2) is more appropriately described as a cost rather than a reward. We will revise the terminology accordingly, while keeping the implementation and experimental results unchanged.
Q1: Milestone and Error Events.
A6: We thank the reviewer for this question. For evaluating milestone and error sensitivity, we manually annotate 50 held-out demonstration episodes per task.
-
Event Counts. The event counts used for metric computation are as follows:
Task Total Milestones Total Errors Avg. Milestones Avg. Errors Shirt Folding 405 4 8.10 0.08 Box Packaging 733 330 14.66 6.60 Paper Roll Organization 739 9 14.78 0.18 -
Task Characteristics:
- Box Packaging: Requires precise manipulation of rigid objects, where discrete errors are easier to identify and localize.
- Shirt Folding & Paper Roll Organization: Involve deformable materials, where continuous deformation and adjustments make precise error localization challenging, leaving fewer reliably annotated error events.
Q2: Will the code, data, and model weights be released to reproduce the core non-real-robot results.
A7: We thank the reviewer for this question. As stated in our Reproducibility Statement, we will publicly release the code, model weights, and evaluation datasets necessary to reproduce our core non-real-robot results upon acceptance.
We have incorporated all corresponding revisions into the manuscript, with changes highlighted in blue. Thanks again for helping us improve the paper, and we hope the response can resolve these concerns!
Reviewer response
As stated in our Reproducibility Statement.
I forgot that this section was after the conclusion (and not in the appendix), sorry.
I have read the rebuttal and remain positive on acceptance.
First Response to Reviewer 1yPn
We sincerely thank Reviewer 1yPn for their encouraging assessment of our work and valuable suggestions. Below, we provide clarifications and additional evidence addressing the remaining concerns.
A1: We thank the reviewer for the suggestion and provide additional experiments with a video-conditioned VLM baseline.
A2: We thank the reviewer for this insightful suggestion. We additionally evaluate MDR and ErrDR under different detection thresholds.
Setting: We apply thresholds of 1%, 5%, and 10% to the event response score (ERS), compared with the original threshold of 0. All other settings remain unchanged.
Results:
Table 1: MDR vs. Threshold
Table 2: ErrDR vs. Threshold
Analysis:
A3: We thank the reviewer for this insightful comment. To address this concern, we extend the ablation studies to Box Packaging and Paper Roll Organization.
Setting: We keep all other experimental settings unchanged. The model takes three-view images and the current proprioceptive state as inputs.
Results:
Table 1: Paper Roll Organization
Table 2: Box Packaging
Analysis: