acceptodds
Under review as a conference paper at ICLR 2027

ViVa: A Video-Generative Value Model for Robot Reinforcement Learning

Abstract

Vision-language-action (VLA) models have advanced robot manipulation through large-scale pretraining, but real-world deployment remains challenging due to partial observability and delayed feedback. Reinforcement learning addresses this via value functions, which assess task progress and guide policy improvement. However, existing value models built on vision-language models (VLMs) struggle to capture temporal dynamics and physical interactions, undermining reliable value estimation in long-horizon tasks. In this paper, we propose ViVa, a video-generative value model that repurposes a pretrained video generator to jointly predict future proprioception and a scalar value. By grounding value estimation in anticipated embodiment dynamics, ViVa leverages spatiotemporal priors to intrinsically couple value with foresight beyond static snapshots. ViVa achieves state-of-the-art results in metric-based evaluation across three tasks, producing reliable value signals that accurately track task progress and detect execution errors. Integrated into RECAP, it achieves an average success rate of 80%, highlighting the promise of video-generative models for value estimation.

Primary area:
applications to robotics, autonomy, planning
Keywords:
Video-Generative Value Model, Reinforcement Learning, Robot Manipulation

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_1yPnOpenReview ↗
Rating
4
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

The core idea is that video generation models provide a better prioir for value estimation than VLMs because they are trained to predict future states instead of just semantic understanding of the current state. The authors propose training a video generation model into a value model (ViVa) to test this. They then compare with VLM baselines (both using ViVa as a stand-alone model and as supervision for RECAP on 3 tasks) and ablate over several of the design decision in ViVa.

Critical strengths
  • Clear/easy to read.
  • The core idea/intuition makes sense and it is clear how that intuition is embodied in the proposed method.
  • The design/choice of experiments does a mostly solid job to validate their method.
    Overall, this paper was a fun and interesting read: the reviewer is quite positive on acceptance.
Critical weaknesses

None of these are particularly "critical" weaknesses but the reviewer will mention them regardless:

  • Using a video understanding model in the VLM (instead of conditioning on static state) is an interesting (and missing) baseline---which may already solve the static discriminative representations problem. Overall, it seems best motivated to condition on video regardless of using a VLM or video diffusion backbone.

  • The proposed MDR and ErrDr metrics are very sensitive as they have no threshold. It would be useful to see versions of the main results with various thresholds (>1%, >5%, >10%, etc. percent change). This would provide stronger evidence for the claim that ViVa provides clearer / more finegrained supervision than baselines (a more quantified version of the claim in figure 4).

  • The ablation studies in 4.5 suffer from being on the shirt environment, which does not seem to provide very finegrained signal (especially for error detection). In several cases, it seems like a setting/method either gets a near perfect score or fails completely. This makes the results somewhat difficult to interpret and leaves some question about how much true improvement comes from each design choice. Especially when there seem to be cases where certain choices trade-off between milestone and error recognition.

  • The method seems quite sensitive to the settings of K and λ (although this could also be an artifact of the shirt environment). This also leaves some questions about how "well-tuned" ViVa is compared to the Pi_{0.6} baseline.

Minor:

  • Equation 2 being a reward that is minimized (by an any agent using ViVa) is somewhat unintuitive. The reviewer would prefer calling this a cost or flipping the sign in equation 2 and training ViVa to predict the negative of the reward. But this is a minor presentation detail up to the authors.
Questions for authors
  • How many milestone / error events exist (on average) per epidode for each environment?
  • Will the code, data, and model weights be released to reproduce the core (non-real robot) results?
3 comments
AuthorsOpenReview ↗

First Response to Reviewer 1yPn

We sincerely thank Reviewer 1yPn for their encouraging assessment of our work and valuable suggestions. Below, we provide clarifications and additional evidence addressing the remaining concerns.


W1: Video-conditioned VLM baseline.

A1: We thank the reviewer for the suggestion and provide additional experiments with a video-conditioned VLM baseline.

  • Setting: We extend the original VLM with six frames at offsets −50, −40, −30, −20, −10, and 0, each horizontally concatenating three camera views. Other settings remain unchanged.
  • Results: | Task | Model | MS ↑ | ES ↑ | MDR ↑ | ErrDR ↑ | |---|---|---:|---:|---:|---:| | Box Packaging | VLM | 0.0131 | 0.0062 | 0.7653 | 0.5061 | | | Video-Conditioned VLM | 0.0141 | −0.0012 | 0.8499 | 0.4000 | | | ViVa | 0.0341 | 0.0263 | 0.7326 | 0.6121 | | Shirt Folding | VLM | −0.2664 | −0.2985 | 0.4074 | 0.5000 | | | Video-Conditioned VLM | 0.2761 | −0.1162 | 0.9259 | 0.5000 | | | ViVa | 0.0954 | 0.0120 | 0.9901 | 1.0000 | | Paper Roll Organization | VLM | 0.0225 | 0.0280 | 0.7037 | 0.6667 | | | Video-Conditioned VLM | 0.0174 | −0.0108 | 0.8417 | 0.2222 | | | ViVa | 0.0386 | 0.0552 | 0.7618 | 0.7778 |
  • Analysis:
    1. VLM vs. Video-Conditioned VLM: Adding temporal context improves MDR across all three tasks, but generally reduces ErrDR, with negative ES throughout.
    2. Video-Conditioned VLM vs. ViVa: ViVa achieves higher ES and ErrDR across all three tasks, suggesting that temporal conditioning alone is insufficient to match ViVa's error sensitivity.

W2: Threshold sensitivity of MDR and ErrDR.

A2: We thank the reviewer for this insightful suggestion. We additionally evaluate MDR and ErrDR under different detection thresholds.

  • Setting: We apply thresholds of 1%, 5%, and 10% to the event response score (ERS), compared with the original threshold of 0. All other settings remain unchanged.

  • Results:

    Table 1: MDR vs. Threshold

    TaskMethod>0>1%>5%>10%
    BoxVLM0.7650.4730.0680.014
    ViVa0.7330.6470.3720.228
    ShirtVLM0.4070.4000.3850.361
    ViVa0.9900.9800.7410.346
    PaperVLM0.7040.6060.1850.062
    ViVa0.7620.6250.3380.153

    Table 2: ErrDR vs. Threshold

    TaskMethod>0>1%>5%>10%
    BoxVLM0.5060.2520.0580.015
    ViVa0.6120.5580.3760.230
    ShirtVLM0.5000.2500.0000.000
    ViVa1.0000.5000.2500.250
    PaperVLM0.6670.4440.3330.111
    ViVa0.7780.5560.3330.111
  • Analysis:

    • Threshold Sensitivity: MDR and ErrDR decrease with stricter thresholds, indicating that some event responses have small magnitudes.
    • ViVa's Robustness: ViVa better preserves event detection performance under stricter thresholds, suggesting stronger sensitivity to substantial task events.

W3: Limited scope of ablation studies.

A3: We thank the reviewer for this insightful comment. To address this concern, we extend the ablation studies to Box Packaging and Paper Roll Organization.

  • Architecture Design Ablation:
    • Setting: We keep all other experimental settings unchanged. The model takes three-view images and the current proprioceptive state as inputs.

    • Results:

      Table 1: Paper Roll Organization

      ValueFuture ImageFuture Proprio.MS ↑ES ↑MDR ↑ErrDR ↑
      ✓✗✗0.019−0.0430.6440.444
      ✓✓✗0.0010.0070.5620.556
      ✓✓✓0.041−0.0050.7050.444
      ✓✗✓0.0390.0550.7620.778

      Table 2: Box Packaging

      ValueFuture ImageFuture Proprio.MS ↑ES ↑MDR ↑ErrDR ↑
      ✓✗✗0.0470.0080.7780.527
      ✓✓✗0.0600.0110.7830.548
      ✓✓✓0.0430.0040.7890.452
      ✓✗✓0.0340.0260.7330.612
    • Analysis:

      • Future Proprioception: Consistently improves ES and ErrDR over value-only prediction on both tasks.
      • Future Image Prediction: Adding future-image prediction reduces both metrics compared with future proprioception alone, supporting our lightweight design.
  • Scope of Additional Ablations: Section C.5 of the revised manuscript provides further ablations, including pretraining ablations and sensitivity analyses of KK and λ\lambda across all three tasks (Figures 8 and 9, respectively).

AuthorsOpenReview ↗

Second Response to Reviewer 1yPn


W4: Sensitivity to K and λ.

A4: We thank the reviewer for raising this concern. We clarify our baseline configuration, hyperparameter selection, and cross-task generalization.

  • Fair Baseline Configuration: The π0.6∗\pi^*_{0.6} baseline follows its default training configuration without additional task-specific tuning. It also outperforms ViVa on Box Packaging (Table 1), demonstrating that the baseline remains competitive.
  • Task-Agnostic Hyperparameter Selection: ViVa is jointly trained on all three tasks using shared K=50K=50 and λ=0.5\lambda=0.5, without task-specific tuning. The choice of K=50K=50 was informed by the advantage horizon used in π0.6∗\pi^*_{0.6}.
  • Generalization Across Temporal Scales: Despite differences in temporal scales, the shared configuration performs well across all three tasks. ViVa also generalizes to the unseen Pants Folding task without adjusting either hyperparameter.

W5: Reward formulation issue.

A5: We thank the reviewer for this professional suggestion. We agree that Eq. (2) is more appropriately described as a cost rather than a reward. We will revise the terminology accordingly, while keeping the implementation and experimental results unchanged.


Q1: Milestone and Error Events.

A6: We thank the reviewer for this question. For evaluating milestone and error sensitivity, we manually annotate 50 held-out demonstration episodes per task.

  • Event Counts. The event counts used for metric computation are as follows:

    TaskTotal MilestonesTotal ErrorsAvg. MilestonesAvg. Errors
    Shirt Folding40548.100.08
    Box Packaging73333014.666.60
    Paper Roll Organization739914.780.18
  • Task Characteristics:

    • Box Packaging: Requires precise manipulation of rigid objects, where discrete errors are easier to identify and localize.
    • Shirt Folding & Paper Roll Organization: Involve deformable materials, where continuous deformation and adjustments make precise error localization challenging, leaving fewer reliably annotated error events.

Q2: Will the code, data, and model weights be released to reproduce the core non-real-robot results.

A7: We thank the reviewer for this question. As stated in our Reproducibility Statement, we will publicly release the code, model weights, and evaluation datasets necessary to reproduce our core non-real-robot results upon acceptance.


We have incorporated all corresponding revisions into the manuscript, with changes highlighted in blue. Thanks again for helping us improve the paper, and we hope the response can resolve these concerns!

Reviewer_1yPnOpenReview ↗

Reviewer response

As stated in our Reproducibility Statement.

I forgot that this section was after the conclusion (and not in the appendix), sorry.


I have read the rebuttal and remain positive on acceptance.

Reviewer_ugknOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

The paper builds a value model for real-robot RL from a pretrained video diffusion model. The current camera views and proprioception are fed in as clean latent frames, and the model is trained with flow matching to produce two extra latent frames. Offline, ViVa is compared with GVL, TOPReward and a π*0.6-style VLM value classifier on 50 held-out episodes for each of three bimanual tasks (shirt folding, box packaging, paper roll organization). Online, ViVa replaces the VLM value model in two RECAP rounds that start from a Gigabrain-0 policy, and average real-robot success goes from 63.3% to 80.0% with 10 trials per task. Authors also include ablations on shirt folding analyzing the backbone, initialization, prediction targets, horizon and loss weight.

Critical strengths
  1. The value model is tested inside a real RL loop, not just on offline metrics. Both RECAP variants use the same pipeline and start from the same base policy, on three long-horizon bimanual tasks. RECAP with ViVa doubles the success of the imitation-learning starting point and is ahead of RECAP with the VLM value on all three tasks. Real-robot RL comparisons like this are expensive to run, and this is super impressive.

  2. The evaluation is thorough and informative. With annotated milestones and errors on 50 held-out episodes per task, ViVa catches more execution errors than the VLM classifier on all three tasks, tracks shirt-folding progress much better and holds up better on the unseen pants. That is the property RECAP needs to tell good rollouts from bad ones, so the offline and online results back each other up. One more thing: future proprioception beats future images as the auxiliary target on most of the event metrics, and it is cheaper to train.

Critical weaknesses
  1. The paper is not positioned against close prior work, so the claim of what is new is overstated. Cosmos Policy (ICLR 2026), which is not cited, already fine-tunes a pretrained video diffusion model to write proprioception, future proprioception and a Monte Carlo value into latent frames, and reads the value out by averaging over the latent volume, much as ViVa does. GigaBrain-0.5M*, which the paper cites only as an example VLA, uses Wan2.2 to predict future frames together with a value latent frame for RECAP-style training, and compares that value against a VLM value predictor. So the stated contribution and the framing in the introduction overstate what is new. Once both works get credit, what remains is narrower: a standalone video-generative value model tested inside RECAP on real robots, the finding that future proprioception beats future frames as an auxiliary target, and the event-level evaluation. It still counts for something, but it is a smaller step than the paper suggests.

  2. The real-robot gain over the VLM-value baseline, and what causes it, need firmer evidence. With 10 trials per task, the headline gain is not statistically significant. Figure 5 also needs explaining. The text says both variants start from the same Gigabrain-0 policy, yet their rollout-0 bars differ, and the box rollout-0 bar does not match Gigabrain-0 in Table 5. Offline, the VLM baseline differs from ViVa in more than the backbone: it is a return classifier, no auxiliary target is mentioned for it, and it is a smaller model than the Wan2.2 backbone. So Table 6 does not isolate the backbone, and Table 7 only compares video pretraining with random initialization. The paper does not show that the online gain holds up at this sample size, or that video pretraining, rather than these other differences, is what helps. That would take more trials, a definition of rollout-0 and one run where only the backbone changes. Together with the first weakness, this is why I lean towards rejection.

Questions for authors
  1. (Clarification) What exactly is rollout-0 in Figure 5, the untouched Gigabrain-0 policy or a policy already trained with each value model? Were the trials for the two variants run from matched initial conditions, interleaved, and scored blind? Can you add trials or report confidence intervals?

  2. (Clarification) How does ViVa differ from Cosmos Policy and GigaBrain-0.5M*, beyond dropping the future-image frames? Another flag from literature survey: an earlier paper, "ViVa: Video-Trained Value Functions for Guiding Online RL from Diverse Data" (2025), already uses the name and also learns value functions from video. It should be cited, and a different name would avoid confusion.

  3. (Suggestion) Could you train the VLM and Wan2.2 backbones with the same head, loss and future-proprioception target, and add a same-architecture model initialized from image or VLM pretraining? The text says Table 6 compares backbones under the same value-modeling formulation, but its Qwen row looks like the π*0.6-style classifier, and its Wan2.2 row matches neither default ViVa nor value-only in Table 8.

  4. (Clarification) How many error events, and how many episodes with errors, are scored per task, and is ErrDR averaged per event or per episode? On shirt folding, ErrDR only ever takes three values across the tables, which would fit about two events, while the Figure 7 percentages suggest more. Please also report the spread over training seeds.

  5. (Clarification and suggestion) Please define Norm, w and ε in the ERS metric, and say whether the metrics use the smoothed curves of Figure 4. ERS is a relative change, so a method whose values sit lower gets a larger ERS for the same absolute change; does the ordering survive with absolute changes? A baseline that just predicts normalized elapsed time, plus ERS measured at random non-event frames, would show how much the event metrics reflect event sensitivity rather than a rising curve.

  6. (Clarification) What does the VLM baseline see, how many views and whether proprioception is included? Was it trained on your return or on π0.6's return, and was it co-trained with web data as in π0.6? Which VLMs did you use for GVL and TOPReward?

  7. (Clarification) For RECAP, what advantage horizon and threshold did you use, and were human corrections involved? Your return behaves like a cost, higher for worse states and for failure, while RECAP's advantage assumes higher is better, so what advantage formula and sign convention did you use? Round 1 collects 150 successful paper-roll episodes while the initial policy succeeds 20% of the time; how were these gathered, and did the two variants share round-1 data?

  8. (Clarification) Your return divides by each episode's own length, so for a success the label says nothing about speed, unlike π*0.6's return; why this choice, and how are failed episodes ended? With one denoising step at inference, how much does V vary across noise samples, and would a plain regression head on the same backbone do as well? Please also say how the Table 11 latencies were measured, including hardware, batch size, number of views, and whether VAE and T5 encoding are included.

6 comments
AuthorsOpenReview ↗

First Response to Reviewer ugkn

We sincerely thank Reviewer ugkn for the constructive feedback and apologize for any confusion caused by insufficient experimental details. Below, we provide clarifications and additional evidence addressing the remaining concerns.


W1 & Q2: Limited novelty compared with prior work.

A1: We thank the reviewer and will add the missing citations. We respectfully emphasize that ViVa goes beyond simplifying prior work. Our key contributions are:

  • Decoupled Value Modeling: Unlike Cosmos Policy and GigaBrain-0.5M*, which incorporate value signals into policy learning, ViVa investigates what makes video representations suitable for robotic value estimation, analyzing discriminative VLM limitations and video-generative priors.
  • Unified Progress-Aware Reward: Unlike Cosmos's terminal returns and GigaBrain's step counts, ViVa combines normalized progress with explicit success/failure outcomes, decoupling progress from absolute episode duration.
  • Error-Sensitive Representation: Our experiments identify future proprioception as key to error-sensitive value estimation, enabling a simple, effective, and lightweight prediction design without future-image generation.
  • Value Estimation Benchmark: We introduce a dedicated benchmark with four metrics (MS/ES/MDR/ErrDR) to assess progress tracking and error sensitivity, validated through real-world policy improvements.

Regarding the earlier ViVa (Dashora et al., 2025): Despite the shared name, the methodologies differ fundamentally. Dashora et al. use ICVF for reward shaping, whereas our ViVa leverages a video-generative backbone to jointly predict future proprioception and scalar values for RL.


W2&Q1: Limited statistical significance and inconsistent rollout-0 results.

A2: Thanks for the comment. We clarify the experimental settings and address the statistical concerns below.

  • Clarification of Figure 5 and Table 5. The two results represent different training settings:
    • Figure 5 (Rollout-0): Both RECAP variants start from the same untouched GigaBrain-0 checkpoint but are trained with demonstrations and their respective VLM-based or ViVa-based value models, following iteration-0 in π0.6∗\pi^*_{0.6}. This explains why their rollout-0 results differ.
    • Table 5 (GigaBrain-0 SFT): Also initialized from the same untouched GigaBrain-0 checkpoint, but trained via SFT on demonstrations and successful trajectories from both rollout rounds, using the same collected data pool as RECAP.
  • Additional Real-Robot Evaluation.
    • Setting: We use the same VLM and ViVa checkpoints as in Rollout 2 of Table 5, conducting 20 additional trials per task, for a total of 30 including the original 10. Both policies are tested under matched initial object configurations, with interleaved trials and success scoring blinded to policy identity.
    • Results:
      ModelShirt FoldingBox PackagingPaper Roll OrganizationOverall
      RECAP (VLM)20/30 (66.7%)15/30 (50.0%)20/30 (66.7%)55/90 (61.1%)
      RECAP (ViVa)28/30 (93.3%)22/30 (73.3%)23/30 (76.7%)73/90 (81.1%)
    • Analysis:
      1. Real-Robot Success Rate: ViVa consistently outperforms VLM across all three tasks, improving the overall success rate from 61.1% to 81.1%, a 20-percentage-point absolute gain.
      2. Statistical Reliability: The improvement is statistically significant (two-sided Fisher's exact test, p = 0.0049), with a 95% CI of [6.8%, 32.3%] for the absolute success-rate difference.

AuthorsOpenReview ↗

Second Response to Reviewer ugkn


W2&Q3: Clarification of Experimental Settings and Controlled Backbone Comparison.

A3: We thank the reviewer for the suggestion. We clarify the experimental settings and provide an additional controlled comparison below.

  • Clarification of Tables 6 and 8:
    1. Table 6 Setting (Image-Only Input): To ensure a consistent comparison, we align the input-output settings with the original VLM baseline. All backbones use only three camera views to predict scalar values without future proprioception. Qwen2.5-VL adopts the π0.6∗\pi^*_{0.6}-style classifier.
    2. Table 8 Setting (Images + Current Proprioceptive State): Table 8 evaluates different prediction targets using both camera views and current proprioceptive states as input. Therefore, its Wan2.2 results differ from those in Table 6 due to differences in input and prediction settings.
  • Controlled Backbone Comparison:
    • Setting: We retain the pretrained VLM backbone and adopt the same Wan2.2 output-head architecture. Both models use aligned inputs and jointly predict value and future proprioception with identical flow-matching objectives and loss weights.
    • Results:
      TaskModelMS ↑ES ↑MDR ↑ErrDR ↑
      Box PackagingVLM + Flow Matching0.03720.00250.78580.5424
      ViVa0.03410.02630.73260.6121
      Shirt FoldingVLM + Flow Matching0.0431−0.01490.84200.5000
      ViVa0.09540.01200.99011.0000
      Paper Roll OrganizationVLM + Flow Matching0.0410−0.01360.73210.5556
      ViVa0.03860.05520.76180.7778
    • Analysis:
      1. Progress Tracking: With the same flow-matching objective, the VLM achieves competitive MS and MDR.
      2. Error Sensitivity: ViVa consistently outperforms the VLM in ES and ErrDR across all three tasks, indicating benefits beyond the shared training objective.
  • Backbone Size & Pretraining:
    • Backbone Size: Table 6 shows that CogVideoX (2B) outperforms the larger Qwen2.5-VL (3B) under matched input-output settings, suggesting that model size alone cannot explain the gains.
    • Pretraining: To our knowledge, no compatible image-only pretrained Wan2.2 checkpoint is publicly available for a strictly matched-architecture comparison. Nevertheless, our matched-objective results provide further evidence supporting the benefits of video-generative priors.

Q4: Error Event Statistics and Aggregation.

A4: We thank the reviewer for these questions. We clarify the evaluation protocol and provide additional results below.

  • Event Statistics & Averaging: We evaluate 50 held-out demonstration episodes per task, with 4/330/9 error events across 4/50/7 episodes containing errors for Shirt Folding, Box Packaging, and Paper Roll Organization, respectively. ErrDR is computed per event, not per episode, as defined in Eq. (11).
  • Figure 7 Clarification: Figure 7 qualitatively illustrates execution errors, while ErrDR requires precisely localized event timestamps. In deformable-object tasks, continuous deformation and adjustments can make error onset ambiguous. We therefore exclude such events from quantitative evaluation while retaining them as qualitative examples.
  • Training-Seed Variability:
    • Setting: We train both ViVa and the VLM baseline with three independent random seeds, reporting the mean ± standard deviation of the evaluation metrics under unchanged experimental settings.
    • Results:
      TaskModelMS ↑ES ↑MDR ↑ErrDR ↑
      Box PackagingVLM0.0124 ± 0.00070.0076 ± 0.00160.7490 ± 0.01980.5091 ± 0.0211
      ViVa0.0435 ± 0.00430.0202 ± 0.01550.7826 ± 0.02770.5394 ± 0.0452
      Shirt FoldingVLM0.1852 ± 0.1781−0.0301 ± 0.00710.8896 ± 0.02100.5000 ± 0.0000
      ViVa0.1470 ± 0.0117−0.0108 ± 0.00940.9918 ± 0.00620.8333 ± 0.2357
      Paper Roll OrganizationVLM0.0244 ± 0.0026−0.0051 ± 0.00620.7501 ± 0.02820.4815 ± 0.0524
      ViVa0.0423 ± 0.00180.0047 ± 0.00660.8173 ± 0.05170.6296 ± 0.0524
    • Analysis: Across three training seeds, ViVa achieves higher mean scores on 11 of 12 metrics, including MDR and ErrDR across all three tasks, supporting its effectiveness under different training initializations.

AuthorsOpenReview ↗

Third Response to Reviewer ugkn


Q5: ERS metric definition and validity.

A5: We thank the reviewer for these questions. We provide the following clarifications and additional experimental results.

  • Metric Definition:
    1. ERS Computation: In the ERS formulation (Eq. 3), Norm(V)=1−clip(V,0,2)2\mathrm{Norm}(V)=1-\frac{\mathrm{clip}(V,0,2)}{2} maps predicted values to [0,1][0,1]. We use w=30w=30 frames to average predictions before and after each event and ε=10−9\varepsilon=10^{-9} to prevent division by zero.
    2. Curve Smoothing: Figure 4 is smoothed for visualization only. All metrics use the original, unsmoothed predictions.
  • Absolute Changes:
    • Setting: We recompute MS and ES using absolute changes instead of relative changes, keeping event directions and windows unchanged.
    • Results:
      TaskModelAbsolute MS ↑Absolute ES ↑
      Box PackagingVLM0.00070.0040
      ViVa0.00170.0150
      Shirt FoldingVLM0.07680.0058
      ViVa0.03140.0084
      Paper Roll OrganizationVLM0.00980.0237
      ViVa0.01000.0387
    • Analysis: These results confirm that ViVa's advantage in error sensitivity persists under absolute changes, demonstrating that it is not driven by relative normalization.
  • Elapsed-Time Baseline:
    • Setting: We evaluate a linear baseline increasing from 0 to 1 along each demonstration trajectory.
    • Results:
      TaskModelMS ↑ES ↑MDR ↑ErrDR ↑
      Box PackagingElapsed-Time0.0499−0.02631.0000.000
      ViVa0.03410.02630.7330.612
      Shirt FoldingElapsed-Time0.1270−0.09981.0000.000
      ViVa0.09540.01200.9901.000
      Paper Roll OrganizationElapsed-Time0.0508−0.03411.0000.000
      ViVa0.03860.05520.7620.778
    • Analysis:
      1. Monotonicity, Not Event Tracking: The baseline achieves perfect MDR simply because its predictions increase monotonically, regardless of actual task events.
      2. Zero Error Awareness: The baseline yields zero ErrDR and negative ES across all three tasks, whereas ViVa achieves positive ES and substantial ErrDR.
  • Random Non-Event Frames:
    • Setting: For each annotated event, we sample 100 random non-event controls at similar progress (difference ≤ 0.1), excluding windows overlapping annotated events.
    • Results:
      TaskFrame TypeMS ↑ES ↑
      Box PackagingRandom0.0280−0.0551
      Event0.03410.0263
      Shirt FoldingRandom0.0556−0.1844
      Event0.09540.0120
      Paper Roll OrganizationRandom0.0522−0.0239
      Event0.03860.0552
    • Analysis: These results support that ViVa's error-sensitive responses are associated with actual execution errors rather than arbitrary value fluctuations.

Q6: VLM baseline configuration.

A6: We thank the reviewer for these questions and clarify the configurations below:

  • Observations and Inputs: As clarified in A3, our VLM baseline adopts the value-model configuration of π0.6∗\pi^*_{0.6}, taking three camera views without proprioception as input and predicting a scalar value.
  • Return Definition: We use the same return definition as π0.6∗\pi^*_{0.6}.
  • Training Data: No web data is used during value-model training for any compared method.
  • Backbones (GVL & TopReward): GVL and TopReward use Qwen2.5-VL-7B, while our VLM baseline uses the smaller Qwen2.5-VL-3B and still outperforms both methods (Table 1).

Q7: RECAP advantage formulation and data collection.

A7: We thank the reviewer and clarify the following details:

  • Advantage Horizon & Threshold: Following π0.6∗\pi^*_{0.6}, we use an advantage horizon of 50 steps and set the threshold to label approximately 30% of samples as positive.
  • Human Corrections: Following π0.6∗\pi^*_{0.6}, human corrections are included during rollout collection to teach the policy to recover from execution errors.
  • Advantage Sign Convention: As shown in Appendix B.3, we reverse the predicted return's direction before computing RECAP advantages, ensuring that higher values indicate better states and positive advantages indicate favorable actions.
  • Round 1 Data Collection & Sharing:
    • Data Collection: The 150 successful paper-roll episodes include human-corrected trajectories, explaining the discrepancy with the initial 20% autonomous success rate.
    • Data Sharing: ViVa and VLM do not share the same Round 1 data. Both start from the same GigaBrain-0 policy but independently collect rollouts using their respective value-guided policies, following identical collection protocols and episode budgets.

Reviewer_Jg4QOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes ViVa, a value model built on a pretrained video generation model (Wan2.2). Using flow matching, it jointly generates the empirical return and the proprioceptive state K=50 steps ahead, both encoded as latent frames without extra heads. The authors argue that spatiotemporal video priors make the value model more sensitive to milestones and execution errors than VLM-based value models. They introduce event-level metrics and show that replacing the value model in RECAP improves real-robot success on three tasks from 63.3% to 80.0% on average.

Critical strengths

1)The idea is simple and well motivated: video generators model how interactions evolve over time, which static VLM representations miss. The modification to the backbone is minimal. 2)ViVa is cheaper than the VLM value model in both training (4 vs. 6 GPU-days) and inference (0.18 vs. 0.32 s/frame). 3)The downstream evaluation is controlled: same initial policy, same RECAP pipeline, only the value model is swapped. 4)The event-level evaluation (milestone sensitivity, error detection) is a useful complement to global correlation metrics.

Critical weaknesses

1)Limited statistical evidence. Each task has only 10 real-robot trials (24/30 vs. 19/30 aggregated; Fisher exact p≈0.25 by my calculation). There are no multiple seeds and no error bars. 2)Backbone vs. objective confound. The VLM baseline uses a 201-way classification target, while the video models use the proposed joint flow-matching objective. Without a VLM trained with the same objective, the gains cannot be attributed to the video prior. 3)Inconsistent numbers across tables. The Wan2.2 row in Table 6 (MS 0.137, ErrDR 0.500) differs from the ViVa shirt results in Tables 2/7/8 (MS 0.095, ErrDR 1.000). The default configuration's ES is reported as 0.012, 0.013 and 0.014 in different tables. This suggests run-to-run variance comparable to the differences being claimed. 4)Simple reward design. The return is based only on time fraction and final success, so errors that are later recovered from are not penalized. The normalization is not fully specified. 5)Narrow ablations and generalization. Ablations cover a single task, and generalization is tested only from shirts to pants.

Questions for authors

Q1. Can you report a VLM backbone trained with the identical joint flow-matching objective? Q2. How many milestone and error events are there per task, and how are they annotated? Q3.Can you provide multi-seed results or more real-robot trials per task? Q4.Which setting produced Table 6 versus Tables 2/7/8, and why do the numbers differ? Q5.Does multi-step denoising, or sampling multiple values, improve calibration or error detection?

4 comments
AuthorsOpenReview ↗

First Response to Reviewer Jg4Q

We sincerely thank Reviewer Jg4Q for the insightful suggestions and constructive feedback. Below, we provide clarifications and additional evidence addressing the remaining concerns.


W1&Q3: Limited statistical evidence.

A1: We thank the reviewer for highlighting this limitation. We conducted additional real-robot evaluations to strengthen the statistical evidence.

  • Setting: We use the same VLM and ViVa checkpoints as in Rollout 2 of Table 5 and conduct 20 additional trials per task for each model. Combined with the original 10 trials, each model is evaluated over 30 trials per task.
  • Results:
    ModelShirt FoldingBox PackagingPaper Roll OrganizationOverall
    RECAP (VLM)20/30 (66.7%)15/30 (50.0%)20/30 (66.7%)55/90 (61.1%)
    RECAP (ViVa)28/30 (93.3%)22/30 (73.3%)23/30 (76.7%)73/90 (81.1%)
  • Analysis:
    1. Real-Robot Success Rate: ViVa consistently outperforms VLM across all three tasks, improving the overall success rate from 61.1% to 81.1%, a 20-percentage-point absolute gain.
    2. Statistical Reliability: The improvement is statistically significant (two-sided Fisher's exact test, p = 0.0049), with a 95% CI of [6.8%, 32.3%] for the absolute success-rate difference.

W2&Q1: Backbone vs. objective confound.

A2: We thank the reviewer for this valuable suggestion. We introduce a VLM baseline trained with the identical joint flow-matching objective as ViVa.

  • Setting: We retain the pretrained VLM backbone and adopt the same Wan2.2 output-head architecture. Both models use aligned inputs and jointly predict value and future proprioception with identical flow-matching objectives and loss weights.
  • Results:
    TaskModelMS ↑ES ↑MDR ↑ErrDR ↑
    Box PackagingVLM + Flow Matching0.03720.00250.78580.5424
    ViVa0.03410.02630.73260.6121
    Shirt FoldingVLM + Flow Matching0.0431−0.01490.84200.5000
    ViVa0.09540.01200.99011.0000
    Paper Roll OrganizationVLM + Flow Matching0.0410−0.01360.73210.5556
    ViVa0.03860.05520.76180.7778
  • Analysis:
    1. Progress Tracking: With the same flow-matching objective, the VLM achieves competitive MS and MDR.
    2. Error Sensitivity: ViVa consistently outperforms the VLM in ES and ErrDR across all three tasks, indicating benefits beyond the shared training objective.

W3&Q4: Inconsistent numerical results.

A3: We thank the reviewer for identifying these inconsistencies and apologize for the confusion.

  • Wan2.2 Results (Different Configurations): The results correspond to two distinct model configurations:

    • Table 6: Image-only input with value prediction, aligned with the VLM baseline for a fair backbone comparison.
    • Tables 2/7/8: Image and proprioceptive inputs with both value and future-proprioception prediction, representing the full ViVa model.

    Thus, the differences arise from distinct input-output configurations rather than inconsistent results.

  • ES Discrepancies (Transcription Errors): We verified the original evaluation records and confirmed that the correct ES is 0.012. The values 0.013 and 0.014 in Tables 9 and 10 were transcription errors. The corrected ES remains the highest in both ablations, leaving the main conclusions unchanged. We sincerely apologize for this oversight.


W4: Simplified reward design.

A4: We thank the reviewer for the insightful comment and clarify both points below.

  • Reward Design: Our reward provides task-agnostic supervision without requiring event-level annotations. While recovered errors are not explicitly penalized (as doing so would require costly task-specific engineering), ViVa nevertheless demonstrates strong sensitivity to execution errors (Figure 3, Table 2).
  • Normalization: As defined in Eq. (2), we normalize the per-step progress cost by each episode's length, with terminal costs of 0 for success and 1 for failure. For downstream RL with RECAP, the predicted value is further mapped to a progress score St=1−clip⁡(Vθ(xt,ℓ),0,2)2∈[0,1]S_t=1-\frac{\operatorname{clip}(V_\theta(x_t,\ell),0,2)}{2}\in[0,1]. This fixed mapping requires no dataset-level normalization statistics and provides a consistent scale for RECAP advantage estimation.

AuthorsOpenReview ↗

Second Response to Reviewer Jg4Q


W5: Limited ablations and generalization.

A5: We thank the reviewer for this comment. To address these concerns, we conduct additional experiments on both ablation studies and generalization.

  • Architecture Design Ablation:
    • Setting: All variants take three-view images and the current proprioceptive state as inputs, with all other experimental settings unchanged.

    • Results: Table 1: Ablation Results on Paper Roll Organization

      ValueFuture ImageFuture Proprio.MS ↑ES ↑MDR ↑ErrDR ↑
      ✓✗✗0.019−0.0430.6440.444
      ✓✓✗0.0010.0070.5620.556
      ✓✓✓0.041−0.0050.7050.444
      ✓✗✓0.0390.0550.7620.778

      Table 2: Ablation Results on Box Packaging

      ValueFuture ImageFuture Proprio.MS ↑ES ↑MDR ↑ErrDR ↑
      ✓✗✗0.0470.0080.7780.527
      ✓✓✗0.0600.0110.7830.548
      ✓✓✓0.0430.0040.7890.452
      ✓✗✓0.0340.0260.7330.612
    • Analysis:

      • Future Proprioception: Consistently improves ES and ErrDR over value-only prediction on both tasks.
      • Future Image Prediction: Adding future-image prediction reduces ES and ErrDR compared with future proprioception alone on both tasks, supporting our choice to omit this component.
  • Scope of Additional Ablations: Section C.5 of the revised manuscript provides further ablations, including pretraining ablations and sensitivity analyses of KK and λ\lambda across all three tasks (Figures 8 and 9, respectively).
  • Generalization:
    • Setting: Given the limited rebuttal period, we collected 20 Hoodie Folding trajectories with manually annotated milestone keyframes for zero-shot evaluation, without additional training or fine-tuning.
    • Results: Table 3: Zero-Shot Generalization on Hoodie Folding
      ModelMS ↑MDR ↑
      VLM7.46×10−57.46 \times 10^{-5}0.538
      ViVa0.1010.662
    • Analysis:
      • Milestone Sensitivity: ViVa achieves higher MS, indicating stronger sensitivity to annotated milestones.
      • Zero-Shot Generalization: ViVa achieves higher MDR on unseen Hoodie Folding trajectories without fine-tuning, providing evidence of generalization to a novel task.

Q2: How many milestone and error events are there per task, and how are they annotated?

A6: We thank the reviewer for this question. We manually annotate 50 held-out demonstration episodes per task.

  • Event Counts: The milestone/error event counts used for metric computation are 733/330 for Box Packaging, 405/4 for Shirt Folding, and 739/9 for Paper Roll Organization.
  • Annotation: Milestones are marked upon completion of key manipulation steps, while errors are annotated based on observable execution deviations. We exclude error events whose onset cannot be reliably localized at the frame level. All methods use the same evaluation data and event annotations.
  • Task Characteristics:
    1. Box Packaging: This task involves precision-demanding manipulation of rigid objects, where discrete errors are easier to identify and localize.
    2. Shirt Folding & Paper Roll Organization: These tasks involve deformable materials, where continuous deformation and adjustments make precise frame-level error localization challenging.

AuthorsOpenReview ↗

Third Response to Reviewer Jg4Q


Q5: Does multi-step denoising or sampling multiple values improve calibration or error detection?

A7: We thank the reviewer for the constructive feedback and present additional experiments on denoising steps and noise sampling.

  • Denoising Steps:
    • Setting: We evaluate ViVa with 1, 4, 8, and 16 denoising steps on Box Packaging, keeping all other settings unchanged.
    • Results:
      StepsMS ↑ES ↑MDR ↑ErrDR ↑Latency (ms/frame) ↓
      10.0340.0260.7330.612136
      40.0310.0120.7640.515416
      80.0310.0120.7670.533788
      160.0310.0090.7640.5331532
    • Analysis:
      1. Performance: One-step denoising achieves the highest MS, ES, and ErrDR, while additional steps yield only modest improvements in MDR.
      2. Efficiency: One-step denoising requires only 136 ms/frame, offering a favorable performance–latency trade-off.
  • Sampling Multiple Values:
    • Setting: We conduct additional experiments with five independent random noise samples for ViVa, reporting the mean ± standard deviation of the evaluation metrics. All other settings remain unchanged, with VLM results from Tables 2&3.
    • Results:
      TaskModelMS ↑ES ↑MDR ↑ErrDR ↑
      Stack BoxVLM0.01310.00620.76530.5061
      ViVa0.0387 ± 0.00060.0615 ± 0.00060.8156 ± 0.00400.5285 ± 0.0073
      Fold ShirtVLM-0.2664-0.29850.40740.5000
      ViVa0.0958 ± 0.00030.0122 ± 0.00350.9891 ± 0.00120.9000 ± 0.2000
      Toilet PaperVLM0.02250.02800.70400.6667
      ViVa0.0383 ± 0.00030.0540 ± 0.00180.7615 ± 0.00410.7111 ± 0.0544
    • Analysis: Across five noise samples, ViVa outperforms VLM on all four metrics across all three tasks, demonstrating effective progress tracking, error sensitivity, and robustness to random noise initialization.

We have incorporated all corresponding revisions into the manuscript, with changes highlighted in blue. Thanks again for your thoughtful and valuable feedback. We have done our best to address all the concerns raised and sincerely hope our responses are helpful.

Thank you for the thorough response, which addresses my questions. I retain my score.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.