- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes Executable Video Alignment (EVA), a post-training framework that improves video-based robot planning through action-space feedback. An inverse dynamics model trained on robot demonstrations is frozen and used to decode generated videos into robot commands. Penalties on acceleration, jerk, and violations of velocity and acceleration limits serve as rewards for Flow-GRPO updates to the video generator. Experiments on RoboTwin, LIBERO, and a real bimanual robot show improvements in visual motion consistency and task execution, including an increase in RoboTwin success from 45.81% to 52.29%. The aligned model also produces more effective video–action training data for downstream vision-language-action policies.
Critical strengths
-
Action-space feedback provides a practical and interpretable alignment signal. EVA derives rewards from the smoothness and motion-limit violations of IDM-decoded commands, without requiring reference actions for generated rollouts or physical execution during reward computation. Applying this feedback to language-conditioned video planning is a useful contribution that connects visual generation quality with downstream robot execution.
-
The evaluation provides evidence beyond improvements in visual appearance. The paper evaluates direct execution in simulation and on a real robot, as well as the usefulness of generated data for downstream policy training. Importantly, the gains persist with an independently trained ViT-based IDM that was not used for reward computation, helping alleviate concerns that the improvements are specific to the reward-producing decoder.
Critical weaknesses
-
Substantial post-training cost with insufficient evidence of cost-effectiveness. Simulation post-training takes approximately six days on eight A800 GPUs, while EVA improves RoboTwin success by only 2.0 percentage points over IDM-aux SFT (52.29% versus 50.29%). Although the paper describes comparable optimization budgets, it does not report wall-clock time or GPU-hours for the alternative. The additional benefit of RL post-training therefore needs to be better justified relative to its computational cost.
-
The reward is susceptible to over-optimization, and the mitigation relies on qualitative checkpoint selection. Appendix C reports that prolonged optimization can produce high-reward failures, including nearly static rollouts, incorrect contacts, and unrealistic robot geometry. The authors mitigate this through KL regularization and qualitative inspection of robot integrity and task progress on a held-out validation set. However, the inspection procedure and stopping criteria are not sufficiently operationalized, making it difficult to assess reproducibility and sensitivity to checkpoint selection.
-
The real-robot evaluation does not sufficiently isolate the claimed improvement in executability. The success-rate gains are encouraging, but the evaluation is restricted to one robot platform and a set of standard tabletop tasks. It does not quantify whether the improvements correspond to reduced command discontinuities, fewer motion-limit violations, or better trajectory tracking during physical execution. More targeted measurements and challenging motion conditions would better establish the practical significance of the proposed alignment.
-
The stability of the reported improvements remains insufficiently characterized. The paper does not report variability across independent post-training runs or uncertainty estimates for the main execution comparisons. This is particularly relevant to the relatively small differences between EVA and alternative training or inference strategies. Although the consistent improvements across many tasks are encouraging, the available results do not establish how reliably these smaller advantages persist under training and evaluation randomness.
Questions for authors
-
Could the authors specify the validation inspection frequency, the operational criteria for robot integrity and task progress, and the exact early-stopping and checkpoint-selection procedure? Using validation data separate from the final test set, could the authors show how reward, task progress, and execution performance evolve during training, including when reward over-optimization begins to degrade performance?
-
Under matched computational budgets, does EVA’s 2.0-percentage-point advantage over IDM-aux SFT persist across independent training runs or paired evaluations? Please report wall-clock time or GPU-hours for both methods and quantify the uncertainty in their performance difference.
4 comments
Response to Reviewer XZP3 (Part II)
W3. Executability on the real robot
Thank you for this helpful suggestion. We analyzed failures across 200 real-robot trials per method on AgileX, covering five seen and five OOD tasks with 20 trials per task. We observed three recurring failure types.
- Command-limit violations, which trigger termination.
- Visual degradation–related failures, where distorted videos lead to discontinuous or task-inconsistent IDM-decoded actions.
- Insufficient precision of actions decoded from monocular video, causing inaccurate manipulation.
EVA reduces the first two categories in our evaluation.
| Failure category | Base | EVA (RL) |
|---|---|---|
| Command-limit violations | 13 | 2 |
| Visual degradation–related failures | 28 | 5 |
W4. Uncertainty in the RL–SFT comparison
We repeated the evaluation of the fixed RL and SFT checkpoints using three matched evaluation seeds. Each batch contains 1,050 trials across 21 tasks, with 50 trials per task.
| Evaluation / statistic | EVA (RL) | EVA (IDM-aux SFT) | RL–SFT difference (pp) |
|---|---|---|---|
| Seed 1, successes | 549/1,050 | 519/1,050 | +2.86 |
| Seed 2, successes | 553/1,050 | 528/1,050 | +2.38 |
| Seed 3, successes | 547/1,050 | 530/1,050 | +1.62 |
| Mean success ± SD (%) | 52.35 ± 0.29 | 50.06 ± 0.56 | +2.29 ± 0.62 |
| 95% CI of the mean difference | — | — | [+0.73, +3.84] |
SD is the sample standard deviation across three evaluation seeds. The 95% CI uses a paired Student's interval for the mean difference between fixed checkpoints.
Manuscript updates
The revised PDF incorporates the comparisons and analyses reported above.
- W1/Q2 and W4 — Contribution and training-method comparison in Section 5.5, Table 4(a) (L434–465), and the auxiliary objective, budgets, and repeated evaluations in Appendix I.1–I.2 (L1300–1334).
- W2/Q1 — Checkpoint analysis in Appendix C, Figure S6 (L978–992).
- W1/Q2 and W3 — Hardware SFT comparison and failure counts in Appendix I.3, Tables S7–S8 (L1352–1380).
Kindly Reminder
Dear Reviewer XZP3,
Thank you again for your careful review and for recognizing the value of action-space feedback.
In response to your concerns about computational cost and the reliability of the RL–SFT difference, we have clarified that both methods use approximately 1,000 GPU-hours of post-training. We have also added three paired evaluations of fixed checkpoints, yielding a mean RL advantage of 2.29 percentage points with a 95% confidence interval of [+0.73, +3.84], together with the auxiliary-SFT hardware comparison (Appendix I.1–I.3).
We have further specified the validation and checkpoint-selection procedure and added Figure S6 to illustrate reward over-optimization through changes in visual quality and closed-loop success. To examine practical executability, we have added hardware failure statistics across 200 trials per method, quantifying reductions in command-limit violations and visual degradation–related failures (Table S8).
We would greatly appreciate it if you could review our responses and the revised manuscript and let us know whether these clarifications and additional results address your concerns. Your feedback would help us identify any remaining issues, and we would be happy to provide further clarification.
Yours sincerely,
The Authors
Thank you for the additional results and clarifications. The matched-budget comparison, paired evaluations, hardware SFT baseline, and failure statistics address several of my concerns and strengthen the evidence for practical improvements.
However, I remain unconvinced that the contribution and demonstrated benefits are sufficient given the substantial post-training budget. I acknowledge the hardware gains of 6 and 11 percentage points over auxiliary SFT, alongside the smaller simulation gain. My concern is the overall value of the approach, rather than a claim that RL uses more compute than SFT.
The checkpoint analysis is helpful, but stopping when visual quality and task progress “largely saturate” still leaves substantial judgment in model selection. The repeated evaluations reduce uncertainty for the fixed checkpoints, while variability from training and checkpoint selection remains uncharacterized.
These additions increase my confidence that the approach is useful, but do not sufficiently change my overall assessment. I therefore maintain my current rating of 2 (weak rejection).
Response to Reviewer XZP3 (Part I)
Thank you for your careful review and for recognizing the value of action-space feedback. Your questions motivated additional training-method comparisons and execution diagnostics.
Q1 / W2. Validation and checkpoint selection
Thank you for this valuable suggestion. We select checkpoints based on visual quality evaluated on a separate validation set of 100 samples, excluded from both training and final testing.
We added Figure S6 to track reward, visual quality, and closed-loop success at selected checkpoints.
The reported 136-step checkpoint falls within the visual-quality plateau (Appendix C, L912–915). Further optimization increases reward while visual quality and execution success decline, illustrating reward over-optimization.
Q2 / W1. Computational cost and the benefit of RL
Thank you for asking us to make the computational comparison explicit. EVA's central contribution is using IDM-derived action-space feedback to improve video generation. Section 5.5 (L430–437) and Table 4(a) evaluate auxiliary SFT and RL as two implementations of this idea. We now label them EVA (IDM-aux SFT) and EVA (RL) to make their shared contribution explicit.
EVA (IDM-aux SFT) adds the negative of the same full IDM reward to the supervised objective,
LIDM-aux SFT(θ)=LFM(θ)−λauxE[R(Vθ)].
Here, Vθ is the predicted video used for the auxiliary term. R contains the same acceleration, jerk, velocity-limit, and acceleration-limit penalties used by RL.
Both methods start from the same Base checkpoint. We now specify their post-training budgets alongside the reported RoboTwin results.
Repeated-evaluation means and sample SDs use three evaluation seeds per fixed checkpoint, with 1,050 trials per seed. The paired RL–SFT difference and its 95% CI are reported in W4.
We additionally report a hardware comparison between RL and IDM-aux SFT, using the same five seen and five OOD tasks, with 20 trials per task (Table S7, L1352–1364). RL exceeds auxiliary SFT by 6 percentage points on seen tasks and 11 points on OOD tasks, providing further empirical support for choosing RL as our main implementation.