acceptodds
Withdrawn from ICLR 2027

Aligning Video World Models with Executable Robot Actions via Inverse Dynamics Rewards

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Video generative models offer a promising basis for robotic planning by predicting future visual rollouts from observations and task instructions. However, their training objectives typically lack explicit constraints on robot executability. Generated rollouts can exhibit embodiment-related artifacts, including arm deformation and temporal discontinuities, while the actions decoded from these rollouts may be unstable or violate robot motion limits. We term this mismatch between generated robot behavior and embodiment-specific execution requirements the *executability gap*. We introduce **Executable Video Alignment (EVA)**, a post-training framework that uses action-space feedback to improve both the visual integrity of generated robot motion and the feasibility of decoded actions. EVA trains an inverse dynamics model (IDM) on robot demonstrations and freezes it to decode actions from generated videos. We compute a reward from these actions that penalizes non-smooth motion and violations of velocity and acceleration limits, and use it to guide reinforcement-learning updates to the video generator. By feeding decoded-action constraints back into video generation, EVA links visual and action-level improvements through a shared alignment signal. Experiments on RoboTwin, LIBERO, and a real bimanual robot show that EVA reduces embodiment-related visual artifacts and improves task success when executing IDM-decoded actions. Further experiments show that video models fine-tuned with EVA generate more effective video–action pairs for training downstream VLAs.

Primary area:
applications to robotics, autonomy, planning
Keywords:
Video world models, Robotic manipulation, Reward-based alignment

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_XZP3OpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes Executable Video Alignment (EVA), a post-training framework that improves video-based robot planning through action-space feedback. An inverse dynamics model trained on robot demonstrations is frozen and used to decode generated videos into robot commands. Penalties on acceleration, jerk, and violations of velocity and acceleration limits serve as rewards for Flow-GRPO updates to the video generator. Experiments on RoboTwin, LIBERO, and a real bimanual robot show improvements in visual motion consistency and task execution, including an increase in RoboTwin success from 45.81% to 52.29%. The aligned model also produces more effective video–action training data for downstream vision-language-action policies.

Critical strengths
  1. Action-space feedback provides a practical and interpretable alignment signal. EVA derives rewards from the smoothness and motion-limit violations of IDM-decoded commands, without requiring reference actions for generated rollouts or physical execution during reward computation. Applying this feedback to language-conditioned video planning is a useful contribution that connects visual generation quality with downstream robot execution.

  2. The evaluation provides evidence beyond improvements in visual appearance. The paper evaluates direct execution in simulation and on a real robot, as well as the usefulness of generated data for downstream policy training. Importantly, the gains persist with an independently trained ViT-based IDM that was not used for reward computation, helping alleviate concerns that the improvements are specific to the reward-producing decoder.

Critical weaknesses
  1. Substantial post-training cost with insufficient evidence of cost-effectiveness. Simulation post-training takes approximately six days on eight A800 GPUs, while EVA improves RoboTwin success by only 2.0 percentage points over IDM-aux SFT (52.29% versus 50.29%). Although the paper describes comparable optimization budgets, it does not report wall-clock time or GPU-hours for the alternative. The additional benefit of RL post-training therefore needs to be better justified relative to its computational cost.

  2. The reward is susceptible to over-optimization, and the mitigation relies on qualitative checkpoint selection. Appendix C reports that prolonged optimization can produce high-reward failures, including nearly static rollouts, incorrect contacts, and unrealistic robot geometry. The authors mitigate this through KL regularization and qualitative inspection of robot integrity and task progress on a held-out validation set. However, the inspection procedure and stopping criteria are not sufficiently operationalized, making it difficult to assess reproducibility and sensitivity to checkpoint selection.

  3. The real-robot evaluation does not sufficiently isolate the claimed improvement in executability. The success-rate gains are encouraging, but the evaluation is restricted to one robot platform and a set of standard tabletop tasks. It does not quantify whether the improvements correspond to reduced command discontinuities, fewer motion-limit violations, or better trajectory tracking during physical execution. More targeted measurements and challenging motion conditions would better establish the practical significance of the proposed alignment.

  4. The stability of the reported improvements remains insufficiently characterized. The paper does not report variability across independent post-training runs or uncertainty estimates for the main execution comparisons. This is particularly relevant to the relatively small differences between EVA and alternative training or inference strategies. Although the consistent improvements across many tasks are encouraging, the available results do not establish how reliably these smaller advantages persist under training and evaluation randomness.

Questions for authors
  1. Could the authors specify the validation inspection frequency, the operational criteria for robot integrity and task progress, and the exact early-stopping and checkpoint-selection procedure? Using validation data separate from the final test set, could the authors show how reward, task progress, and execution performance evolve during training, including when reward over-optimization begins to degrade performance?

  2. Under matched computational budgets, does EVA’s 2.0-percentage-point advantage over IDM-aux SFT persist across independent training runs or paired evaluations? Please report wall-clock time or GPU-hours for both methods and quantify the uncertainty in their performance difference.

4 comments
AuthorsOpenReview ↗

Response to Reviewer XZP3 (Part I)

Thank you for your careful review and for recognizing the value of action-space feedback. Your questions motivated additional training-method comparisons and execution diagnostics.

Q1 / W2. Validation and checkpoint selection

Thank you for this valuable suggestion. We select checkpoints based on visual quality evaluated on a separate validation set of 100 samples, excluded from both training and final testing.

  • Frequency. We evaluate validation videos every 4 training steps.
  • Criteria. We assess robot integrity through geometric consistency and temporal continuity, and task progress through movement toward the instructed goal, following the visual-quality rubric in Section 5.2 (L307–313).
  • Selection. We stop once improvements in these criteria largely saturate and use the checkpoint at this stopping point for evaluation (Appendix C, L972–979).

We added Figure S6 to track reward, visual quality, and closed-loop success at selected checkpoints.

Training stepVisually perfectClosed-loop successNormalized reward, 0–100
12084%52%82.2
16084%53%83.4
20074%47%85.3

The reported 136-step checkpoint falls within the visual-quality plateau (Appendix C, L912–915). Further optimization increases reward while visual quality and execution success decline, illustrating reward over-optimization.

Q2 / W1. Computational cost and the benefit of RL

Thank you for asking us to make the computational comparison explicit. EVA's central contribution is using IDM-derived action-space feedback to improve video generation. Section 5.5 (L430–437) and Table 4(a) evaluate auxiliary SFT and RL as two implementations of this idea. We now label them EVA (IDM-aux SFT) and EVA (RL) to make their shared contribution explicit.

EVA (IDM-aux SFT) adds the negative of the same full IDM reward to the supervised objective,

LIDM-aux SFT(θ)=LFM(θ)−λauxE[R(V^θ)].\mathcal{L}_{\text{IDM-aux SFT}}(\theta) = \mathcal{L}_{\mathrm{FM}}(\theta) - \lambda_{\mathrm{aux}}\mathbb{E}[R(\widehat{V}_\theta)].

Here, V^θ\widehat{V}_\theta is the predicted video used for the auxiliary term. RR contains the same acceleration, jerk, velocity-limit, and acceleration-limit penalties used by RL.

Both methods start from the same Base checkpoint. We now specify their post-training budgets alongside the reported RoboTwin results.

MethodGPU-hoursTable 4(a) success (%)Gain over Base (pp)Repeated mean ± SD (%)
Base—45.81——
EVA (IDM-aux SFT)≈1,00050.29+4.4850.06 ± 0.56
EVA (RL)≈1,00052.29+6.4852.35 ± 0.29

Repeated-evaluation means and sample SDs use three evaluation seeds per fixed checkpoint, with 1,050 trials per seed. The paired RL–SFT difference and its 95% CI are reported in W4.

We additionally report a hardware comparison between RL and IDM-aux SFT, using the same five seen and five OOD tasks, with 20 trials per task (Table S7, L1352–1364). RL exceeds auxiliary SFT by 6 percentage points on seen tasks and 11 points on OOD tasks, providing further empirical support for choosing RL as our main implementation.

AuthorsOpenReview ↗

Response to Reviewer XZP3 (Part II)

W3. Executability on the real robot

Thank you for this helpful suggestion. We analyzed failures across 200 real-robot trials per method on AgileX, covering five seen and five OOD tasks with 20 trials per task. We observed three recurring failure types.

  1. Command-limit violations, which trigger termination.
  2. Visual degradation–related failures, where distorted videos lead to discontinuous or task-inconsistent IDM-decoded actions.
  3. Insufficient precision of actions decoded from monocular video, causing inaccurate manipulation.

EVA reduces the first two categories in our evaluation.

Failure categoryBaseEVA (RL)
Command-limit violations132
Visual degradation–related failures285

W4. Uncertainty in the RL–SFT comparison

We repeated the evaluation of the fixed RL and SFT checkpoints using three matched evaluation seeds. Each batch contains 1,050 trials across 21 tasks, with 50 trials per task.

Evaluation / statisticEVA (RL)EVA (IDM-aux SFT)RL–SFT difference (pp)
Seed 1, successes549/1,050519/1,050+2.86
Seed 2, successes553/1,050528/1,050+2.38
Seed 3, successes547/1,050530/1,050+1.62
Mean success ± SD (%)52.35 ± 0.2950.06 ± 0.56+2.29 ± 0.62
95% CI of the mean difference——[+0.73, +3.84]

SD is the sample standard deviation across three evaluation seeds. The 95% CI uses a paired Student's tt interval for the mean difference between fixed checkpoints.

Manuscript updates

The revised PDF incorporates the comparisons and analyses reported above.

  • W1/Q2 and W4 — Contribution and training-method comparison in Section 5.5, Table 4(a) (L434–465), and the auxiliary objective, budgets, and repeated evaluations in Appendix I.1–I.2 (L1300–1334).
  • W2/Q1 — Checkpoint analysis in Appendix C, Figure S6 (L978–992).
  • W1/Q2 and W3 — Hardware SFT comparison and failure counts in Appendix I.3, Tables S7–S8 (L1352–1380).
AuthorsOpenReview ↗

Kindly Reminder

Dear Reviewer XZP3,

Thank you again for your careful review and for recognizing the value of action-space feedback.

In response to your concerns about computational cost and the reliability of the RL–SFT difference, we have clarified that both methods use approximately 1,000 GPU-hours of post-training. We have also added three paired evaluations of fixed checkpoints, yielding a mean RL advantage of 2.29 percentage points with a 95% confidence interval of [+0.73, +3.84], together with the auxiliary-SFT hardware comparison (Appendix I.1–I.3).

We have further specified the validation and checkpoint-selection procedure and added Figure S6 to illustrate reward over-optimization through changes in visual quality and closed-loop success. To examine practical executability, we have added hardware failure statistics across 200 trials per method, quantifying reductions in command-limit violations and visual degradation–related failures (Table S8).

We would greatly appreciate it if you could review our responses and the revised manuscript and let us know whether these clarifications and additional results address your concerns. Your feedback would help us identify any remaining issues, and we would be happy to provide further clarification.

Yours sincerely,
The Authors

Thank you for the additional results and clarifications. The matched-budget comparison, paired evaluations, hardware SFT baseline, and failure statistics address several of my concerns and strengthen the evidence for practical improvements.

However, I remain unconvinced that the contribution and demonstrated benefits are sufficient given the substantial post-training budget. I acknowledge the hardware gains of 6 and 11 percentage points over auxiliary SFT, alongside the smaller simulation gain. My concern is the overall value of the approach, rather than a claim that RL uses more compute than SFT.

The checkpoint analysis is helpful, but stopping when visual quality and task progress “largely saturate” still leaves substantial judgment in model selection. The repeated evaluations reduce uncertainty for the fixed checkpoints, while variability from training and checkpoint selection remains uncharacterized.

These additions increase my confidence that the approach is useful, but do not sufficiently change my overall assessment. I therefore maintain my current rating of 2 (weak rejection).

Reviewer_ZFE7OpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper introduces Executable Video Alignment (EVA), which post-trains a video planner using rewards derived from a frozen inverse dynamics model. Generated videos are decoded into robot commands, and penalties on acceleration, jerk, and motion-limit violations provide feedback for Flow-GRPO updates to the generator. Experiments report improved visual integrity and direct execution on RoboTwin, LIBERO, and an AgileX bimanual platform. Hardware success improves from 51% to 64% on five seen tasks and from 42% to 60% on five OOD tasks, with 20 trials per task. Further experiments evaluate synthetic data for VLA training and transfer to an independently trained action decoder.

Critical strengths
  1. The approach addresses an important disconnect between visual planning and physical control. Using IDM-decoded commands as feedback provides an embodiment-dependent signal that generic visual-quality rewards lack. Keeping the decoder fixed makes the Base–EVA comparison informative. The independent ViT-IDM evaluation is especially useful: success improves from 43.5% to 49.7% with a decoder excluded from reward computation, providing evidence that the gains extend beyond adaptation to the reward-producing decoder.
  2. The empirical evaluation connects generated plans to actual execution. It covers 21 RoboTwin tasks, three LIBERO suites, and ten hardware tasks, including five tasks outside the collected training set. The real-robot improvements are substantial relative to Base, and the reward-component, auxiliary-training, smoothing, and rejection-sampling comparisons help characterize the source of improvement. The downstream VLA experiment also provides initial evidence that better generated video–action pairs can improve policy training, although that evidence remains limited to three simulated tasks.
Critical weaknesses
  1. The executability surrogate has a substantial and only partially controlled failure mode. Smooth commands within coordinate-wise limits do not ensure valid contact, collision avoidance, or task progress. Appendix C explicitly reports high-reward failures involving incorrect contact, unrealistic geometry, and nearly static videos after prolonged optimization. KL regularization and qualitative assessment of a held-out validation set are used for checkpoint selection, but the selection criteria and their relationship to actual execution success are not quantified sufficiently. Thus, the evidence supports useful motion regularization more clearly than a generally reliable executability reward. A checkpoint-level analysis would clarify how much the reported outcome depends on manual stopping decisions.
  2. The incremental benefit of reinforcement learning over simpler use of the same action-space signal is not yet firmly established. On the full 21-task RoboTwin evaluation, IDM-aux SFT reaches 50.29% success versus 52.29% for EVA, compared with 45.81% for Base. This is a modest additional gain given the reported six-day RL run on eight A800 GPUs. Repeated-training uncertainty is not reported, and the hardware evaluation does not include the auxiliary-training alternative. The real-robot results support the overall alignment idea, but do not isolate whether the expensive RL procedure is necessary. Despite these concerns, the consistent execution gains and direct hardware evidence make the contribution useful enough for a weak acceptance recommendation.
Questions for authors
  1. Can you provide an explicit validation/checkpoint-selection protocol and plot execution success, task progress, and actual motion violations across post-training checkpoints, including the onset of reward over-optimization?
  2. Does EVA consistently outperform IDM-aux SFT across independent training runs and on the hardware tasks, with comparable optimization resources and the same decoder?
  3. What are the measured command-limit violations and failure categories during real execution? Please also report planning latency and wall-clock task duration to clarify the practical control setting.
2 comments
AuthorsOpenReview ↗

Response to Reviewer ZFE7

Thank you for recognizing the practical value of EVA and for suggesting these training and hardware analyses.

Q1 / W1. Checkpoint behavior and the reward surrogate

Thank you for this valuable suggestion. We treat the reward as a partial executability surrogate and control over-optimization through KL regularization and validation-based stopping (Appendix C, L965–979).

  • Every 4 training steps, we assess 100 videos on a separate validation set, excluded from training and final testing.
  • Selection uses robot integrity and task progress, following the visual-quality rubric in Section 5.2 (L307–313). We stop when these criteria largely saturate and use the checkpoint at that stopping point.

Figure S6 plots reward, visually perfect, and closed-loop success at selected checkpoints, with success evaluated every 40 steps for analysis. Visual quality plateaus over steps 120–160, then declines alongside execution success despite rising reward, supporting validation-based stopping. Task progress is included in the visual rubric. Executed violation counts are aggregated for the final models in Q3.

Q2 / W2. Auxiliary SFT, hardware performance, and uncertainty

Thank you for this suggestion. Auxiliary SFT and RL are both implementations of EVA's action-space feedback. On RoboTwin, both start from the same Base checkpoint and use the same full IDM reward and decoder, with approximately 1,000 GPU-hours per method (Section 5.5, L430–437, Table 4(a)). We added repeated evaluations and the auxiliary-SFT hardware comparison using five seen and five OOD tasks, with 20 trials per task (Section 5.1, L296–298).

Method / statisticRoboTwin success (%)Hardware seen (%)Hardware OOD (%)
Base—5142
EVA (IDM-aux SFT)50.06 ± 0.565849
EVA (RL)52.35 ± 0.296460
RL–SFT difference (pp)+2.29 ± 0.62+6+11
95% CI of the mean difference (pp)[+0.73, +3.84]——

RoboTwin values are mean ± sample SD across three evaluation seeds for fixed checkpoints, with 1,050 trials per seed. The 95% CI uses a paired Student's tt interval.

Q3. Hardware failures and practical execution time

Across 200 AgileX trials per method, we observed three recurring failure types.

  1. Command-limit violations, which trigger termination.
  2. Visual degradation–related failures, where distorted videos lead to discontinuous or task-inconsistent IDM-decoded actions.
  3. Insufficient precision of actions decoded from monocular video.

EVA reduces the first two categories in our evaluation.

Failure categoryBaseEVA (RL)
Command-limit violations132
Visual degradation–related failures285

A complete video-generation, IDM-decoding, and action-execution cycle averages 150 seconds. Using this average, estimated task durations are 2.5 minutes for Place2Basket (one cycle) and 5–7.5 minutes for StackBowl (two–three cycles). The policy replans after each chunk (Appendix C, L960–963). Trials terminate upon a command-limit violation or after five cycles without success.

Manuscript updates

The revised PDF incorporates these additional analyses and implementation details.

  • W1/Q1 — Checkpoint analysis in Appendix C, Figure S6 (L978–992).
  • W2/Q2 — Auxiliary objective, budgets, and repeated evaluations in Appendix I.1–I.2 (L1300–1334), and hardware SFT comparison in Appendix I.3, Table S7 (L1352–1363).
  • Q3 — Hardware failure counts and execution timing in Appendix I.3, Table S8 and accompanying text (L1366–1388).
AuthorsOpenReview ↗

Kindly Reminder

Dear Reviewer ZFE7,

Thank you again for recognizing the practical value of EVA and for your constructive suggestions.

In response to your comments, we have clarified the validation and checkpoint-selection procedure and added Figure S6 to show how reward, visual quality, and closed-loop success evolve during post-training, including late-stage degradation. We have also clarified the comparable computational budgets and added paired evaluations across three evaluation seeds for fixed RL and SFT checkpoints, together with the auxiliary-SFT hardware comparison (Appendix I.1–I.3). On hardware, RL exceeds auxiliary SFT by 6 percentage points on seen tasks and 11 points on OOD tasks. Finally, we have added hardware failure counts, the average end-to-end generation–decoding–execution time, and cycle-based task-duration estimates to clarify the practical execution setting (Appendix I.3).

We would greatly appreciate it if you could review our response and the revised manuscript and let us know whether these additions address your concerns. Your feedback would help us understand any remaining questions, and we would be happy to provide further clarification.

Yours sincerely,
The Authors

Reviewer_PBP2OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper presents Executable Video Alignment (EVA), a framework for post-training robotic video generators using action-space feedback from a frozen inverse dynamics model (IDM). The IDM decodes generated videos into robot commands, whose smoothness and velocity/acceleration-limit violations define a reward for reinforcement-learning updates to the generator. Experiments on RoboTwin 2.0, LIBERO, and a real robot platform show improved task success when executing IDM-decoded actions. Additional experiments demonstrate that EVA-generated video–action pairs improve downstream VLA performance relative to pairs generated by the unaligned model.

Critical strengths

Practical action-space feedback with execution-based validation. EVA uses a frozen IDM to provide embodiment-specific feedback without executing generated rollouts during post-training. Consistent improvements across RoboTwin 2.0, LIBERO, and real-robot experiments support the practical usefulness of this approach. Comparisons with auxiliary supervised training, action smoothing, and reward-based rejection sampling, together with reward-component ablations, help establish the value of the proposed feedback signal.

Critical weaknesses
  • Insufficient characterization of the executability gap. Although the paper reports improved human-rated visual plausibility and downstream execution success, it does not sufficiently isolate the relationship between visual artifacts, IDM decoding errors, and execution failures. The reward diagnostics provide initial associative evidence, but do not establish which failure mechanisms EVA addresses or how much they account for the execution gains. This limits support for the claimed contribution of identifying and characterizing the executability gap.
  • Insufficient evidence of reward reliability during post-training. EVA relies entirely on motion penalties computed through a frozen IDM, without task-success feedback or environment execution during training. The paper acknowledges that optimization can produce high-reward but stationary, task-incorrect, or geometrically implausible videos, and mitigates this through qualitative validation-based stopping. The independent-decoder evaluation supports transfer of the final model’s gains, but does not establish how reward-producing IDM errors or checkpoint selection affect training robustness. This leaves an important uncertainty about the reliability and reproducibility of the core alignment procedure.
Questions for authors

Although the paper reports the demonstration datasets and RL optimization settings, the construction and sampling distribution of observation–instruction contexts for RL post-training are insufficiently specified. How are observation–instruction contexts sampled during RL post-training? Are observations drawn from demonstration initial states, intermediate demonstration states, or generated rollout histories? Since decoded actions are not executed during training, how does the context distribution cover states encountered during closed-loop deployment?

2 comments
AuthorsOpenReview ↗

Response to Reviewer PBP2

Thank you for recognizing the value of our action-space feedback concept. We appreciate the opportunity to clarify the implementation details.

Q1. Context sampling and closed-loop deployment

How are observation–instruction contexts sampled during RL post-training? Are observations drawn from demonstration initial states, intermediate demonstration states, or generated rollout histories?

RL randomly samples video clips from the training demonstrations and pairs the corresponding observation with the task instruction, c=(oτ,ℓ)c=(o_\tau,\ell). The observation comes from an intermediate demonstration state. Section 3 (L153–156) defines this conditioning, and Appendix C (L909–910) specifies eight candidate videos per prompt.

Since decoded actions are not executed during training, how does the context distribution cover states encountered during closed-loop deployment?

During closed-loop execution, each action chunk changes the environment, so subsequent contexts may be unseen during training. Sampling intermediate clips broadens coverage across task stages. Conditioning on recorded observations is analogous to teacher forcing and follows the training paradigm commonly used by VLA policies. Our real-robot closed-loop results provide empirical support for generalization under this setting, with success increasing from 51% to 64% on seen tasks and from 42% to 60% on OOD tasks (Figure 5).

W1. Characterizing the executability gap

The reward diagnostics provide initial associative evidence, but do not establish which failure mechanisms EVA addresses or how much they account for the execution gains.

EVA primarily reduces motion-related failures. Table 1 (L286–292) shows the largest improvement in kinematic plausibility, consistent with the failure analysis in Appendix A (L812–820) and reward diagnostics in Figure S3. We additionally counted execution failures across 200 hardware trials per method.

Failure categoryBaseEVA (RL)
Command-limit violations132
Visual degradation–related failures285

Command-limit violations terminate execution. Visual degradation–related failures involve discontinuous or task-inconsistent IDM-decoded actions. These measurements quantify two concrete failure reductions accompanying the success gains.

W2. Reward reliability and checkpoint selection

The paper acknowledges that optimization can produce high-reward but stationary, task-incorrect, or geometrically implausible videos, and mitigates this through qualitative validation-based stopping. The independent-decoder evaluation supports transfer of the final model’s gains, but does not establish how reward-producing IDM errors or checkpoint selection affect training robustness.

We observed reward hacking during prolonged optimization, including nearly static or geometrically implausible videos with high rewards, as analyzed in Appendix C (L965–970) and Figure S5.

We mitigate this with KL regularization and validation-based checkpoint selection. Every 4 training steps, we assess robot integrity and task progress on a held-out validation set, separate from training and final testing. We stop once these criteria largely saturate and use the checkpoint at this stopping point for evaluation. Figure S6 shows the plateau followed by declining visual quality and closed-loop success despite increasing reward, supporting this stopping criterion.

Manuscript updates

The revised PDF incorporates these clarifications and additional results.

  • Q1 — Context sampling in Section 4.3 (L240–241) and Appendix C (L917–922).
  • W1 — Hardware failure counts in Appendix I.3, Table S8 (L1366–1380).
  • W2 — Checkpoint analysis in Appendix C, Figure S6 (L978–992).
AuthorsOpenReview ↗

Kindly Reminder

Dear Reviewer PBP2,

Thank you again for your time and constructive feedback.

In response to your comments, we have clarified how RL contexts are sampled from intermediate demonstration states and discussed generalization under closed-loop deployment (Section 4.3 and Appendix C). We have also added hardware failure statistics across 200 trials per method, showing reductions in command-limit violations from 13 to 2 and visual degradation–related failures from 28 to 5 (Table S8). Regarding reward over-optimization, our response specifies the validation and checkpoint-selection procedure, and the added Figure S6 shows how reward, visual quality, and closed-loop success evolve during training, including the onset of performance degradation.

We would greatly appreciate it if you could review our response and the revised manuscript and let us know whether these clarifications and additional results address your concerns. Your feedback would help us understand any remaining issues, and we would be happy to provide further explanation or clarification.

Yours sincerely,
The Authors

Comments on the paper

2 comments
AuthorsOpenReview ↗

Global response

Dear Reviewers,

We highly appreciate the constructive feedback, which has helped us clarify the method and strengthen the experimental analysis. Revisions are highlighted in blue\textcolor{blue}{\text{blue}} in the updated manuscript.

  1. Reward reliability and checkpoint selection (PBP2 W2, XZP3 Q1/W2, ZFE7 Q1/W1) — We have added checkpoint curves to Appendix C (Figure S6), showing how visual quality and closed-loop success plateau and then decline despite increasing reward. Our responses clarify that we evaluate 100 validation videos every 4 training steps and select the checkpoint when robot integrity and task progress largely saturate. The validation set is separate from training and final testing.

  2. Auxiliary SFT and computational cost (XZP3 Q2/W1, ZFE7 Q2/W2) — We have revised Section 5.5 (Table 4(a)) to present EVA (IDM-aux SFT) and EVA (RL) as two implementations of action-space feedback. Both use the same full IDM reward and approximately 1,000 GPU-hours of post-training, and both improve success over Base. We have added the auxiliary objective, computational budgets, and rationale for choosing RL as the main implementation to Appendix I.1.

  3. Repeated evaluation and hardware comparison (XZP3 Q2/W1/W4, ZFE7 Q2/W2) — Across three paired evaluations of fixed checkpoints, RL achieves a mean advantage of 2.29 percentage points over auxiliary SFT, with a 95% confidence interval of [+0.73, +3.84]. The added hardware comparison shows RL gains of 6 points on seen tasks and 11 on OOD tasks. We have added the evaluation protocols and results to Appendix I.2–I.3 (Tables S6–S7).

  4. Real-robot failure modes and execution time (PBP2 W1, XZP3 W3, ZFE7 Q3) — Across 200 AgileX trials per method, EVA reduces command-limit violations from 13 to 2 and visual degradation–related failures from 28 to 5 compared with Base. We have added failure definitions and counts to Appendix I.3 (Table S8), together with the 150-second average video-generation, IDM-decoding, and action-execution cycle, cycle-based task-duration estimates, and termination conditions.

  5. Context sampling and closed-loop deployment (PBP2 Q1) — We have clarified in Section 4.3 and Appendix C that RL contexts pair observations from randomly sampled intermediate demonstration states with task instructions. During deployment, the policy replans from the latest observation after each action chunk. Our response discusses the training–deployment context difference and cites the seen/OOD hardware results in Figure 5 as empirical support for generalization.


Finally, we thank the reviewers again for their time and consideration and welcome further questions or suggestions.

Best regards,
The Authors

AuthorsOpenReview ↗

Willing to Address Additional Comments

Dear reviewers,

We would like to highly appreciate all reviewers' efforts and time again in providing valuable comments and constructive suggestions for improvement of our submission. We hope that the clarifications and additional evaluations provided in the responses have addressed all reviewers' questions and concerns.

We are always ready to provide additional clarifications should you have any questions and concerns during the discussion period, due on 18th Nov, only two days remain.

Thank you very much!

The Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.