acceptodds
Under review as a conference paper at ICLR 2027

Beyond Successful Robot Demonstrations: Learning from Human Interactions and Modeling Failure Dynamics

Abstract

Action-conditioned world models offer a promising foundation for generalist robotic intelligence by predicting the consequences of actions. However, existing robotic datasets provide limited coverage of the diverse interaction dynamics required for generalizable world modeling. Large-scale egocentric human videos offer complementary interaction experience, yet the embodiment gap between human hands and robotic manipulators hinders effective cross-embodiment learning. Meanwhile, the predominance of successful demonstrations leaves failure dynamics substantially underrepresented, limiting the ability of world models to anticipate unsuccessful executions. To address these limitations, we introduce HF-WM, an action-conditioned world model that combines Human interaction learning with Failure-aware post-training. To bridge the embodiment gap, we develop a dual-branch framework that couples video generation with auxiliary pointflow prediction, using stabilized pointflow as a unified motion representation across human and robot interactions while suppressing egocentric camera motion. To improve failure modeling, we further perform Group Relative Policy Optimization (GRPO) post-training on perturbed actions derived from successful demonstrations, guided by action alignment and physical plausibility rewards, enabling the model to learn plausible failure dynamics without paired ground-truth videos. Extensive experiments demonstrate that HF-WM achieves state-of-the-art performance in action-conditioned video generation and exhibits strong utility for policy evaluation in downstream robotic tasks.

Primary area:
applications to robotics, autonomy, planning
Keywords:
world model, Controllable Video Generation, Reinforcement Learning

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_CgPSOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper is motivated by the difficulty of training robot world models to predict diverse interactions outcomes from datasets dominated by successful demonstrations. HF-WM combines video generation with stabilized pointflow supervision to learn from human interaction videos while addressing embodiment differences and camera motion. It then applies GRPO to perturbed actions, using action alignment and physical plausibility rewards to improve failure modeling without paired failure videos. Experiments on successful and failed robot trajectories show improved video quality and action following. Evaluation across six robot policies also shows close agreement between predicted and real world success rates and rankings.

Critical strengths

Improving world models' predictions of unsuccessful interactions is an important research problem. Perturbing successful demonstrations and applying reward-guided post-training offers an interesting way to address it without real failure videos.

Critical weaknesses
  • The paper motivates its approach by using human interaction data and failure-aware training to improve interaction dynamics modeling. Although the experiments show gains, this could come from better prediction of the robot's own motion or its effects on objects or the environment. Predicting the robot's motion is a more constrained problem because it is already largely specified by the input skeleton sequence, and additional data for this purpose should be relatively straightforward to collect or synthesize. PPredicting how objects respond to robot actions is more important for robotic manipulation and provides a stronger motivation for learning from human interactions. However, the current quantitative metrics and ablations do not distinguish how much improvement occurs in this capability, which is my main concern. The downstream experiment in the appendix (figure 12) provides some supporting evidence, but the improvement is also incremental.

  • Figure 5 shows a good correlation between predicted and real world policy success rates, but no comparisons with other world models. It is therefore unclear whether HF-WM offers an improvement over existing methods as a policy evaluator.

Questions for authors
  • Could the authors clarify whether the actions used during training represent commands sent to the robot or the robot's actual executed trajectories?
  • Figure 9 requires geometric perturbations to preserve the original contact, grasp and release events. Since translating or rotating a trajectory can change whether contact occurs, this requirement might penalize valid failure predictions. Could the author clarify this design?
5 comments

Q1: Clarification on Action Conditioning in HF-WM

We thank the reviewer for this question. HF-WM is an action-conditioned world model that predicts future robot motion and scene dynamics from an initial observation and a numerical action sequence. Specifically, the numerical actions are converted into 2D skeletons using URDF-based forward kinematics and camera calibration parameters, which then condition future video generation. The input actions represent commanded motions rather than measured executed trajectories. The conditioning skeletons are computed entirely from numerical actions, without using any information from future video frames. The recorded future videos serve only as training supervision. While the action skeletons provide spatial constraints on robot motion, they do not directly specify its visual realization. The model must still synthesize temporally coherent robot motion, including appearance, pose changes, and occlusions, while ensuring consistency with the given actions.

Q2: Clarification on Interaction Preservation under Geometric Perturbations

We thank the reviewer for pointing out this concern. We agree that translating or rotating the end-effector trajectory may alter the original contact relationships. To mitigate this issue, we do not apply geometric perturbations uniformly across all trajectories. Instead, we determine the perturbation type and timing based on the gripper event patterns in the original trajectories, aiming to avoid interfering with critical interaction events. Specifically, as shown in Table 4 of the main paper, we first categorize trajectories into seven classes based on gripper-state transitions. For the four classes with explicit gripper transitions, we schedule geometric perturbations after the final gripper-state transition to avoid introducing spatial deviations during critical grasping and release phases. Perturbations are applied after the closing event for Close hold (Open→Closed), after reopening for Close reopen (Open→Closed→Open), after opening for Open hold (Closed→Open), and after the final closing event for Open close (Closed→Open→Closed). This temporal scheduling preserves the key gripper events in the original trajectories, while applying geometric perturbations primarily to the subsequent motion segments. For the remaining three classes without explicit gripper-state transitions, namely Remain open, Partial hold, and Remain closed, there are no explicit grasping or release transitions that require temporal protection. Therefore, geometric perturbations can be applied throughout the motion window while keeping the original gripper-state sequence unchanged. This design is also reflected in the sampling strategy in Table 4. For the four classes with explicit gripper events, we sample unperturbed actions, gripper perturbations, and geometric perturbations with probabilities of 50%, 35%, and 15%, respectively. For the other three classes, we do not apply gripper perturbations and increase the geometric perturbation probability to 50%. The rationale is that, for trajectories with explicit interaction events, we primarily use gripper perturbations to modify interaction conditions and explore alternative object interaction outcomes, while geometric perturbations are used to study spatial motion variations while preserving the original interaction process. For trajectories without explicit gripper events, geometric perturbations are instead used to increase spatial motion diversity. Thus, the two perturbation types serve complementary purposes: geometric perturbations are scheduled to avoid critical gripper events and preserve the original interaction process, whereas gripper perturbations directly modify interaction conditions to explore counterfactual outcomes of grasping, holding, and releasing. We will clarify this design and its motivation in the revised manuscript.

W1: Object-Centric Evaluation of Interaction Dynamics

We thank the reviewer for raising this important question. An action-conditioned world model should predict both commanded robot motion and its effects on manipulated objects and the environment. The existing IDM metric mainly measures action consistency, while full-frame reconstruction metrics do not isolate object-level appearance and dynamics. We therefore report an object-centric evaluation of object appearance, spatial state, and motion.

Evaluation protocol. For region-specific evaluation, we use GPT-5 for initial localization and SAM3 for frame-level tracking to obtain manipulated-object and robot-arm masks exclusively from the ground-truth videos. We apply the same masks to every method without re-detecting or re-segmenting generated outputs. We report PSNR and SSIM for the manipulated object, robot arm, and full frame. For the object region, N-EPE measures RAFT optical-flow endpoint error within ground-truth object-motion regions; lower is better. For the robot arm, we report the Franka-specific IDM error defined in the main paper.

Table Manipulated-object, robot-arm, and full-frame evaluation. Region-specific metrics are computed using masks derived from ground-truth videos.

MethodManipulated-object PSNR ↑\uparrowManipulated-object SSIM ↑\uparrowManipulated-object N-EPE ↓\downarrowRobot-arm PSNR ↑\uparrowRobot-arm SSIM ↑\uparrowRobot-arm IDM ↓\downarrowFull-frame PSNR ↑\uparrowFull-frame SSIM ↑\uparrowFull-frame LPIPS ↓\downarrow
OSCAR25.5620.8480.0011714.3210.48000.57923.3570.8400.142
Kinema4D23.7230.8250.0013512.0690.34110.91420.0150.8040.157
Cosmos-Predict2.523.3020.7220.0018111.5640.39671.26818.8740.7460.266
Ctrl-World21.7970.6070.0011614.0740.44020.94921.6830.7700.305
Cosmos319.7760.5520.0014411.4210.38341.10820.2330.7660.178
TesserAct12.6170.4760.0049510.0430.31841.66317.3830.7570.263
Ablations
HF-WM w/o human data24.1850.7870.00091115.0830.53690.40621.5730.8350.117
HF-WM (SFT-only)27.1960.8810.00088815.1070.55160.35525.8280.8870.086
HF-WM (with GRPO)28.4180.8940.00085415.6760.55360.33226.0770.8890.082

Results. Both HF-WM (SFT-only) and HF-WM (with GRPO) outperform all six competing methods across all three manipulated-object metrics, and also lead in robot-arm reconstruction, action fidelity, and full-frame quality. More importantly, the ablations directly validate our two core design choices. The consistent gains from incorporating human interaction data demonstrate that our human-to-robot learning framework effectively transfers interaction knowledge beyond robot-only training, improving the prediction of object-state transitions and motion dynamics. GRPO further improves every reported metric over SFT-only, showing that failure-aware post-training strengthens not only action adherence but also the modeling of interaction-induced physical consequences. Together, these broad and complementary gains establish HF-WM's clear advantage at the manipulated-object, robot-arm, and full-frame levels, demonstrating that it more faithfully predicts both commanded robot motion and its downstream effects on the environment.

We would also like to clarify that Figure 12 in the appendix is a supplementary experiment intended to demonstrate that HF-WM's predictions can support test-time action selection and improve closed-loop performance. In our LIBERO-90 setup, this pipeline increases the success rate from 27.78% to 30.0% without updating the base policy. The low baseline success rate indicates that these tasks remain challenging for the evaluated policy. Although HF-WM predicts the outcomes of candidate actions, the benefits of selection also depend on the quality of the candidates generated by π0.5\pi_{0.5} and the accuracy of RoboMeter's scoring. The magnitude of this end-to-end gain therefore should not be treated as a direct measure of HF-WM's predictive capability.

Our primary downstream validation is policy evaluation, which more directly assesses how accurately HF-WM predicts real-world policy performance. As shown in the table accompanying our response to W2, under the same evaluation protocol, HF-WM achieves a Pearson correlation of 0.983, compared with 0.345 for OSCAR and 0.142 for Ctrl-World. Its success-rate MAE is 2.41 percentage points, representing reductions of approximately 31.6% and 74.5%, respectively. These comparisons provide direct evidence of HF-WM's effectiveness in our primary downstream setting and its consistent advantages over the evaluated baselines.

W2: Additional Comparison Experiments on Downstream Tasks

We thank the reviewer for this suggestion. To further strengthen the comparison on downstream tasks, we selected OSCAR and Ctrl-World, two relatively strong and representative baselines from Table 5. Following exactly the same evaluation protocol as HF-WM, we evaluated each model on the same 90 RoboArena samples across six robot policies, resulting in 90×6=54090 \times 6 = 540 rollouts per model. We report the task success rate for each policy and compute both Pearson's correlation coefficient and success-rate MAE (in percentage points) between the six-dimensional policy-level success-rate vector predicted by each world model and its real-world counterpart. As shown in the table, HF-WM achieves both the highest Pearson correlation (0.9830.983) and the lowest success-rate MAE (2.412.41 percentage points), outperforming OSCAR (0.3450.345, 3.523.52 points) and Ctrl-World (0.1420.142, 9.449.44 points). Although OSCAR obtains relatively high absolute success rates for several policies, its cross-policy performance trend is less consistent with the real-world results. Ctrl-World, meanwhile, substantially underestimates the overall task success rates. These results demonstrate that HF-WM more faithfully captures both the relative performance trend across robot policies and their absolute real-world success rates.

Table Comparison with additional world-model baselines on RoboArena. Each model is evaluated on the same 90 samples across six robot policies, resulting in 540 rollouts per model. For each world model, the value in parentheses is the signed deviation from the corresponding real-world result in percentage points. Negative and positive deviations denote underestimation and overestimation, respectively, and smaller absolute deviations are better.

Modelπ0\pi_{0}-FAST success rate (%)PG-FAST success rate (%)PG-FAST+ success rate (%)PG-FSQ success rate (%)PG-flow success rate (%)π0\pi_{0}-flow success rate (%)Pearson's rr ↑\uparrowSuccess-rate MAE (pp) ↓\downarrow
Real World20.011.113.310.010.04.4----
OSCAR13.3 (-6.7)10.0 (-1.1)11.1 (-2.2)10.0 (0.0)14.4 (+4.4)11.1 (+6.7)0.3453.52
Ctrl-World2.2 (-17.8)2.2 (-8.9)2.2 (-11.1)1.1 (-8.9)2.2 (-7.8)2.2 (-2.2)0.1429.44
HF-WM (Ours)15.6 (-4.4)8.9 (-2.2)11.1 (-2.2)8.9 (-1.1)6.7 (-3.3)3.3 (-1.1)0.9832.41
Reviewer_K4D3OpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

HF-WM learns robot dynamics from human videos through stabilized point-flow supervision, then applies GRPO to perturbed robot actions using action-alignment and physical-plausibility rewards. It evaluates video prediction, policy-success estimation, and world-model-guided action selection.

Critical strengths

The ablations support the human-video training design. Human data raises PSNR from 21.448 to 24.997; adding point-flow supervision and stabilization raises it to 25.828 (Table 2). The intermediate unstabilized variant reaches 25.294, isolating the benefit of camera-motion correction. The model removes the auxiliary flow branch at inference.

The failure experiments support GRPO’s contribution to video prediction. The evaluation includes 100 real failed executions (Section 4.2). On the combined evaluation set, full GRPO reduces LPIPS from 0.086 to 0.082 and IDM error from 0.355 to 0.332 (Table 3). The project page’s GRPO and individual-reward ablations illustrate how post-training changes generated failure outcomes (§04–05).

Critical weaknesses

The downstream evaluation does not isolate GRPO’s added value. The paper and project page show policy-outcome agreement and gains from HF-WM-based action selection (Section 4.4; project page §02). Neither compares SFT-only and post-GRPO world models in the same downstream pipeline. The video-generation ablation therefore leaves unclear how much failure-aware post-training contributes to policy assessment or action selection.

Physical-reward reliability remains a minor uncertainty. The project page’s selected/rejected candidates and reward-only examples support the design (§05). They do not quantify the physical evaluator’s agreement with independent judgments on contact/support violations. Because severe-violation decisions can suppress the action reward (Section 3.3), a small audit would clarify reliability beyond these examples.

Questions for authors

Using the same 540 policy-session traces and success evaluator, how do SFT-only and post-GRPO checkpoints compare in success-rate MAE and episode-level outcome errors?

Do independent judgments agree with the physical reward on a subset of selected and rejected candidates, particularly for severe contact or support violations?

1 comment

Q1: Downstream Contribution of Failure-Aware GRPO

We thank the reviewer for this important suggestion. To isolate the downstream contribution of failure-aware GRPO, we evaluate the SFT-only and post-GRPO checkpoints on the same 540 policy-session traces using the same GPT-5 success evaluator. The table below reports each policy's predicted success rate, Pearson's correlation, and the success-rate mean absolute error (MAE) in percentage points.

Table Controlled downstream comparison of the SFT-only and post-GRPO checkpoints on the same 90 sessions for each of six policies (540 rollouts per checkpoint). The values in parentheses are signed deviations from the corresponding real-world success rates in percentage points. Negative and positive deviations denote underestimation and overestimation, respectively, and smaller absolute deviations are better.

Modelπ0\pi_{0}-FASTPG-FASTPG-FAST+PG-FSQPG-flowπ0\pi_{0}-flowPearson's rr ↑\uparrowSuccess-rate MAE ↓\downarrow (pp)
Real world20.011.113.310.010.04.4----
HF-WM (SFT-only)13.3 (-6.7)7.8 (-3.3)11.1 (-2.2)6.7 (-3.3)4.4 (-5.6)3.3 (-1.1)0.9313.70
HF-WM (post-GRPO)15.6 (-4.4)8.9 (-2.2)11.1 (-2.2)8.9 (-1.1)6.7 (-3.3)3.3 (-1.1)0.9832.41

Post-GRPO reduces the success-rate MAE from 3.70 to 2.41 percentage points (a 35.0% relative reduction) and increases Pearson's rr from 0.931 to 0.983, demonstrating more accurate success-rate estimation and cross-policy performance trends. We have added the corresponding figure to Sec. 4.4 as Fig. 5 and expanded the policy-evaluation analysis in Appendix C.1 of the revised manuscript. For clarity, the titles of these two revised sections are highlighted in blue.

Q2: Independent Validation of Physical-Reward Reliability

We thank the reviewer for this important suggestion. We conducted an additional audit of physical-reward reliability on 50 groups of candidates. Each group contained four videos generated from the same initial observation and action sequence, yielding 200 videos and 300 within-group candidate pairs. Human annotators were shown only the task description and candidate videos, without access to physical scores, selected/rejected labels, or perturbation labels.

In addition to human evaluation, we used an external physical evaluator based on the EZS-Bench protocol. EZS-Bench assesses the physical plausibility of robot manipulation videos through physics-oriented question answering, covering contact, interpenetration, deformation, and temporal consistency. These aspects align closely with the interaction physics examined in our audit. We measured within-group pairwise ranking agreement, defined as the fraction of candidate pairs for which HF-WM's physical reward and an independent assessment agree on the preference direction. Besides the full set of 300 pairs, we examined a subset of 80 pairs identified by human annotators as involving severe contact or support violations, including object interpenetration, unphysical adhesion, and unsupported floating. Both evaluations measure agreement in candidate rankings, as reported below.

Table Pairwise ranking agreement between HF-WM's physical reward and independent assessments.

Independent assessmentAll candidate pairs (n=300n=300) ↑\uparrowPairs involving severe contact/support violations (n=80n=80) ↑\uparrow
Human evaluation89.3%97.5%
EZS-Bench85.3%96.3%

HF-WM's physical reward achieves 89.3% agreement with human judgments overall and 97.5% on the subset involving severe violations, with 96.3% agreement with the external evaluator on the latter. These results provide independent quantitative support for the accuracy of its candidate rankings, particularly for candidates involving severe physical violations, and for its use as a relative quality signal in GRPO.

Reviewer_ev3HOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper fine-tunes Cosmos 3 Nano into an action-conditioned world model (HF-WM) in two stages. Stage 1 uses the 2D skeleton conditioning from OSCAR so that human hands and robot arms share one action space, trains on about 45K Franka trajectories plus 83K egocentric human clips, and adds an auxiliary point-flow branch. The branch reads intermediate video latents from layers 4/6/8/10, predicts future point trajectories, and is dropped at inference. Point flows on human videos are first "stabilized" by subtracting the motion explained by depth and camera pose, so that the supervision reflects hand-object motion rather than head motion. Stage 2 perturbs the actions of successful demonstrations (gripper timing/state or end-effector pose, chosen by a seven-way gripper-event classification with IK checks) and runs Flow-GRPO on the perturbed actions. Since no ground-truth video exists for a perturbed action, the reward is a mix of two learned signals: a pairwise-trained action-alignment scorer (weight 0.4) and a Gemini-based tiered physics checklist (weight 0.6). Any Tier-1 failure caps the reward. Evaluation is on 100 held-out DROID trajectories and 100 real failed executions. HF-WM leads on PSNR/SSIM/LPIPS/FVD/FID/L2-latent/IDM against six baselines, and the gap between success and failure subsets is smaller than for the baselines (Table 5). Ablations cover human data, point flow, stabilization, branch depth, the two rewards, and the perturbations. The model is also used as a policy evaluator on six RoboArena policies (Pearson r = 0.983 over 90 sessions) and for test-time action selection on LIBERO-90 (27.8% to 30.0%).

Critical strengths
  1. The problem is real and well motivated. Robot datasets are almost all successes, and a world model that only knows successes is a poor policy evaluator. The paper collects 100 real failed executions and shows that every baseline degrades on them (Table 5). That observation alone is useful to the community.
  2. The perturbation pipeline in Appendix A.2 is carefully built. Gripper events are classified with hysteresis thresholds, perturbation routes are sampled with explicit quotas (Table 4), geometric perturbations go through IK with joint-limit and continuity checks, and the skeleton is re-rendered and checked for visibility. This is more disciplined than most "we add noise to actions" setups.
  3. The point-flow branch is designed so it cannot shortcut through the action tokens (Fig. 6). Masking the skeleton/action keys from the branch is a small but sensible choice, and the layer-depth ablation (Table 6) supports attaching it early.
Critical weaknesses

W1. The central claim, that the model learns failure dynamics, is not tested directly. The physics reward is explicitly designed not to treat task failure as a violation (Sec. 3.3, A.2.2), and the action-alignment reward only checks that the robot follows the commanded skeleton. So neither reward asks the model to predict that a cup falls when the gripper opens early. Whether GRPO actually produces correct failure outcomes rests on one qualitative example (Fig. 11). The 100 real failure clips give aggregate PSNR/FVD, which show the videos look more like the recording, but no metric tells us whether the model distinguishes a failing action from a succeeding one. MiraBench (which the paper cites) has an optimism-bias protocol, and CureWM / FACT report success-vs-failure discrimination. Something along those lines is needed: for the same start frame, feed the original and a perturbed action, and measure whether the predicted outcomes separate (e.g. AUROC of a success score, or human judgments of the predicted outcome).

W2. The gains from GRPO, and especially from the perturbations, are small and reported without variance. Table 3: PSNR 25.83 to 26.08, LPIPS 0.086 to 0.082, IDM 0.355 to 0.332. Table 7 shows "GRPO w/o action perturbations" reaches 26.319/0.893/0.066 on the success subset, essentially the same as the full model. The perturbations only move the failure subset (PSNR 25.43 to 25.83, FID 15.74 to 14.27). Everything is single-seed with no confidence intervals, so it is not possible to tell which of these differences are real. Also, Table 3 shows action-alignment-only equals w/o GRPO to the third decimal (25.826 vs 25.828), yet Sec. 4.3 says this reward "preserves action-consistent motion but compromises temporal consistency". The text and the table do not agree.

W3. The baseline comparison does not isolate the contribution of the method.

  • The Cosmos 3 row in Table 1 appears to be the untuned model (the "embodied data only" variant in Table 2 already beats it). OSCAR is built on Cosmos-Predict2.5-2B. There is no run of the OSCAR recipe on Cosmos 3 Nano with the same data, so gains from a stronger backbone and more data (45K + 83K) cannot be separated from gains from the method.
  • The paper's own "embodied data only" model (PSNR 21.4) is below OSCAR (23.4); most of the jump comes from adding human data, which OSCAR also uses. This suggests the comparison is mainly about data and backbone.
  • TesserAct and Cosmos-Predict2.5 are text-conditioned. Comparing their PSNR on action-conditioned generation says little.
  • The IDM metric uses an inverse-dynamics model fine-tuned from Cosmos 3, and L2-latent uses the Wan2.2 VAE, both shared with HF-WM's backbone. This may favor the proposed model. W4. The 100 failure trajectories are not described. Which policy produced them, which lab/scene, how they relate to the DROID distribution, and whether the scenes overlap with the GRPO training clips are all unstated. Since every baseline drops on this set, the drop could be scene shift rather than failure dynamics. A control of successful trajectories from the same scenes would settle this. W5. The policy-evaluation experiment does not connect back to the paper's motivation.
  • n = 6 policies, with real success rates between 3.3% and 20%. A Pearson r of 0.983 on six points in such a narrow range is weak evidence. MMRV = 0.002 is nearly guaranteed when the gaps between policies are this small.
  • There is no ablation here: does the GRPO stage, or the human data, change the correlation? The paper argues that failure modeling matters for policy evaluation, but this is never measured.
  • No world-model baseline (OSCAR reports the same style of virtual evaluation; Ctrl-World does too) is run under the same protocol.
  • The LIBERO-90 result (27.78% to 30.0%) lacks episode counts and confidence intervals, and the gain could come from RoboMeter alone.

W6. Reward reliability is not validated, and reward hacking is not discussed. GRPO rollouts are 256x256, 12 denoising steps, 40 frames. A VLM judging interpenetration or unsupported floating on videos of this quality is unverified: no agreement with human raters is reported, and the action-alignment scorer's held-out accuracy is not given. Other GRPO-on-world-model papers (EVA, PAVXploreRL) report reward hacking; this paper shows no training curves or failure cases of the reward. Two closed-source models (Gemini 3.7 Flash for rewards, GPT-5 for evaluation) are in the loop, which affects reproducibility.

W7. Closely related work is missing, and part of the framing is borrowed. The related-work section is two paragraphs. Work that overlaps directly and is not discussed includes: PAVXploreRL (RL post-training with physical-plausibility and action-adherence rewards on perturbed OOD actions without paired video; nearly the same setup), ReWorld and RoboAlign-R1 (reward-model-based post-training of robot world models), EVA (IDM reward + GRPO), RLVR-World, GrndCtrl/RLWG, Dream2Fix (counterfactual failure synthesis by perturbing actions in a world model with a physical verifier), FACT and CureWM (failure data for world models), and JOPAT (point tracks as an auxiliary modality in a world-action model). The introduction lists the skeleton representation as the paper's first insight, but Sec. 3.2 and App. B.1 say it is taken from OSCAR together with its human-data injection strategy. The paper should state plainly which parts are new: the stabilized point-flow branch, the perturbation pipeline, and the specific reward design.

W8. "Learning across embodiments" is not evaluated across embodiments. All results are on Franka. OSCAR reports per-embodiment results on five robots; this paper reports none. There is also no control for data quantity: adding 83K human clips helps, but adding 83K more robot clips might help as much.

W9. Stabilization depends on camera poses "provided by the dataset". EPIC-Kitchens does not ship per-frame camera poses. The paper should say where the poses came from and whether the method still works with estimated poses, since that determines whether it scales to in-the-wild video.

Questions for authors

See Weakneess.

10 comments

W1: Direct Evaluation of Failure-Outcome Prediction

We thank the reviewer for this insightful suggestion. To directly assess whether GRPO improves the modeling of failure dynamics, we conduct a counterfactual evaluation under matched initial conditions, following the evaluation protocol of the concurrent work CureWM. For each test sample, we start from the same LIBERO simulator state and initial observation, and construct two action conditions: the original successful action and a perturbed action whose failure outcome is verified by the simulator. Both SFT-only and GRPO-post trained HF-WM models generate the corresponding 81-frame future videos. Each generated video is independently evaluated three times by GPT5 in a blind setting to obtain a predicted success score, with simulator outcomes serving as ground-truth labels. We evaluate success–failure discrimination using AUROC, the mean success–failure score gap (ΔSF\Delta_{\mathrm{SF}}), and pairwise ranking accuracy (Pair-ranking).

Across 100 counterfactual pairs, GRPO improves AUROC from 0.553 to 0.781, ΔSF\Delta_{\mathrm{SF}} from 0.015 to 0.198, and Pair-ranking accuracy from 31% to 75%. These results demonstrate that GRPO enables the model to better distinguish successful and failed outcomes induced by different actions under matched initial conditions. In particular, the substantial improvements in the success–failure score gap and pairwise ranking accuracy demonstrate that GRPO enables the model to better capture the distinct outcomes induced by counterfactual action perturbations. These results validate the effectiveness of our proposed failure-aware post training design, showing that combining counterfactual action perturbations with reward-guided optimization improves the modeling of action-conditioned failure dynamics.

Table Counterfactual success–failure discrimination under matched initial conditions.

ModelAUROC ↑\uparrowΔSF\Delta_{\mathrm{SF}} ↑\uparrowPair-ranking ↑\uparrow
HF-WM (SFT-only)0.5530.01531%
HF-WM (GRPO)0.7810.19875%

W2: Interpretation of Experimental Results

We thank the reviewer for the insightful comments. We would like to clarify the distinct contributions of GRPO and action perturbations based on the results reported in the main paper.

First, the primary benefit of GRPO lies in improving failure prediction, with more pronounced gains on failed executions than in the overall averages. As shown in Table 7 in the main paper, compared with the model without GRPO, the full GRPO model improves both generation quality and action consistency on the failure subset, reducing FID and IDM by 16.0% and 10.6%, respectively. These gains directly reflect the objective of GRPO post-training: improving the modeling of failure dynamics. Thus, the modest change in overall PSNR does not fully capture its contribution.

Second, action perturbations further improve failure prediction, and the similar performance on successful cases does not diminish this contribution. As shown by the comparison between “GRPO w/o action perturbations” and “GRPO (full)” in Table 7, the two configurations achieve nearly identical results on the success subset, while the full model improves all reported metrics on the failure subset, with LPIPS, FID, and IDM reduced by 7.5%, 9.3%, and 10.2%, respectively. Since action perturbations are designed to expand action-condition coverage beyond successful demonstrations, their concentrated gains on failed executions are consistent with this objective. We therefore consider improved failure prediction, rather than marginal changes on successful cases, the primary contribution of this component.

Finally, similar PSNR values do not necessarily imply equivalent temporal consistency or interaction plausibility. PSNR measures frame-level reconstruction error and cannot independently assess temporal consistency in background regions or the physical plausibility of object interactions. The qualitative observations in Sec. 4.3 and Fig. 10 concern intermediate rollouts during GRPO training, whereas Table 3 in the main paper reports aggregate quantitative results. As Table 3 shows, action alignment alone leaves overall reconstruction metrics nearly unchanged while slightly improving IDM. In contrast, combining action alignment with physical plausibility achieves the best results across all reported metrics, supporting the complementary roles of the two rewards.

W3: Clarification on the Fairness of Comparative Experiments

1. HF-WM uses fewer downstream training trajectories than OSCAR. HF-WM uses approximately 45,000 robot trajectories and 83,000 human interaction trajectories, totaling 128,000. OSCAR reports 94,830 robot trajectories and 85,827 human interaction trajectories, totaling 180,657. Based on these reported counts, HF-WM uses approximately 29.148% fewer downstream training trajectories. Thus, attributing the improvement to a larger downstream training set is inconsistent with the experimental setup.

2. Table 2 in the main paper already includes the requested control with the same backbone and training data. The “+ human” configuration follows OSCAR's training recipe on Cosmos 3 Nano, using the same robot and human training data as full HF-WM, without PointFlow or GRPO. Starting from this baseline, which already incorporates the benefits of human data, stabilized PointFlow increases PSNR from 24.997 to 25.828, and GRPO further improves it to 26.077; IDM error decreases correspondingly from 0.391 to 0.355 to 0.332. These gains are obtained with the backbone and training data held fixed, directly demonstrating the additional contributions of the proposed components. Comparing “embodied data only” with OSCAR, which also uses human data, involves different training conditions and cannot negate these contributions. The relevant evidence for method attribution is provided by the controlled ablations in Tables 2 and 3 of the main paper.

3. Text-conditioned models provide reference baselines for comparing conditioning approaches. Several action-conditioned world modeling studies use text-conditioned reference baselines: OSCAR reports PSNR for both TesserAct and Cosmos-Predict2.5; Kinema4D and SkeLAM report PSNR for TesserAct; UMI-WM reports PSNR for text-conditioned Cosmos-Predict2.5; and EA-WM includes both models in its quantitative comparisons. We include these baselines to examine how different conditioning approaches reproduce specific robot executions. High-level text conveys task goals but typically leaves motion trajectories, gripper states, and interaction timing underspecified, whereas explicit action conditioning directly constrains these details. Here, PSNR measures agreement between generated videos and recorded executions, providing a reference for assessing the limitations of text control and the value of action conditioning for precise execution prediction.

4. HF-WM remains the best-performing method under alternative evaluators. Attributing HF-WM's superior performance to evaluation bias solely because of shared backbone components is not justified. L2-latent re-encodes the final RGB videos from all methods using the same frozen VAE, without using HF-WM's internal latents. Likewise, IDM is a frozen evaluator applied uniformly to all methods. It takes only the final RGB videos as input, without access to model identities, generator weights, or intermediate features. The evaluation therefore uses no HF-WM-specific internal information.

We also directly tested this concern by replacing the evaluators, and HF-WM remained the best-performing method. With the Wan2.1 VAE, its latent MSE is 0.070, compared with 0.101 for the strongest baseline, OSCAR. With the open-source VERA-DROID IDM, HF-WM again achieves the best result (0.923). As reported in Table 1 of the main paper, it also ranks first on PSNR, SSIM, LPIPS, FVD, and FID, which do not depend on either evaluator. Thus, HF-WM's SOTA performance persists under alternative evaluators and cannot be explained solely by the original evaluators' relationship to its backbone.

Table Comparison under alternative evaluators. Latent MSE is computed using the Wan2.1 VAE, and IDM error is measured with the open-source VERA-DROID model. Lower is better; bold values indicate the best results.

MetricOSCARKinema4DCtrl-WorldCosmos3Cosmos-Predict2.5TesserActHF-WM
Wan2.1 latent MSE ↓\downarrow0.1010.1260.1610.1660.2270.3220.070
VERA-DROID IDM ↓\downarrow1.0951.3981.3991.5311.7162.1650.923

W4: Real-World Failure-Set Provenance and Controlled Counterfactual Evaluation

We thank the reviewer for pointing out that the real-world failure test set was insufficiently described. We clarify that the 100 real-world failure trajectories were collected independently in our laboratory from failed rollouts of π0.5\pi_{0.5} on real-world manipulation tasks. The experiments were conducted using a Franka Research 3 robot arm equipped with a Robotiq 2F-85 gripper. These trajectories are independently collected real-robot execution records and are neither from the DROID dataset nor part of the GRPO training videos.

Regarding the lack of paired successful and failed trajectories, π0.5\pi_{0.5} was unable to successfully complete some relatively challenging manipulation tasks. Consequently, we could not obtain corresponding successful executions for these failure trajectories during real-world data collection. This was not an intentional omission of paired successful data, but rather a limitation imposed by the actual execution capabilities of the policy and the practical conditions of real-world data collection. We agree that evaluating exclusively on these real-world failure trajectories cannot fully rule out the influence of scene distribution shifts on model performance.

To further control for this potential confounding factor and directly evaluate failure-dynamics modeling, we conduct the controlled counterfactual evaluation described in our response to W1. In the LIBERO simulation environment, we construct 100 successful–failed action pairs, starting from identical initial states and observations, where the failed actions are verified by the simulator. We compare the corresponding future video predictions from SFT-only and GRPO-post trained HF-WM. The results show that GRPO improves AUROC from 0.553 to 0.781, the success–failure score gap from 0.015 to 0.198, and pairwise ranking accuracy from 31% to 75%. Since each successful–failed action pair shares the same initial scene, this evaluation directly measures the model's ability to distinguish different action outcomes while controlling for initial scene variations.

In summary, the real-world failure trajectories evaluate the model's predictive performance on actual robot failures, while the paired LIBERO counterfactual evaluation further verifies the contribution of GRPO to failure-dynamics modeling. Together, these experiments provide complementary evidence from real-world generalization and controlled action-outcome discrimination.

Comments on the paper

1 comment

Dear Reviewers,

We are approaching the deadline of the discussion period (Nov. 18). Please go through authors' rebuttal at your earliest convenience, and engage in a discussion or leave any follow-up comments.

Thank you for your time and participation.

AC

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.