- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper studies failures of embodied video world models in robot manipulation, where the manipulated target is spuriously duplicated, disappears, or deforms across frames. The authors refer to violations of target-instance count conservation as “Model Laziness” and propose EVEWorld, which consists of two components. Instance-Guided Restoration (IGR) copy-pastes an additional target instance into demonstration videos and trains the model to restore the clean video with a region-weighted denoising objective. Temporal Instance Alignment (TIA) performs local feature matching, cross-frame feature transport, and correspondence supervision at a selected Transformer layer. The detector and target localizations serve only as offline training supervision; at generation time, TIA operates directly on the model’s hidden features.
The authors further propose the Model Laziness Rate (MLR), which uses the initial target count as its reference, exempts under-counts caused by robot occlusion, and records a violation only when it persists over two consecutive sampled frames. The main experiments fine-tune GigaWorld-0 on 92 GR1 videos and evaluate on DreamGenBench, with additional experiments on WorldArena, EWMBench, PBench, and FlowWAM/RoboTwin. Compared with a Standard SFT baseline trained under the same budget, EVEWorld reduces MLR on DreamGenBench from 7/63 to 1/63 and improves Gemini-IF from 53.57% to 60.85%. On WorldArena 1.0, MLR drops from 43/157 to 21/157.
Critical strengths
-
A well-motivated problem with a targeted solution. The paper studies a failure mode of clear practical relevance: when world models are used as data engines or policy evaluators, duplicated or vanishing objects are a real obstacle. The motivation is concrete, and each component maps directly onto the targeted failure. IGR turns an extra instance and its clean counterpart into a paired restoration task. TIA addresses the cross-frame inconsistencies that can remain even when the instance count is preserved.
-
Broad evaluation with well-matched controls. The main paper evaluates on multiple benchmarks that cover in-domain performance, cross-domain and cross-distribution generalization, a second backbone, and robustness across random seeds. Each comparison uses a Standard SFT control with the same initialization, data, and training budget, so the gains can be attributed to the proposed supervision rather than to the training setup.
Critical weaknesses
-
IGR may delete legitimate instances, and the full method does not seem to transfer well to other backbones. IGR is trained only to remove synthetically added instances, yet the paper reports no retention or false-deletion rates in real multi-instance scenes (N₀ > 1), nor whether the evaluation set contains such cases. Moreover, although Sec. 5.3 cites FlowWAM as evidence of cross-backbone generalization, TIA alone is best on every metric there (MLR 22.54%), and adding IGR degrades all five (MLR 35.21%; Table 5).
-
The core evidence gives limited support to the main conclusions, and the reliability of the current MLR computation is questionable. MLR on DreamGenBench is computed on only 63 eligible samples (Table 1 caption), so the 85.7% and 87.5% reductions correspond to violation events falling from 7 to 1 and from 8 to 1. MLR is also computed automatically with a single detector, GroundingDINO, and the paper reports no quantitative agreement (e.g., precision and recall) with human labels. The lower MLR therefore cannot be clearly attributed to fewer genuine physical violations.
Questions for authors
-
(Clarification) Could the authors analyze whether IGR training causes the model to erroneously delete legitimate instances? Reporting how many of the eligible samples have N₀ > 1 would help assess this.
-
(Suggested experiment) Could the authors report MLR on DreamGenBench with sufficient statistical power, for example by using more random seeds? The same question applies to the WorldArena 1.0 comparison in Table 3.
-
(Suggested experiment) Could the authors validate MLR with human inspection and annotation? As a cheaper alternative, does the ranking of models stay the same when MLR is computed with a different detector that is not used during training?
-
(Clarification) In Table 6, IGR alone lowers overall Gemini-IF (51.85 vs. 53.57 for Standard SFT), and TIA alone raises it only slightly (55.29), yet their combination reaches 60.85, with Env jumping from 51.72 to 72.41. In addition, on FlowWAM (RoboTwin, Table 5), TIA alone performs best on every metric (MLR 22.54%), while adding IGR to TIA degrades all five metrics (MLR 35.21%). Why does combining the two components bring a large gain on GigaWorld-0 but perform worse than TIA alone on FlowWAM?
5 comments
Response to Reviewer Qyz7 Questions
Thank you very much for taking the time to review and for your support. We try our best to address your questions as follows.
Q1. Legitimate-Instance Preservation
IGR reconstructs the clean video, retaining legitimate instances in its training target. Three independent human-raters assess generated videos: 23/100 WA1 and 32/100 WA2 inputs have N0 > 1, yielding 165 videos across three generators per benchmark. W1 gives the loss counts and exclusions for valid visibility changes.
EVEWorld matches SFT's pooled multi-instance loss count (8/55) and reduces disappearance by 60.0% across the full audit (W1; Tables 20-21). IGR-only also lowers total MLR under all five evaluators (Q3); MLR combines duplication and disappearance. As auxiliary scene-quality evidence, WA1 Background Consistency rises from 88.67 to 88.91 and EWMBench Scene from 84.38 to 86.37 (Tables 3-4).
Q2. Additional Seeds and Statistical Support
The improvement holds across all four DreamGenBench inference seeds and in the WorldArena comparison. W2 reports per-seed counts and SD on the same 63 tasks, pooled counts, and WA1's 157-input results with 95% Wilson intervals. DreamGen MLR decreases by 9.13 percentage points (79.3% relative) versus SFT. Seed SD measures variation across repeated generation; Wilson intervals quantify each model's violation rate.
On 60 fixed WA1 tasks with three inference seeds, human judgments give 48/180 violations for SFT and 18/180 for EVEWorld (Table 25). This supplements the original 157-input comparison.
Q3. Human Validation and Alternative Detector
Human consensus confirms fewer EVEWorld violations on all three benchmarks (W2: 978-video audit and precision/recall; Tables 17-23). Raters checked legitimate visibility changes and required violations at two consecutive sampled timestamps.
EVEWorld has the lowest MLR under all five evaluators on 60 WA1 tasks x 3 inference seeds (180 videos/model; Table 25; %):
| Evaluator | Official GR1 | SFT | IGR | TIA | EVEWorld |
|---|---|---|---|---|---|
| Human | 21.11 | 26.67 | 17.22 | 18.89 | 10.00 |
| Gemini | 21.11 | 23.89 | 20.00 | 18.33 | 11.67 |
| Qwen | 20.00 | 26.11 | 19.44 | 19.44 | 12.22 |
| GroundingDINO | 23.89 | 25.56 | 19.44 | 21.11 | 11.67 |
| OWLv2 | 27.78 | 25.00 | 21.11 | 26.11 | 15.56 |
OWLv2 replaces GroundingDINO for target detection/counting at evaluation; GroundingDINO robot localization and SAM2 tracking remain fixed. This comparison uses one human judgment/video, with some seed-004 labels reused from the main audit.
Q4. Cross-Backbone Complementarity of IGR and TIA
The combination improves Overall/Env/Object beyond either component alone (GigaWorld, 250 steps; Table 6; Gemini-IF, %):
| Variant | Overall | Env | Object | Behavior |
|---|---|---|---|---|
| SFT | 53.57 | 51.72 | 42.00 | 67.02 |
| IGR | 51.85 | 49.43 | 36.67 | 69.50 |
| TIA | 55.29 | 55.17 | 42.67 | 68.79 |
| EVEWorld | 60.85 | 72.41 | 50.33 | 64.89 |
EVEWorld gains 5.56/17.24/7.66 points over TIA on Overall/Env/Object; Behavior decreases by 3.90. Env/Object measure instruction following with unseen environments/objects.
IGR learns content restoration; TIA supplies temporal correspondence. Their combination must learn both which instances to preserve and how to propagate them. GigaWorld uses full FT; the original FlowWAM used Rank-32 LoRA. The FlowWAM sweep supports capacity-dependent joint benefits (W1): limited low-rank updates may constrain restoration and alignment; full FT offers greater adaptation capacity.
W1 gives the E7 five-metric comparison. EVEWorld MLR trajectories (%, lower is better; --: unavailable; Tables 52-55):
| Epoch | Rank-32 | Rank-64 | Rank-128 | Full FT |
|---|---|---|---|---|
| E4 | 33.50 | 29.50 | 24.16 | 20.96 |
| E5 | 33.94 | 27.17 | 21.18 | 17.87 |
| E6 | 36.24 | 23.84 | 21.63 | 15.81 |
| E7 | 37.42 | 24.44 | 22.28 | 16.39 |
| E8 | -- | 25.50 | 23.06 | 17.62 |
Under full FT, EVEWorld has lower MLR than TIA at every E4-E8 checkpoint, alongside higher PSNR/SSIM; LPIPS is lower at E4-E7, while TIA retains lower Flow-EPE (Table 55). Table 5 reports E6; W1 uses E7, the last checkpoint shared by all settings.
GigaWorld training dynamics (Table 36; Gemini-IF, %):
| Step | SFT Overall | EVE Overall | EVE Env | EVE Object | EVE Behavior |
|---|---|---|---|---|---|
| 150 | 57.78 | 60.43 | 71.20 | 50.26 | 64.60 |
| 200 | 60.20 | 53.40 | 65.10 | 39.66 | 60.80 |
| 250 | 53.57 | 60.85 | 72.41 | 50.33 | 64.89 |
| 300 | 44.20 | 45.95 | 58.00 | 38.00 | 46.97 |
| 350 | 41.60 | 52.50 | 64.35 | 44.35 | 53.85 |
| 400 | 39.80 | 60.30 | 72.00 | 49.10 | 65.00 |
| 450 | 38.30 | 68.00 | 78.50 | 59.18 | 70.90 |
| 500 | 37.20 | 67.60 | 78.20 | 58.63 | 70.60 |
EVEWorld's IF recovers after step 300 and peaks at 450, while SFT declines. Main results retain the matched 250-step budget. EVEWorld has fewer violations at every scanned step from 150 to 500 (Table 35).
Additional Downstream Utility and Sincere Appreciation
Additional Downstream Utility and Sincere Appreciation
We sincerely thank you for your thoughtful feedback and constructive suggestions, which helped us further strengthen our work. Beyond improving physical consistency in video generation, EVEWorld brings consistent performance gains in downstream policy learning and action planning across Piper real-robot and RoboTwin simulation tasks, demonstrating its practical utility as both a data engine and a world-model-based planner.
1. EVEWorld as a Data Engine
We train downstream π0.5 policies using SFT- or EVEWorld-generated data, with matched initialization and training budgets. For Piper, each task uses 100 real demonstrations and 100 generated trajectories; RoboTwin uses 40 original demonstrations and 25 generated action-supervision episodes. Wan and Giga denote Wan2.2 and GigaWorld-0, respectively. All scores are percentages (%).
| Platform / Split | Original-only | Wan SFT → EVE | Giga SFT → EVE |
|---|---|---|---|
| Piper / ID | 27.2 | 38.3 → 50.0 | 30.9 → 43.3 |
| Piper / OOD-appearance | 21.9 | 34.5 → 49.3 | 27.2 → 40.0 |
| Piper / OOD-position | 5.3 | 15.9 → 27.3 | 10.6 → 20.0 |
| RoboTwin / ID | 59.0 | 61.8 → 71.7 | 59.3 → 64.0 |
| RoboTwin / OOD-appearance | 39.3 | 42.2 → 49.0 | 41.2 → 47.2 |
| RoboTwin / OOD-position | 22.7 | 24.0 → 28.2 | 23.2 → 27.3 |
EVEWorld outperforms matched SFT across all 12 comparisons, on both real robots and simulation, including ID and OOD settings. It also outperforms the original-data-only baseline on every split.
2. EVEWorld for Action Planning
EVEWorld also improves executable action planning using frozen world models and task-specific action decoders on Piper real robots and RoboTwin. Wan and Giga denote Wan2.2 and GigaWorld-0, respectively. All scores are percentages (%).
| Platform / Split | Wan SFT → EVE | Giga SFT → EVE |
|---|---|---|
| Piper / ID | 19.2 → 30.6 | 11.6 → 23.6 |
| Piper / OOD-appearance | 11.9 → 27.9 | 6.6 → 17.2 |
| Piper / OOD-position | 5.3 → 9.3 | 0.0 → 0.0 |
| RoboTwin / ID | 23.0 → 31.0 | 21.0 → 26.0 |
| RoboTwin / OOD-appearance | 22.0 → 29.0 | 16.0 → 17.0 |
| RoboTwin / OOD-position | 16.0 → 20.0 | 11.0 → 12.0 |
EVEWorld improves 11 out of 12 planning comparisons, with one tie and no regressions, demonstrating that improved physical evolution modeling translates into better downstream executable planning.
Real-robot demonstrations are available on our anonymous project page under "Real Robot."
These results are now included in the revised manuscript (Section 5 and Appendix C.5).
We sincerely hope that EVEWorld can provide useful insights and foundations for future research in embodied world models, while contributing to advances in practical downstream robotic applications.
We greatly appreciate your time and valuable feedback throughout the review process. If you find that these additional results strengthen the contribution of our work, we would be truly grateful if you would consider revisiting your assessment and, if appropriate, raising your score. Thank you again for your consideration!
Response to Reviewer Qyz7 Weaknesses
We're truly grateful for your thoughtful feedback and support. Below, we've done our best to respond to each of your points.
W1. Instance Preservation and Cross-Backbone Generalization
We thank the reviewer for raising this important point. We directly audited legitimate-instance preservation in multi-instance scenes with three independent human annotators (Table 21). Across WorldArena 1.0/2.0, the 55 shared N0 > 1 inputs yield 165 videos across three generators per benchmark. Majority-vote labels exclude valid visibility changes and instruction-required removal.
Released denotes GigaWorld-0/FlowWAM. EVEWorld matches or improves upon these baselines and matches SFT's pooled multi-instance loss count (8/55). Across 326 audited videos/model, disappearance decreases by 60.0% (25 to 10), alongside fewer duplications (45 to 24; Table 20).
We compared FlowWAM LoRA ranks 32/64/128 and full fine-tuning on the same 2,250 training episodes. E7 results (1,974 steps; TIA -> EVEWorld; Table 56):
The EVEWorld-minus-TIA MLR gap shrinks from +13.29 to +0.41 and +0.03 percentage points at ranks 32/64/128, then reverses to -0.93 under full FT. Under full FT, EVEWorld improves PSNR, SSIM, LPIPS, and MLR over TIA at E7 (Table 56). EVEWorld has lower MLR than TIA at every full-FT checkpoint from E4 to E8; TIA retains lower Flow-EPE, and IGR-only has slightly lower MLR at E5/E7/E8 (Table 55). At E6, EVEWorld improves all five metrics over matched SFT, including MLR from 39.45% to 15.81% (Table 5).
The joint benefit of IGR and TIA depends on adaptation capacity. Q4 gives the GigaWorld component ablation, FlowWAM checkpoint trajectories, and a mechanism interpretation.
EVEWorld has fewer violations than SFT at every tested GigaWorld step from 150 to 500 (Table 35); Q4 gives the complete IF sweep.
W2. Sample Size and Evaluation Reliability
We expanded DreamGenBench from one to four matched inference seeds on the same 63 eligible tasks (252 videos/model). EVEWorld has fewer violations than both baselines at every seed (Table 14):
SD is the sample SD across seeds. The pooled reduction versus SFT is 9.13 percentage points (79.3% relative). On 157 shared eligible WA1 inputs, violations fall from 43/157 to 21/157: 27.39% to 13.38%, with respective 95% Wilson CIs (%) [21.02, 34.84] and [8.92, 19.58] (Table 16). On 60 WA1 tasks x 3 seeds, human MLR falls from 26.67% to 10.00% (Table 25).
We validated MLR against three-human-rater majority votes on 978 videos (2,934 judgments). Raters independently checked occlusion, valid exits, hidden-instance reveals, and overlap separation. Every rater finds fewer EVEWorld violations on every benchmark (Table 23). GroundingDINO detects 162/165 genuine violations and flags only 15/813 human-negative videos (Table 22):
Human-consensus MLR falls on all three benchmarks: DreamGenBench 9.52% -> 2.38%, WA1 26% -> 10%, WA2 26% -> 17% (SFT -> EVEWorld; Tables 17-19).
EVEWorld has the lowest MLR under all five evaluators on the 60-task comparison (Table 25). See Q3 for the full five-evaluator comparison. With OWLv2 target detection, MLR falls from 25.00% (SFT) to 15.56% (EVEWorld); robot localization and SAM2 tracking stay fixed. This comparison uses one human judgment per video, with some labels reused from the main audit.
EVEWorld improves policy learning and planning in real and simulated environments with both backbones. ID results, SFT → EVEWorld (%; Tables 10–11); Wan/Giga: Wan2.2/GigaWorld-0:
Policies trained only on original demonstrations score 27.2% on Piper (Real Robot) and 70.0% on RoboTwin. EVEWorld improves policy learning over these baselines and matched SFT on both platforms.