- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes Primitive Embodied World Models (PEWM), a framework that organizes robot demonstrations into short, semantically atomic motion primitives. A video diffusion model predicts primitive-level visual trajectories conditioned on language instructions and start–goal heatmaps. Executable end-effector trajectories are then recovered using a pretrained pose estimator and geometric transformations. A VLM planner composes these primitives for longer tasks. The framework is evaluated through RLBench experiments, real-robot manipulation, video-generation metrics, and component ablations.
Critical strengths
-
A coherent and reasonably comprehensive system. The paper connects primitive-level data collection and annotation, video generation, spatial grounding, pose extraction, and hierarchical execution into a concrete robotic pipeline.
-
A useful emphasis on data organization. Aligning short video segments with individual manipulation intents is a plausible way to improve language–motion alignment and facilitate reuse across tasks. Recovering trajectories without training a dedicated action decoder is also an interesting design choice.
-
Evaluation includes physical execution. Beyond generated-video examples, the paper reports real-robot task success, compositional evaluations, and ablations. These provide evidence that the framework has practical utility.
Critical weaknesses
-
The conceptual advance and resulting insight are limited. Most components build on established techniques, and the contribution appears primarily to be their integration around primitive-level data. The experiments do not clearly isolate whether the gains arise from semantic segmentation, additional annotations, spatial guidance, the VLM planner, or the overall training recipe.
-
The baseline comparisons are insufficiently controlled. Several simulation results are taken directly from prior papers without establishing equivalent training data and evaluation protocols. The supplementary comparison against Colosseum-derived results is particularly difficult to interpret. Recent WAM approaches, such as Fast-WAM and DreamZero, also deserve discussion and, where feasible, experimental comparison, given the paper’s claims about control, generalization, and efficiency.
-
The empirical evidence supports narrower conclusions than the paper claims. The main simulation evaluation covers nine RLBench tasks, with a modest average improvement over 4DWM and regressions on three tasks. Real-robot comparisons cover three tasks with 20 trials each. These results demonstrate feasibility, but provide limited support for broad claims about scalability, systematic compositional generalization, or data efficiency.
-
The efficiency reporting is internally inconsistent. The main text reports distilled generation at 12 FPS, whereas the appendix reports 32 frames in 16 seconds and approximately 31 seconds per primitive including pose estimation. The appendix also describes acceleration techniques as future work. These configurations need to be reconciled before the real-time deployment claim can be assessed.
Questions for authors
-
What is the independent benefit of semantic primitives? Can the authors compare semantic segmentation against fixed-length segmentation using the same demonstrations, backbone, annotation budget, and planner?
-
How are training and evaluation separated? Please clarify which tasks, objects, and action–object combinations are excluded from training, separately for the video model and VLM planner. What target-task supervision is used when the video model remains frozen?
-
Are the baseline protocols comparable? Please specify differences in training data, task variations, observations, execution budgets, and success criteria. Which exact Colosseum conditions produced the numbers used in Table 8?
-
Why is explicit video generation followed by pose extraction preferable to learned action prediction? A representative WAM baseline or a controlled action-decoder comparison would help establish the benefits and costs of this design.
-
Which configuration produces the reported real-robot results? Please reconcile the latency measurements and identify whether the distilled model was used in the success-rate evaluations. Report end-to-end latency, including planning, video generation, and pose extraction.
Dear Reviewers,
Please read the authors’ response and let us know whether it addresses your concerns. If any concerns remain, please briefly indicate which ones.
Thank you for your time and for your thoughtful reviews.
Best,
AC