- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper trains a video world model on 18 heterogeneous data sources to support action control, camera control, and visually specified embodiment. Camera and effector trajectories are expressed in a reference-camera coordinate system, with a common action template and shared encoder accommodating missing annotations. A frozen VLM conditions noise-specific generation experts. Single-video pretraining is followed by cross-view adaptation of the high-noise expert while retaining the low-noise expert. Evaluation covers human and robot action-conditioned generation, camera-controlled generation, and video re-shooting; composition and human-to-robot transfer are illustrated qualitatively.
Critical strengths
-
The control interface addresses a practical obstacle to heterogeneous world-model training. Reference-camera motion coordinates, local articulation fields, and learned null embeddings give different annotations a reasonably clear physical interpretation. Keeping missing controls distinct from stationary controls is particularly sensible. Combining these sources in one model is useful even when the individual architectural components are established.
-
The reported task-level results are strong across several settings. For example, video re-shooting reaches 164.03 FVD versus 384.27 for TrajectoryCrafter, while also improving camera errors and PSNR (Table 3). The human and robot experiments include relevant specialized baselines rather than only general video generators. This breadth gives the system contribution some empirical substance.
Critical weaknesses
-
The main compositional-control claim is not directly established by the quantitative evaluations. Separate action and camera benchmarks do not show that both can be changed independently while preserving the other, or that this works for held-out combinations. Figure 5 is a qualitative illustration, and the human-to-robot examples in Figure 8 additionally use a rendered target robot image, as Appendix A.3 correctly explains. These are useful demonstrations, but there is no quantitative transfer or unseen-embodiment evaluation. Since this capability is the main distinction from existing controlled generators, the gap materially limits the contribution supported by the experiments.
-
The experiments do not isolate the two central design choices well enough. The encoder ablation keeps reference-camera alignment fixed, so it tests parameter sharing rather than the benefit of geometric alignment. The noise-expert ablation is not a comparison of high-noise-only paired adaptation against adapting both experts. Consequently, the proposed explanations for cross-domain transfer and preserving refinement remain plausible rather than demonstrated. Reproduction is also difficult: Appendix A describes the procedure but does not give numerical training budgets, source mixture sizes, routing threshold, or sampling settings needed to assess these comparisons.
Questions for authors
-
Is there quantitative evidence for simultaneous action-camera control on held-out combinations, reporting both control errors and preservation of the factor held fixed? For embodiment transfer, which target embodiments were absent from training, and what conditions, including rendered initial images, were provided? A comparison with a joint-control method such as Uni3C would help locate the contribution.
-
Can you provide controlled comparisons with unaligned source coordinates and with both experts adapted on the same paired data, holding data and training budget fixed? Please also specify the routing threshold, training steps and mixture sizes, and inference resolution, frame count, denoising steps, and guidance settings.
I currently lean toward weak rejection. The system and single-task results are promising, but stronger evidence for composition and the claimed alignment/adaptation mechanisms is needed for acceptance.