- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes HumanWorld, a human-centric world model that jointly supports camera control, full-body human motion control, and human–scene interaction. The method introduces a latent World Interaction module to model action-induced scene changes and conditions a pretrained video diffusion model on the resulting features. Experiments demonstrate competitive performance on camera control, human animation, and human–object interaction benchmarks.
Critical strengths
- Meaningful problem. Extending explorable world models to support fine-grained human–scene interactions is an interesting and important direction.
- Unified framework. The method combines camera control, human motion control, and interaction modeling within a single framework.
- Promising results. The method achieves competitive performance across multiple benchmarks, with particularly strong results on BEHAVE.
Critical weaknesses
- Insufficient evaluation of interactive controllability. The HOI experiments rely on ground-truth pose sequences extracted from existing videos. There are no counterfactual experiments showing how different human actions change object states within the same initial scene. It would be great to see results of the same initial scene, different action sequences.
- Lack of joint camera and interaction evaluation. Camera control and HOI are mainly evaluated separately. The paper does not demonstrate whether the same interaction remains consistent across independently controlled camera trajectories, despite this being a central contribution. Similar as above, it would be great to see results of the same initial scene, same action sequences, with different camera trajectories (I didn't see this in the supplementary video).
- Limited evidence of interaction correctness. The HOI evaluation relies on PSNR, SSIM, LPIPS, and FVD, which do not directly measure contact consistency, object displacement accuracy, or physical plausibility.
- Unclear disentanglement of world dynamics and observations. Although Eq. (2) defines camera-independent state transitions, the actual scene representation depends on camera-conditioned features and generated video latents (Eqs. (3)–(5)). It is unclear whether the model learns persistent, viewpoint-independent world states.
- Unclear 3D human motion representation. The method assumes world-space 3D human poses, while the datasets primarily provide 2D pose annotations. The paper does not clearly explain how globally aligned 3D poses are obtained. Also in the supplementary videos, there are significant flickering of 2D poses, with some limbs suddenly disappearing and reappearing -- not sure where that comes from given that this is supposed to be "groundtruth".
- Limited ablation studies. The paper only ablates the entire World Interaction module, without isolating the contributions of camera modulation, individual interaction components, or the proposed training losses.
- Incomplete implementation details. Appendix Table 6 explicitly contains an unfinished placeholder, while referenced architectural configurations are missing. The output resolution is unspecified, and a sentence in Appendix C is incomplete.
Questions for authors
- Counterfactual action control. Can the authors demonstrate generating different human–object interactions from the same initial image using different input pose sequences? This would help establish whether the model can produce action-dependent scene changes beyond reconstructing existing interactions.
- Joint camera and interaction control. Can the authors provide examples or quantitative results showing the same human–object interaction under different independently specified camera trajectories? How consistent are the resulting object states and contact relationships across views?
- Interaction correctness. Can the authors evaluate human–object interaction accuracy using more direct metrics, such as contact consistency, object trajectory error, or penetration rate? The current PSNR, SSIM, LPIPS, and FVD metrics do not directly measure interaction correctness.
- World-state disentanglement. How does the implementation maintain a persistent, viewpoint-independent world state? In particular, Eqs. (3)–(5) construct the scene representation using camera-conditioned features and generated video latents, without an explicit state transition from . How does this formulation realize the camera-independent dynamics described in Eq. (2)?
- 3D pose representation. How are world-space 3D human poses obtained from datasets primarily annotated with 2D keypoints? How are these poses aligned with the camera trajectories, and what 3D information is retained when the poses are projected into 2D DWPose sketches for conditioning?
We thank the reviewers for their constructive feedback. We provide additional quantitative evaluation and controlled qualitative examples to address the concerns about interaction accuracy, joint control, and the role of the World Interaction module.
Quantitative interaction evaluation.
We supplement the visual-quality metrics with object-mask mIoU and CenterL1 on BEHAVE. We use SAM2 to segment objects in generated videos and measure mask overlap and centroid L1 error relative to the target. These metrics directly evaluate the spatial accuracy of the interacting objects.
HumanWorld achieves the highest mIoU and lowest CenterL1 among the evaluated methods. Removing the World Interaction module reduces mIoU from 0.4549 to 0.4126 and increases CenterL1 from 0.0694 to 0.0882. This ablation supports the module's contribution to object localization and spatial alignment during interaction. These measurements complement the visual-quality evaluation; contact consistency and penetration require separate metrics.
Counterfactual actions and joint camera–interaction control.
Figure 1 presents controlled examples sharing the same initial image. With the camera trajectory held fixed, changing the prescribed action from pushing forward to pulling back produces corresponding changes in object motion. With the action held fixed, introducing a rotating camera preserves the illustrated interaction outcome across viewpoints. These examples provide qualitative evidence of action-dependent scene evolution under independently specified camera trajectories. They support behavioral consistency across views, while persistent viewpoint-independent latent states require further validation.
Role of the interaction representation.
Figure 2 visualizes interaction-derived attention on ARCTIC sequences using a fixed-noise forward pass through the middle interaction injector. Attention follows the manipulated object and active interaction region as the action progresses, suggesting that the learned representation captures interaction-relevant information. Together with the mIoU and CenterL1 ablation above, this analysis supports the World Interaction module's role in translating prescribed human motion into corresponding scene responses.