acceptodds
Withdrawn from ICLR 2027

BODYWORLD: STREAMING HUMAN-CENTRIC WORLDS WITH INDEPENDENT BODY AND CAMERA CONTROL

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Human-centric video worlds require control over both what a person does and how the action is observed. We introduce BodyWorld, a framework that streams video from a reference image and independently editable body and camera commands. Decoupled geometric conditioning combines articulated body guidance with complementary scene appearance and viewing geometry. To maintain consistency as motion and viewpoint change, a static point-cloud memory anchors the scene, while Pose-Addressable History Memory recalls relevant generated observations to preserve human appearance beyond recent context. A three-stage curriculum learns joint control, introduces block-causal generation through pose-addressed teacher forcing, and distills few-step continuation with self forcing and distribution matching. Experiments across human motion, camera motion, pose recurrence, and long sequences show improved visual fidelity over prior methods and better appearance consistency across visibility changes. With cached history reuse, two-step inference reaches 13.36 generation FPS on a single NVIDIA H200 GPU.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
world model, streaming video generation, Human-Centric Video Generation

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_ZKY8OpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

BodyWorld generates human videos from a reference image, body poses, and camera trajectories. It combines body-normal conditioning with separate point-cloud and camera-ray adapters, then trains a causal generator using teacher forcing and self-forcing distribution matching. Its pose-addressable memory retrieves two older latents alongside five recent ones to recover appearance details when a pose returns. The paper reports improved image quality and few-step generation on one H200. I find the appearance-memory results promising, but have reservations about the training ablations and the evidence connecting the main quality results to the fast streaming implementation.

Critical strengths

The gains over existing pipelines are worth taking seriously. In Table 1(a), four-step BodyWorld improves PSNR from Uni3C's 17.874 to 19.460 dB and reduces LPIPS from 0.313 to 0.250. The near-static-camera subset is also a sensible inclusion: it gives animation methods without camera control a more suitable comparison, and BodyWorld still performs well there. Appendix A.3 documents the sequence-level training/evaluation separation and makes the overlap between evaluation tracks explicit.

PAHM is the part of the method I find most convincing. Comparing it with both same-weight Recent-7 inference and a separately trained Recent-7 model is useful. On late Long48-human frames, the same-weight comparison gives a Body DINO improvement of 0.0341, with a 95% interval of [0.0129, 0.0578], and an LPIPS improvement of 0.0099 [0.0009, 0.0189]. The face-mask example helps explain what those gains mean. The authors also report where the intervals include zero, which makes this evidence easier to assess.

Critical weaknesses

My biggest concern is that the quality tables and the speed result describe different inference procedures. The main evaluation uses four steps, searches the historical prefix, and recomputes the selected history window. The 13.36 FPS result uses two steps, a five-entry memory bank, and rolling KV states computed in their original insertion contexts. Both the retrieved observations and their representations can therefore change. Appendix B explains this distinction, but the quality check in Table 6 covers only eight development clips and only PSNR/SSIM. That is a thin basis for transferring the main quality conclusions to the accelerated model, particularly since removing DMD improves PSNR but worsens LPIPS and Body DINO (Section 4.4 and Table 8). I would like to see the accelerated configuration evaluated with the same appearance, pose, and recurrence metrics, especially on Long48. The timing also excludes control encoding and VAE decoding and pre-encodes the full control sequence. It is a valid generation-throughput measurement, but leaves the latency of an online command unanswered.

I could not work out what trains the no-DMD model. Section 3.3 and Appendix A.2 say that the generator uses only the DMD surrogate, with no paired reconstruction loss. Table 3 then contains a separately trained “PAHM + SF, w/o DMD” variant. What objective replaces DMD? As written, removing it seems to remove the generator's stated training signal. This needs an explicit answer before that ablation can support the claimed role of DMD. The no-SF history construction is also unclear. More generally, the appendix gives detailed replay equations but leaves out basic training information: stage lengths, learning rates, batch sizes, and generator/fake-score update ratios. Adapter layers and conditioning-drop settings are missing too. Saying that budgets are matched does not tell a reader how to reproduce them.

The case for the geometric-control design is less clear than the case for PAHM. Decoupled PCD/ray conditioning beats the chosen early-fusion baseline, but adds only 0.0437 and 0.0506 dB of body PSNR over PCD-only, with intervals spanning zero; whole-frame PSNR decreases. There is a supported improvement in moving-human temporal consistency (Appendix C.1). What I wanted to know, though, was whether the ray branch improves camera following. Table 2 compares complete systems and cannot answer that question. A camera-error comparison between PCD-only and decoupled conditioning would be much more direct. Similarly, most tests replay body and camera trajectories from the same source sequence. A few crossed combinations, or a command change during a rollout, would test the independently editable controls more convincingly than replay alone.

Finally, the large gains over external baselines are difficult to attribute. BodyWorld is adapted on 92,503 samples, while the baselines use their official pipelines. How much would an adapted geometry-conditioned backbone gain before adding the proposed components? The paper need not retrain every baseline, but one suitable control would help. The OOD results also deserve a more qualified reading: AR improves PSNR/LPIPS while PCK is 0.935 versus Uni3C's 0.971, and all 32 RICH windows come from one raw sequence. For PAHM, the strongest evidence is improved late appearance, rather than broadly better control or persistent worlds. The 201-frame tests are useful, but Long48-scene still loses 4.8581 dB PSNR from early to late windows. I would keep the claims close to that demonstrated scope.

Questions for authors
  1. What losses and history inputs are used for the no-DMD and no-SF variants? Please give the stage-specific training schedule and adapter configuration as well.
  2. How much quality is retained with two-/three-step rolling-bank inference on the main metrics and long-sequence tests? What is the command-to-output latency once control encoding and decoding are included?
  3. Does the ray branch improve camera following over PCD-only, and do body/camera controls still work when their trajectories are independently recombined?
Reviewer_sPXaOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
1
Soundness
2
Read review
Paper summary

This work proposes a framework for streamable human video generation with disentangled control over both human motion and camera trajectory. To this end, the authors adopt a MotionStream-based curriculum learning strategy. To maintain consistent human appearance throughout the generated video, the work further introduces Pose-Addressable History Memory, which retrieves relevant past frames based on human pose.

Critical strengths

The proposed Pose-Addressable History Memory appears novel and reasonable. Inspired by camera-pose-based memory retrieval approaches from prior work on world models, this work designs an effective mechanism for retrieving memory based on human pose.

Critical weaknesses
  • While the proposed PAHM appears novel, the overall framework closely follows the training pipeline of MotionStream, which consists of three stages: (1) training a controllable bidirectional teacher model, (2) initializing a causal student model, and (3) training the initialized causal student model in a self-forcing manner. Given the limited novelty beyond PAHM, I believe this work falls short of the bar for ICLR.
  • The quality of the figures is also below the standard expected at ICLR. The main method figure (Fig. 2) does not effectively convey the core contribution of this work: it devotes considerable space to unnecessary details such as the full self-forcing procedure, its diagrams are poorly organized, and the font size is too small to read comfortably. Furthermore, the result figures (Figs. 3–7) do not preserve the original aspect ratios of the images.
Questions for authors
  • In the training stage 2, why the teacher forcing is used instead of ODE initialization in Self Forcing? Is there a specific reason for adapting the training framework of MotionStream into this task?
  • Could you provide the video results on the proposed method as well as the baseline works?
Reviewer_TDYrOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper presents BODYWORLD, a framework for streaming human-centric video generation with independently editable body-pose and camera controls. The method combines geometric conditioning, pose-addressed history memory, and a three-stage training pipeline that transitions from bidirectional generation to causal, few-step inference. Its main contribution appears to be integrating these components into a unified streaming system.

Critical strengths
  • Independent body and camera control is a useful capability for interactive video generation.
  • The paper addresses relevant challenges, including maintaining human appearance across pose recurrence and viewpoint changes.
  • The experiments include component ablations and evaluate human motion, camera control, appearance consistency, and inference efficiency.
Critical weaknesses
  1. Limited methodological novelty. Most major components have close precedents: Uni3C supports joint human-pose and camera control; Directing the World explores autoregressive joint control; MotionStream uses causal generation with self forcing and DMD; and concurrent LiveAnimate introduces pose-based retrieval of historical context for streaming human animation. BODYWORLD makes specific adaptations, but the paper does not sufficiently establish which differences constitute a substantial technical advance beyond combining these ideas. In particular, the distinction between Stage I and Uni3C, and between PAHM and LiveAnimate’s pose-retrieval mechanism, needs clearer explanation.
  2. Missing supplementary videos. I could not find supplementary videos accompanying the submission. This substantially limits assessment of a method whose central claims concern video generation and streaming interaction. Static figures and aggregate metrics cannot adequately support the claims of the paper. I would have liked to see videos showing baseline comparisons, pose recurrence under camera motion, long rollouts, and online command changes. This is a must-have for this kind of paper!
  3. Potential mismatch in history retrieval. PAHM retrieves full-frame video latents using similarity between cropped body-normal descriptors. Although these descriptors implicitly capture viewing orientation, they do not explicitly account for camera position or scene overlap. Similar body guidance could therefore retrieve useful human appearance together with an incompatible background. Retaining the historical geometric conditions may help the model interpret these observations, but robustness under substantial camera translation remains unclear. An analysis comparing body-only retrieval with joint body–camera retrieval or human-region-only memory would strengthen the design justification.
  4. Presentation and motivation. The paper is difficult to follow in places. The overview figure is particularly hard to parse in Stage III, where the roles of the student, frozen teacher, fake-score model, and gradient flow could be clearer. Each training stage would also benefit from an introductory explanation of the problem it addresses and why the preceding stage is insufficient. The current presentation emphasizes the procedure more than the rationale.

Overall, the paper targets a useful problem and presents a potentially valuable system. However, I remain unconvinced that the methodological contribution is sufficiently distinct from existing work. The absence of video demonstrations also makes it difficult to assess whether the integration delivers the claimed benefits in temporal consistency and interactive control.

Questions for authors
  • What are the substantive architectural and training differences between Stage I and Uni3C?
  • Beyond the retrieval unit and context budget, what distinguishes PAHM from LiveAnimate’s pose-retrieval mechanism?
  • How does PAHM behave when similar body-normal descriptors occur at substantially different camera positions or against different backgrounds?
  • I would have liked to see videos demonstrating online changes to body and camera commands, together with end-to-end response latency. Unfortunately, this. cannot be provided anymore, and is one of the main reasons for the weak reject rating. Overall, I suggest addressing the comments above and submit to a future venue.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.