acceptodds
Withdrawn from ICLR 2027

IntentAnimator: Human Animation from Compositional and Underspecified Motion Intents

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Controllable human animation is commonly formulated as reproducing the motion of a driving video. However, real creative workflows often combine sparse pose keyframes, selected-joint trajectories, and local action descriptions, while specifying only selected time points, subjects, or body parts. We introduce Intent-Driven Human Animation, a new setting for synthesizing complete and coherent human videos from such compositional and underspecified motion intents. This setting requires jointly grounding heterogeneous constraints while plausibly completing the human dynamics they leave unspecified. We propose IntentAnimator, a unified framework comprising Omni-Intent Grounding (OIG) and Structure-of-Thought (SoT). OIG converts state, path, and text intents into a shared token sequence and uses Intent-RoPE to assign modality-appropriate positional anchors, enabling all intents to be jointly interpreted. Within diffusion, SoT supervises a dedicated structure-video stream in a human body-part domain to progressively complete missing dynamics and uses its evolving and semantically rich representations to guide pixel synthesis, improving motion coherence and reducing human deformation. We further construct a user-intent benchmark from annotations by professional creators and non-expert users, showing that natural motion intent is highly combined and sparse. Extensive experiments demonstrate that IntentAnimator improves intent following, motion plausibility, and visual quality over existing related baselines.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Diffusion model, Human Animation

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_jLkVOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces Intent-Driven Human Animation task that aims to synthesize coherent human videos from compositional and underspecified motion intents, rather than reproducing motion from a fully specified driving video. The proposed Omni-Intent Grounding encodes heterogeneous motion intents, including pose or bounding-box keyframes, joint trajectories, and local text descriptions, into a shared token sequence. Intent-RoPE assigns modality-aware spatiotemporal positional coordinates to these tokens to improve the grounding of motion constraints. Second, Structure-of-Thought introduces a dedicated structural video stream supervised using human body-part representations. The evolving structural features guide pixel-level video synthesis through unidirectional cross-attention during diffusion denoising.

Critical strengths
  1. The paper identifies an important limitation of existing human animation methods: they typically assume access to a fully instantiated driving sequence, whereas real creative workflows often rely on sparse keyframes, selected-joint trajectories, and local action descriptions.

  2. The combination of Omni-Intent Grounding and Structure-of-Thought is well motivated. OIG explicitly handles heterogeneous control modalities and their spatiotemporal relationships, while SoT separates structural motion reasoning from appearance synthesis. This decomposition provides a clear architectural rationale for addressing the two central challenges of the proposed task.

  3. Intent-RoPE preserves continuous temporal coordinates instead of rounding control timestamps to the discrete latent-frame lattice. This is particularly relevant when sparse motion constraints must be aligned precisely with a temporally compressed video diffusion model.

Critical weaknesses
  1. The structural supervision may inherit substantial errors from automatic human reconstruction. The structure-video stream relies on SMPL parameters estimated by CoMotion from in-the-wild videos. Errors in pose estimation, body shape, occlusion handling, and multi-person tracking may be propagated into the structural training targets. Moreover, SMPL-based body-part representations do not explicitly model articulated finger joints, limiting the structural guidance available for fine-grained hand motion. While the paper acknowledges hand-generation artifacts, it does not sufficiently quantify how structural annotation quality affects the final generation results.

  2. The evaluation benchmark is valuable but limited in scale and coverage. Although the benchmark is motivated by real creative behavior, its coverage of human actions, interaction types, control combinations, and levels of underspecification may not fully represent the diversity of real-world animation requirements. The reported aggregate statistics also do not establish whether the method performs consistently across professional and non-expert annotations or across different action categories. More detailed subgroup evaluations and analyses of annotation variability would help establish the generality of the proposed approach.

  3. The baseline comparisons are not fully uniform across methods. The authors appropriately use capability-matched subsets for methods that cannot accept all control modalities. However, comparisons across different subsets do not establish that the proposed framework is consistently superior under identical input conditions and evaluation distributions.

  4. The "POSE-DRIVEN HUMAN ANIMATION" section in related work lacks discussion of the following work [1,2,3,4]. The authors could enhance the analysis of the limitations of these works to make the motivation of this paper clearer.

[1] Comovi: Co-generation of 3d human motions and realistic videos

[2] MoSA: Motion-coherent human video generation via structure-appearance decoupling

[3] Generating Human Motion Videos using a Cascaded Text-to-Video Framework

[4] AnimateAnywhere: Rouse the Background in Human Image Animation

Questions for authors

See weakness.

Reviewer_bmUeOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper introduces IntentAnimator, which generates human videos from a reference image and a combination of sparse poses, joint trajectories and local action descriptions. It grounds these controls through shared tokens and positional embeddings, then uses a jointly generated body-structure video to guide visual synthesis. Comparisons and ablations show improved control following, motion plausibility & visual quality.

Critical strengths
  • The task addresses a practical limitation of existing animation systems. Allowing users to combine selected poses, trajectories, and timed descriptions provides a flexible way to specify motion without supplying a complete driving video.
  • The structural stream is a sensible design for completing missing motion. It receives direct supervision and guides the pixel stream throughout denoising, giving the model an intermediate representation focused on body structure.
  • The improvements over the adapted baseline are convincing.
  • The ablations examine meaningful alternatives and their consistent results support the value of both positional grounding and structural guidance.
  • The benchmark and human evaluation are carefully described
Critical weaknesses
  • The automatic metrics do not directly measure control accuracy. PSNR and SSIM compare complete RGB frames even when only a bounding box or partial pose is specified, while FID and FVD assess overall distributions.
  • Most baselines use their released checkpoints without adaptation to MotionIntent-300K. The same-data UniAnimate-FT comparison helps, but comparisons with the other methods still mix architectural differences with the benefit of task-specific training.
  • The novelty of the problem setting needs a narrower framing. Earlier work already combines temporally sparse sketches with spatially sparse poses. The clearer contribution here is the joint handling of subject-specific, time-anchored controls and the structural generation framework, rather than animation from partial inputs alone.
  • Several central capabilities are demonstrated mainly through selected examples. Aggregate results do not show how performance changes with control sparsity, the number of people, or the combination of modalities. This makes it difficult to assess how reliably the model completes unspecified people and motion.
  • No confidence intervals or repeated-run variability are reported. This matters particularly for the smaller differences between the full model, rounded timestamps, and bidirectional denoising. The benefit of sub-latent temporal positioning also needs a more targeted timing evaluation.
  • The supplementary videos don't demonstrate the inputs and ground truth, which makes it difficult to assess the generated output
Questions for authors
  • Can you report direct pose and path errors on the specified controls, along with event-timing accuracy? In particular, do fractional timestamps improve the timing of events that fall within the same temporally compressed latent interval?
  • How are sparse state controls and paths converted into each baseline’s native input format, especially when a path does not specify frame-by-frame timing? Were any additional baselines adapted to the training data?
  • Can you break down performance by control sparsity, modality combination, and single-person versus multi-person scenes? Results specifically measuring completion of uncontrolled subjects would help support this claim.
  • Can you provide confidence intervals for the human preferences in Table 2, accounting for repeated judgments of the same clips?
  • Does one-way structural guidance still provide the same benefit without stop-gradient? This comparison would help separate the effect of the attention direction from the claimed benefit of preventing appearance-related gradients from affecting structural learning.
  • Can the authors provide results showing the inputs as well as ground truth comparison?
Reviewer_qCXWOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

the main text is over the page limit of 9-page. desk reject

Critical strengths

the main text is over the page limit of 9-page. desk reject

Critical weaknesses

the main text is over the page limit of 9-page. desk reject

Questions for authors

the main text is over the page limit of 9-page. desk reject

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.