acceptodds
Under review as a conference paper at ICLR 2027

SteadyDancer: Harmonized and Coherent Human Image Animation with First-Frame Preservation

Abstract

Preserving first-frame identity while ensuring precise motion control is a fundamental challenge in human image animation. The Image-to-Motion Binding process of the dominant Reference-to-Video (R2V) paradigm overlooks critical spatio-temporal misalignments common in real-world applications, leading to failures such as identity drift and visual artifacts. We introduce SteadyDancer, an Image-to-Video (I2V) paradigm-based framework that achieves harmonized and coherent animation and is the first to ensure first-frame preservation robustly. Firstly, we propose a Condition-Reconciliation Mechanism to harmonize the two conflicting conditions, enabling precise control without sacrificing fidelity. Secondly, we design Synergistic Pose Modulation Modules to generate an adaptive and coherent pose representation that is highly compatible with the reference image. Finally, we employ a Staged Decoupled-Objective Training Pipeline that hierarchically optimizes the model for motion fidelity, visual quality, and temporal coherence. Experiments demonstrate that SteadyDancer achieves state-of-the-art performance in both appearance fidelity and motion control, while requiring significantly fewer training resources than comparable methods.

Primary area:
generative models
Keywords:
Video Generation

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_sd59OpenReview ↗
Rating
3(+1)
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes SteadyDancer, an Image-to-Video (I2V) framework for human image animation. It introduces three main components to address spatio-temporal misalignments and identity drift in existing Reference-to-Video (R2V) methods: (1) a Condition-Reconciliation Mechanism; (2) Synergistic Pose Modulation Modules (SPMM); and (3) a Staged Decoupled-Objective Training Pipeline. The authors also introduce X-Dance, a new benchmark for evaluating non-homologous cross-source animation scenarios.

Critical strengths
  1. This paper accurately identifies and addresses critical limitations of the prevailing R2V paradigm, particularly spatio-temporal misalignments.
  2.  The staged training pipeline requires significantly fewer steps (14.5k) and less data (10.2 hours) than recent DiT-based models while maintaining competitive performance.
  3. The X-Dance benchmark addresses the critical need for evaluating models in realistic, cross-source scenarios.
  4. The "Pose Simulation" strategy during training is an elegant, low-cost solution that effectively simulates and mitigates real-world start-gap artifacts.
Critical weaknesses
  1. Stage 2 aims to preserve the teacher's prior by aligning the student's unconditional component with the teacher's. Why not directly use the frozen teacher model to produce the unconditional prediction? The paper lacks an ablation comparing the proposed Stage 2 against this training-free alternative, leaving the necessity of Stage 2 unsubstantiated.
  2. There is a clear mismatch between Stage 2 training and inference. Stage 2 trains the student's unconditional component, but the inference-time DC-CFG (Eq. 8) relies on negative conditions instead of a pure unconditional prediction. The paper provides no evidence showing that this unconditional distillation actually benefits the final DC-CFG sampling.
  3. While the Synergistic Pose Modulation Modules (SPMM) empirically improve performance, the paper lacks sufficient implementation details (e.g., network architectures, forward process) and deeper insights into exactly why and how these specific designs resolve the spatio-temporal misalignments better than standard adapters.
  4. The figures in the manuscripts are not well organized to show your strength.

I am open to raising my score if the authors can adequately address these concerns.

Questions for authors
  1. The manuscript omits several key hyperparameters necessary for reproducibility. Could you provide details on: (1) LoRA settings (rank, alpha, target modules); (2) Training specifics (resolution, frames, FPS, batch size, optimizer); and (3) Inference settings (DC-CFG guidance scales)? 2. The model is trained on a proprietary dataset of 7,338 curated clips. Do you plan to open-source this data.
  2. How does SteadyDancer scale to long video generation? Specifically, does the Condition-Reconciliation Mechanism successfully prevent identity drift over extended durations. Figure 10 is too small to see the details. You should exhibit a video in the supp to support your claims.
9 comments

R3-01 (W1): Role of the Frozen Teacher and Necessity of Stage 2

Thank you for the question. Our original manuscript did not explain the role of the Stage 2 teacher model clearly enough. Here, the teacher is the original Wan-2.1 I2V model, which generates videos conditioned on a reference image but lacks the pose control capability introduced in our work; its unconditional output is just a standard image-to-video result. Therefore, the teacher is not intended to replace SteadyDancer for pose-driven animation.

In fact, it provides the student with an alignment target for preserving the original I2V generation capability. In Stage 1, introducing pose conditioning enables the model to learn motion control. However, this adaptation affects the generative priors from the base model, leading to degraded visual quality and detail preservation. To solve this, we introduce the frozen teacher in Stage 2 to serve as a distillation regularizer that constrains the student to retain its original I2V generation capability while continuing to learn the pose control task. Specifically, the student continues to receive ground-truth supervision conditioned on both the reference image and driving poses. In parallel, we remove the pose condition so that both student and teacher make predictions under the same reference image condition only to compute the distillation loss. The former task supervision maintains the student's pose adherence, while the latter alignment target from the teacher bounds how far the student drifts from the original generative prior. These two losses jointly guide the student's training.

Thus, the teacher's prediction serves as a training target that constrains the student via distillation loss to retain the generative capability of the original I2V model. Because the teacher itself lacks pose control capability, it cannot directly substitute for the student in human animation conditioned jointly on a reference image and driving poses. Once Stage 2 finishes, final video generation is performed independently by the trained student model, and the teacher is not involved at inference time.

In the revised manuscript, we will clarify the teacher's role as a regularizer on generative priors and revise "unconditional" to "pose-unconditional" to make clear that the reference image condition is retained. We will also detail the computation flow of task supervision and teacher distillation so that both the training objective of Stage 2 and how the teacher is used are clear.

R3-02 (W2): Relationship Between Stage 2 Distillation and Inference-Time DC-CFG

Thank you for pointing out that the relationship between Stage 2 training and inference-time guidance needs clearer explanation.

In Stage 2, the "unconditional component" is defined relative to the newly introduced pose condition. During pose-unconditional teacher distillation, neither the student nor the teacher takes pose input, while both still retain the reference image condition. This prediction is used to construct the teacher distillation target so that the student preserves the generative capability of the original Wan I2V model while learning pose control, rather than acting as a separate generation branch to be called directly during final inference. Meanwhile, task supervision under reference-image and pose conditions jointly updates the student model's parameters together with pose-unconditional teacher distillation, and final inference conditioned on the reference image and pose is carried out by the resulting trained student model.

DC-CFG, on the other hand, follows the conditional-difference formulation of CFG at inference time to adjust the sampling direction for stronger control without extra training.

Specifically, in standard text CFG with negative prompts, each denoising timestep runs two forward predictions with the same noisy latent and reference image: one with a positive prompt and one with a negative prompt. The negative prompt describes undesired content such as overexposure, blur, deformed limbs, static frames, or cluttered backgrounds. The difference between the two predictions defines a guidance direction that strengthens the positive condition while moving away from the negative condition, and the combined prediction takes the negative-prompt prediction as its base term plus the scaled conditional difference:

ϵ^txt=ϵtxt,neg+wtxt(ϵtxt,pos−ϵtxt,neg),\hat{\epsilon} _ {\mathrm{txt}} = \epsilon _ {\mathrm{txt,neg}} + w _ {\mathrm{txt}} \left(\epsilon _ {\mathrm{txt,pos}} - \epsilon _ {\mathrm{txt,neg}}\right),

where wtxtw _ {\mathrm{txt}} controls text guidance strength. Because other conditions such as the reference image and pose remain identical across both passes, this comparison isolates the prediction difference caused by changing the text condition.

We extend the same intuition to pose control. Under the same noisy latent, one pass feeds the target driving pose while the other feeds a negative pose constructed through perturbations such as scaling and translation:

ϵ^pose=ϵpose,neg+wpose(ϵpose,pos−ϵpose,neg),\hat{\epsilon} _ {\mathrm{pose}} = \epsilon _ {\mathrm{pose,neg}} + w _ {\mathrm{pose}} \left(\epsilon _ {\mathrm{pose,pos}} - \epsilon _ {\mathrm{pose,neg}}\right),

where the negative pose is an off-target pose condition constructed for contrast instead of no pose. The prediction difference between the positive and negative poses provides a guidance direction that strengthens the response to the target motion while steering away from the perturbed pose response. It aims at reducing limb blur, motion ghosting, or drift toward ambiguous poses.

Therefore, pose-unconditional distillation in Stage 2 and negative-pose guidance at inference serve distinct purposes. The former constrains the student via training regularization with the frozen teacher to retain its generative prior, whereas the latter uses conditional prediction differences from the trained student to strengthen pose control during inference sampling. In the revised manuscript, we will explicitly clarify the distinction between "pose-unconditional" and "negative-pose conditional" to make the relationship between Stage 2 training and DC-CFG sampling clear.

R3-03 (W3): Implementation Details and Design Insights of SPMM

Thank you for pointing out that the structure and function of SPMM warrant a clearer explanation. SPMM processes the pose representation before it enters the DiT, with its three modules improving the control signal across content adaptivity, temporal context, and correspondence between the generation state and pose. Instead of simply adding another condition-injection layer, our design shapes the input pose into a representation better suited to the current generation task.

SSAE performs content-adaptive feature adjustment to address spatial structural discrepancies. Because the reference subject and driving subject may differ in body shape, limb proportions, or framing, a fixed pose representation struggles to adapt fully to these variations. SSAE generates a dynamic transformation matrix and channel scaling factors from the global content of the input pose, adjusting the channel responses of pose features through two branches. Different poses can thus induce different feature transformations, providing a more adaptable representation for subsequent fusion with the reference image condition. This adjustment takes place at the feature level instead of geometrically retargeting the input skeleton.

TMCM improves the coherence of motion representations through local temporal context. Isolated errors, missing keypoints, or contradictory cues in pose estimation can make frame-by-frame control signals unstable. TMCM adopts three factorized convolutional blocks, where each block first extracts intra-frame spatial structure and then exchanges information across neighboring frames via temporal convolution. Consequently, the pose representation at any given frame draws on surrounding motion context alongside the current detection, mitigating the impact of isolated noise on the generation process. This acts as learned temporal feature modeling rather than fixed low-pass smoothing on the input poses.

FAAU uses the current denoising state to aggregate pose information via attention. Enhancing the pose representation on its own is insufficient to establish its correspondence with the active generation state. FAAU takes the denoising latent as Query and the enhanced pose features as Key and Value, selecting and aggregating relevant pose information via cross-attention followed by normalization, a feed-forward residual block, and projection to obtain the control representation. Its role is to make pose-condition processing account for the current generation state instead of applying a fixed fusion rule independent of the generation process. Here Query comes from the current noisy video latent, while coordination with the reference appearance is achieved jointly through subsequent generation with this representation and the I2V condition.

The three modules follow a hierarchical aggregation scheme where SSAE and TMCM process pose features in parallel and are added to the original pose via a residual connection, after which FAAU processes the result:

zp∗=zp+PSSAE(zp)+PTMCM(zp),zp†=PFAAU(q=z^t,  kv=zp∗).z _ p^{\ast} = z _ p + \mathcal{P} _ {\mathrm{SSAE}}(z _ p) + \mathcal{P} _ {\mathrm{TMCM}}(z _ p), \qquad z _ p^{\dagger} = \mathcal{P} _ {\mathrm{FAAU}}\bigl(q = \hat{z} _ t,\; kv = z _ p^{\ast}\bigr).

Both the enhanced pose representation and the attention output are retained as conditions and concatenated with the image condition and denoising latent before entering the DiT. Attention processing therefore does not replace the original pose information. It builds on spatial and temporal feature enhancement by adding generation-state-aware condition matching.

Compared with approaches that use only static pose encoding and condition injection, these designs respectively introduce pose-dependent feature transformations, local motion context, and generation-state-aware attention aggregation, clarifying their design motivation for addressing structural discrepancies, pose noise, and condition correspondence. Our module ablations and targeted qualitative comparisons (Fig. 12 and Appendix F, Fig. 24) further show the structural or motion failures that emerge when removing the corresponding modules, supporting their roles in the system. In the revised manuscript, we will add architectural specifications, input/output shapes, and the full forward aggregation formulation to make both the design rationale and implementation details clear.

Reviewer_wvHzOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes SteadyDancer, a human image animation framework built on Wan-2.1-I2V-14B that aims to preserve the reference image as the first frame while accurately following a driving pose sequence. To address conflicts between appearance preservation and motion control, the authors introduce a Condition-Reconciliation Mechanism and several pose modulation modules for handling spatial and temporal misalignments. They also propose a three-stage training pipeline that combines motion supervision, condition-decoupled distillation, and pose simulation to improve generation quality and motion continuity.

Critical strengths
  • Practical problem formulation. The paper addresses spatial and temporal misalignment between reference images and driving poses, which is a common challenge in real-world human animation. The I2V-based approach provides a natural way to preserve the first frame while enabling motion control.

  • Good performance on challenging scenarios. The method achieves competitive results on existing benchmarks and shows clear improvements on the proposed X-Dance benchmark. The qualitative comparisons are particularly convincing in cases involving large pose differences and identity preservation.

  • Clear presentation. The paper is easy to follow. The motivation behind each component is clearly explained, and the method overview and qualitative examples help readers understand the overall framework and its design choices.

Critical weaknesses
  • Lack of quantitative evaluation of pose control accuracy. Precise motion control is one of the paper's main claims, but the evaluation mainly relies on visual quality and temporal consistency metrics. Motion Smoothness does not directly measure how accurately the generated video follows the driving poses. Prior works such as HyperMotion have used PCK to evaluate pose accuracy. I think a similar evaluation is necessary, especially on X-Dance, to support the claimed improvements in motion control.

  • Limited evidence for the effectiveness of individual components. The ablation results in Table 2 show only marginal improvements from the proposed pose modulation modules, and removing some modules even improves certain metrics. While the qualitative examples demonstrate their potential benefits, the quantitative results do not clearly establish how much each module contributes. More targeted evaluations on challenging cases would help justify the proposed design.

  • Insufficient evidence for the claimed training efficiency. The paper emphasizes its low training cost by comparing the number of training steps and dataset sizes with prior methods. However, these comparisons involve different model architectures, training strategies, and hardware configurations. In particular, the second training stage requires full-parameter fine-tuning of a 14B model. Without reporting comparable GPU hours or computational costs, it is difficult to assess the actual efficiency advantage over existing methods.

Questions for authors
  • Pose control accuracy. Can the authors provide a quantitative evaluation of how accurately the generated videos follow the driving poses, using metrics such as PCK or AKD? This would be particularly valuable on X-Dance, where motion control under spatial and temporal misalignment is a central claim.

  • Effectiveness of individual modules. The improvements from SSAE, FAAU, and TMCM in Table 2 are relatively small, and removing some modules even improves certain metrics. Can the authors provide more targeted quantitative evaluations on challenging cases to better demonstrate the contribution of each module?

  • Training efficiency. Can the authors report the total GPU hours for each training stage and clarify how the overall training cost compares with existing methods? Since Stage 2 involves full-parameter fine-tuning of a 14B model, comparing training steps alone may not accurately reflect computational efficiency.

6 comments

R2-01 (W1 & Q1): Pose Control Accuracy and Reference Appearance Evaluation

Thank you for the suggestion. We agree that existing metrics do not directly measure adherence to driving poses. We therefore added PCK evaluations on RealisDance-Val and X-Dance using DWPose keypoints extracted from the driving and generated videos, taking the driving keypoints as targets. SteadyDancer outperforms the compared methods on both benchmarks, providing direct quantitative support for our pose control.

To separate keypoint detection failures from localization errors on detected keypoints, we report both PCK (incl. missing), which counts missing predictions for valid target keypoints as incorrect, and PCK (excl. missing), which evaluates alignment only on detected corresponding keypoints alongside Point Coverage. SteadyDancer achieves the best results on both PCK variants and Point Coverage, showing that our gains come from both more accurate keypoint localization and broader coverage rather than ignoring missed detections.

Evaluating pose control also requires checking reference appearance preservation, especially when the reference and driving subjects differ in body shape, appearance, and initial pose, where following driving motions should not cause appearance or identity drift. We therefore proactively added DINO-I (visual feature similarity), CLIP-I (semantic image similarity), and Face-Sim (facial identity similarity) to evaluate reference consistency across complementary levels. Unlike pixel- or structure-level metrics such as SSIM, these metrics assess whether a character retains its reference appearance as poses change rather than mere pixel-layout similarity.

X-Dance Incl. missing counts missing predictions as incorrect; excl. missing is computed only on detected keypoints.

Model[email protected] (incl. missing)[email protected] (incl. missing)[email protected] (excl. missing)[email protected] (excl. missing)Point CoverageDINO-ICLIP-IFace-SimFace Coverage
Moore-AnimateAnyone56.0160.2866.4373.1080.63%0.510.700.1172.80%
HumanVid68.2170.9173.6976.7892.12%0.540.700.0872.40%
MimicMotion58.7384.5660.6687.7694.35%0.590.720.1296.20%
Hyper-Motion39.1662.2845.4274.4980.04%0.530.700.3061.20%
animate-x45.5267.7148.5372.8291.06%0.730.790.3396.80%
UniAnimate-DiT41.7764.1844.4869.4189.52%0.680.800.4088.80%
VACE55.4474.1158.5878.3594.40%0.430.640.1294.40%
Wan-Animate59.6877.6463.0682.4692.42%0.640.780.4195.40%
Ours84.8289.4388.7193.8394.69%0.730.820.4597.20%

RealisDance-Val Incl. missing and excl. missing follow the same definitions as above.

Model[email protected] (incl. missing)[email protected] (incl. missing)[email protected] (excl. missing)[email protected] (excl. missing)Point CoverageDINO-ICLIP-IFace-SimFace Coverage
Moore-AnimateAnyone77.0182.8484.5391.3889.15%0.750.840.2883.16%
HumanVid87.1389.6793.7896.7792.16%0.770.860.3080.41%
MimicMotion30.0254.9131.9258.8590.67%0.640.800.1889.00%
Hyper-Motion66.2877.8372.3485.9788.80%0.760.850.3586.60%
animate-x30.6050.5134.5756.5987.72%0.690.810.2279.73%
UniAnimate-DiT29.6348.3032.4653.3986.49%0.710.820.2981.10%
VACE41.2962.0544.0766.4293.28%0.720.840.2387.29%
Wan-Animate31.8853.6434.5058.7088.56%0.790.870.4086.25%
Ours90.1092.8593.8696.8795.44%0.810.890.4293.13%

SteadyDancer also outperforms the compared baselines across DINO-I, CLIP-I, and Face-Sim, confirming consistent retention of visual features, semantic appearance, and facial identity. Combined with both PCK variants and Point Coverage, these results show that SteadyDancer reconciles accurate motion control with reference appearance preservation under cross-source spatio-temporal misalignment rather than excelling in only one dimension. We will include these results and metric definitions in the revised manuscript.

R2-02 (W2 & Q2): Effectiveness of Individual Pose Modulation Modules

To quantitatively evaluate the effectiveness of the proposed modules better, we have followed your suggestion to construct a targeted evaluation benchmark. For each module, we manually selected 20 videos from multiple existing benchmark datasets, based on the following criteria:

  1. To demonstrate the capability of SSAE in the adaptive extraction of spatial structures, we selected videos featuring complex movements and intricate hand gestures.
  2. To demonstrate the capability of the TMCM in maintaining temporal motion consistency, we selected videos characterized by rapid motion, as well as instances where limb occlusion leads to the loss or jittering of detected pose keypoints.
  3. To demonstrate the capability of the FAAU in performing frame-by-frame attention alignment, we selected videos in which there are significant differences in skeletal structure between the reference image and the driving video.

Regarding the evaluation metrics, we adopt the PCK metric, supplemented by DINO and Face-based metrics, to more precisely assess both appearance preservation and control accuracy.

[email protected] (incl. missing)[email protected] (incl. missing)[email protected] (excl. missing)[email protected] (excl. missing)DINO-ICLIP-IFace-Sim
SSAE54.6172.5257.3076.210.710.820.30
61.0185.2663.0788.790.730.830.25
TMCM75.5581.0681.5788.010.690.790.23
81.9287.7386.1292.460.730.820.28
FAAU69.9983.3472.2486.260.740.830.33
74.8681.1083.1791.210.790.870.36

As shown in the table, by constructing a precise, targeted evaluation dataset and employing these additional metrics, our ablation study effectively demonstrates the efficacy of all three components through quantitative comparison.

To demonstrate these roles more clearly, we have added targeted qualitative ablation comparisons in the revised Appendix F (Fig. 24), illustrating the specific errors that arise when each module is removed and how retaining that module resolves them.

In the revised manuscript, we will strengthen the link among each module, its target failure mode, and the corresponding evidence by presenting the quantitative in Table 2 and new Table, alongside the newly added qualitative comparisons. Thank you once again for your interest in the effectiveness of our various modules, and for offering such crucial suggestions.

R2-03 (W3 & Q3): Training Efficiency and Computational Cost Analysis

Thank you for the suggestion. We agree that training steps and dataset scale alone cannot represent computational cost, which also depends on trainable parameters, input token length, and the training computation path. To address this, we have added a resource analysis across Wan-based models detailing full model parameters, newly added module parameters, trainable parameters, input resolution and frame count, token count, and disclosed training steps and dataset sizes.

To reduce the impact of hardware differences on the comparison, we adopt a unified formulation to estimate per-sample computation. For a standard training path, the computation per sample is approximated as:

Csample≈(4Pfull+2Ptrain)L,C _ {\mathrm{sample}} \approx (4P _ {\mathrm{full}} + 2P _ {\mathrm{train}})L,

where PfullP _ {\mathrm{full}} and PtrainP _ {\mathrm{train}} denote the total parameters and trainable parameters participating in that computation path, respectively, and LL denotes the number of tokens. This formula sums three components: 1) forward-pass parameter matrix operations cost roughly 2PfullL2P _ {\mathrm{full}}L, 2) backpropagating activation gradients along the network still computes input gradients on relevant paths even when weights are frozen and costs roughly 2PfullL2P _ {\mathrm{full}}L, and 3) parameter gradients are computed only for trainable weights at roughly 2PtrainL2P _ {\mathrm{train}}L. Here the factor of 2 follows the convention of counting one multiplication and one addition as two floating-point operations.

Although this estimate does not replace measured GPU hours, all compared methods are built on the Wan model family and are analyzed under common definitions of parameters, tokens, and computation paths, providing reliable evidence for comparing training compute.

For Stage 2, which the reviewer specifically highlighted, we explicitly include the computation of full conditional supervision, student unconditional distillation, and the frozen-teacher forward pass. Both full conditional supervision and student unconditional distillation involve forward and backward passes, whereas the frozen teacher only provides target predictions without backpropagation. The extra overhead of Stage 2 is accounted for in our analysis rather than treated as having the same per-step cost as the LoRA stages.

MethodFull ParamsAdded ParamsTrainable ParamsTraining StepsResolution & FramesTokensTraining DataEst. Total Compute
VACE17.34B3.05B3.05B200k1280×720, 81f75,600>1M5.70×1015×steps×global batch5.70 \times 10^{15} \times \text{steps} \times \text{global batch}
UniAnimate-DiT16.40B0.38M613.80M-832×480, 81f32,760~10k2.19×1015×steps×global batch2.19 \times 10^{15} \times \text{steps} \times \text{global batch}
RealisDance-DiT16.42B27.21M4.22B-480×832, 81f32,7601M2.43×1015×steps×global batch2.43 \times 10^{15} \times \text{steps} \times \text{global batch}
HyperMotion16.40B0.33M16.40B30k576×1024, 49f29,95219,5972.95×1015×steps×global batch2.95 \times 10^{15} \times \text{steps} \times \text{global batch}
Wan-Animate17.27B879.94MS1–S4: 17.27B; S5 (LoRA): 718.27M-1280×720, 78f72,000-S1–S4: 7.46×10157.46 \times 10^{15}; S5: 5.08×10155.08 \times 10^{15} (×steps×global batch\times \text{steps} \times \text{global batch})
SteadyDancer (Ours)16.40B1.02MS1 & S3 (LoRA): 719.70M; S2 (Full): 16.40B12k / 2k / 0.5k832×480, 81f32,7607,338S1 & S3: 2.20×10152.20 \times 10^{15}; S2: 7.52×10157.52 \times 10^{15} (×steps×global batch\times \text{steps} \times \text{global batch})

As shown in the comparison table, our newly added module parameter count is nearly identical to that of prior works, and we adopt LoRA to reduce trainable parameters, resulting in less or nearly the same compute per sample. While full-parameter fine-tuning in Stage 2 increases trainable parameters, it runs for relatively few steps and has minimal overall impact. As for global batch size and training steps, to the best of our knowledge, the compared methods exceed ours in both respects, often by several times, meaning that our overall training compute is substantially lower than that of prior works.

In addition, SteadyDancer uses 7,338 training clips totaling 10.2 hours of video, reflecting lower data requirements relative to several listed methods.

In the revised manuscript, we will clearly distinguish data efficiency, training steps, and estimated computational cost so that our efficiency claims stay aligned with the comparison methodology.

Reviewer_p2UyOpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper addresses image-based pose-driven human image animation. It combines condition reconcilation, spatial and temporal pose modulation, and a three stage training pipeline to handle mismatches between the reference image and driving poses. The experiment section provide extensive comparisons against numerous baselines across three datasets. However, the proposed method underperforms on some metrics, suggesting that the quantitative results do not consistently support the superiority suggested by the qualitative comparisons, while the qualitative results look promising. Moreover, the presentation could be improved as the paper conatins a large amount of content and it affects readability.

Critical strengths
  1. The paper addresses a well-motivated problem, particularly body-proportion mismatches between the reference image and the driving subject at test time.
  2. The qualitative results are visually impressive and demonstrate promising appearance preservation and motion coherence.
Critical weaknesses
  1. Limited justification for the proposed components. The method introduces several modules and training strages. Even though Table 2 provides abaltion results, the score difference across several configurations are marginal. A clearer explanation of why each component is necessary both qualitatively and qualtitatively and it would help establish the benefits of the overall design.
  2. Readability. The paper contains a substantial amount of material, making some tables and figures small and difficult to read. Better organization and closer placement of related tables, figures, and discussions would improve readability.
  3. Not consistently state-of-the-art scores. The quantitative results do not consistently establish the proposed model’s superiority. For example, Table 3 shows competitive performance, but the method does not achieve the best results across many metrics. This does not invalidate the qualitative improvements, but the relationship between the quantitative results and the strong qualitative claims warrants further discussion.
Questions for authors
  1. Supplementary videos (containing first 3 videos in TikTok test set, if possible) would be helpful for a more detailed comparison. I am already familiar with these videos, as I have experience working with this dataset.
7 comments

R1-01 (W1): Justification of Proposed Components and Ablation Results

Thank you for pointing out the need for a clearer explanation of the qualitative and quantitative contributions of each component. We acknowledge that the differences in a few metrics from the ablation experiments in Table 2 are minor. However, we would like to emphasize that readers should focus on the overall trends in the metrics and combine them with qualitative comparisons to gain a more detailed understanding of the role of each design.

Specifically, each module and training strategy is designed to address a specific challenge. In terms of modules, the Condition-Reconciliation Mechanism is designed to simultaneously preserve the visual information of appearance conditions and the control information of pose conditions, thereby reducing interference between the two. The Synergistic Pose Modulation Modules are used to address structural pose errors, significant discrepancies between structure and appearance, and temporal inconsistencies caused by missing or conflicting poses. In terms of training strategies, Condition-Decoupled Distillation improves generation quality, while Pose Simulation mitigates abrupt jumps in the start time.

Our existing qualitative ablation experiments demonstrate the issues that arise when specific components or strategies are removed, as well as how the complete model resolves them. Table 2 provides corresponding quantitative metrics to support these findings. For example, under the same Stage 1 training setup, the full model achieves an FVD of 330.47 on RealisDance-Val, compared with 349.98, 339.55, and 333.52 when removing SSAE, FAAU, and TMCM, respectively. Moving from Stage 2 to Stage 3 also raises reference subject preservation (I2V Subject) on X-Dance from 93.44 to 96.17 and Subject Consistency from 87.74 to 91.61. Therefore, in terms of quantitative results, this ablation experiment demonstrates improvements across most metrics, and the overall trend is reasonable, effectively illustrating the strengths of each module.

As for the issue you raised regarding the small increments in the metrics, we believe that while the existing evaluation metrics can measure overall visual quality and consistency, they fail to adequately reflect local structural errors, missing poses, and discontinuities in the initial stages, which are precisely the focus of this paper. Therefore, we hope that readers will grasp the overall trends revealed by the quantitative comparisons to learn about the functions of the module, and more importantly, alongside visualizations of the corresponding qualitative comparisons. To make this point even clearer, we have included a targeted qualitative ablation comparison analysis in the revised Appendix F (Fig. 24), which uses side-by-side visual comparisons to clearly illustrate the specific issues addressed by each component and their effectiveness. We also hope to see more refined and targeted evaluation metrics developed in the community to better measure these specific failure modes and their improvements.

Overall, while the gains on some individual metrics in the ablation study are modest, the overall quantitative trends together with targeted qualitative results support the design rationale behind our proposed modules and training strategies.

R1-02 (W2): Manuscript Readability and Layout Organization

Thank you for the suggestion, and we apologize for the inconvenience caused by the current layout. In the revised manuscript, we will improve figure dimensions, font sizes, and page layout, focusing on making the key qualitative comparisons, ablation figures, and tables easier to read. We will also place related figures and tables closer to their corresponding discussions so that readers can more easily connect our method designs with the experimental results. In addition, we will condense repetitive content and reorganize the main text and supplementary material to leave more space for the core figures and explanations.

R1-03 (W3): Quantitative Comparisons and Alignment with Qualitative Claims

Thank you for recognizing our qualitative results. We agree that SteadyDancer does not achieve the best score on every metric in the current evaluation.

To more directly evaluate pose control and reference appearance preservation, the two central goals of our work, we have added two quantitative evaluations. Specifically, we adopt PCK to measure the alignment accuracy between generated poses and driving poses, and we add DINO-I, CLIP-I, and Face-Sim to assess reference character preservation across visual feature similarity, semantic similarity, and facial identity similarity. Whether compared to the current low-level metrics, which place greater emphasis on structural similarity, or the VBench metrics, which focus more on image quality, we believe that these two metrics better reflect our controllable accuracy and image fidelity. These new results show that SteadyDancer outperforms the compared baselines in both pose control accuracy and reference appearance preservation, providing more direct quantitative support for the motion adherence and appearance retention observed in our qualitative comparisons.

X-Dance Incl. missing counts missing predictions as incorrect; excl. missing is computed only on detected keypoints.

Model[email protected] (incl. missing)[email protected] (incl. missing)[email protected] (excl. missing)[email protected] (excl. missing)Point CoverageDINO-ICLIP-IFace-SimFace Coverage
Moore-AnimateAnyone56.0160.2866.4373.1080.63%0.510.700.1172.80%
HumanVid68.2170.9173.6976.7892.12%0.540.700.0872.40%
MimicMotion58.7384.5660.6687.7694.35%0.590.720.1296.20%
Hyper-Motion39.1662.2845.4274.4980.04%0.530.700.3061.20%
animate-x45.5267.7148.5372.8291.06%0.730.790.3396.80%
UniAnimate-DiT41.7764.1844.4869.4189.52%0.680.800.4088.80%
VACE55.4474.1158.5878.3594.40%0.430.640.1294.40%
Wan-Animate59.6877.6463.0682.4692.42%0.640.780.4195.40%
Ours84.8289.4388.7193.8394.69%0.730.820.4597.20%

RealisDance-Val Incl. missing and excl. missing follow the same definitions as above.

Model[email protected] (incl. missing)[email protected] (incl. missing)[email protected] (excl. missing)[email protected] (excl. missing)Point CoverageDINO-ICLIP-IFace-SimFace Coverage
Moore-AnimateAnyone77.0182.8484.5391.3889.15%0.750.840.2883.16%
HumanVid87.1389.6793.7896.7792.16%0.770.860.3080.41%
MimicMotion30.0254.9131.9258.8590.67%0.640.800.1889.00%
Hyper-Motion66.2877.8372.3485.9788.80%0.760.850.3586.60%
animate-x30.6050.5134.5756.5987.72%0.690.810.2279.73%
UniAnimate-DiT29.6348.3032.4653.3986.49%0.710.820.2981.10%
VACE41.2962.0544.0766.4293.28%0.720.840.2387.29%
Wan-Animate31.8853.6434.5058.7088.56%0.790.870.4086.25%
Ours90.1092.8593.8696.8795.44%0.810.890.4293.13%

Our original evaluation also reflects clear strengths on several dimensions. SteadyDancer achieves the best FID and FVD on TikTok, the best FVD on RealisDance-Val, and the best results on X-Dance across reference subject preservation (I2V Subject), reference background preservation (I2V Background), Aesthetic Quality, and Imaging Quality. Compared with metrics such as SSIM and PSNR that emphasize pixel-level or structural similarity, this group of metrics evaluates feature distributions, reference appearance retention, and perceptual quality, focusing on dimensions closer to how human viewers judge overall video quality, character appearance, and visual naturalness. These scores therefore align well with the advantages seen in our qualitative comparisons. At the same time, we do not interpret these strengths as universal superiority across all evaluation dimensions. Rather, the newly added pose accuracy and reference similarity metrics directly evaluate our core goals and complement the original evaluation.

In the revised manuscript, we will include these new results and revise overly broad phrasing such as outperforming other methods across the board. Instead, we will frame our conclusions around advantages in pose control accuracy and reference appearance preservation in our evaluation, alongside competitive performance on other dimensions, keeping our claims aligned with the actual evidence.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.