acceptodds
Under review as a conference paper at ICLR 2027

WeLike2Party! In-Context Motion Transfer for Multi-Human Image Animation

Abstract

Human image animation aims to transfer motion from a driving video to subjects in a reference image. Despite remarkable progress in video generation, achieving high-fidelity animation of multiple interacting subjects remains a challenge. Many existing approaches rely on explicit motion representations such as 2D skeletons or parametric body meshes and struggle to preserve identity-motion binding under inter-person occlusion. To address this limitation, we propose WeLike2Party, a multi-human animation framework built on direct in-context video conditioning without explicit pose or mesh extraction at inference. We further introduce Reference Asymmetric RoPE Conditioning to preserve fine-grained appearance details, and Identity Binding Supervision to associate each reference identity with its intended motion trajectory. To support cross-identity training, we construct MotionTwin, a large-scale synthetic dataset comprising K cross-identity video pairs with shared subject and camera motions, totaling hours of photorealistic video. We additionally present MotionTwin-Bench, a cross-identity benchmark specifically designed to evaluate subject-level visual fidelity and identity-motion binding. Extensive experiments on MotionTwin-Bench and real-world videos demonstrate that WeLike2Party outperforms recent state-of-the-art methods in subject-level visual fidelity, identity-motion binding, and overall perceptual quality, particularly in multi-person interactions with substantial occlusion.

Primary area:
generative models
Keywords:
Human Image Animation, Generative Model

Reviews

Among:
Shape:
1234this paper, Top 4%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.33 over 3 reviews, above 96% of the others.
Reviewer_hoKMOpenReview ↗
Rating
4(+1)
Confidence
3
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

This paper proposes a multi-human animation framework built on direct in-context video conditioning without explicit pose or mesh extraction at inference. They further introduce Reference Asymmetric RoPE Conditioning to preserve fine-grained appearance details, and Identity Binding Supervision to associate each reference identity with its intended motion trajectory. They also construct MotionTwin, a dataset containing approximately 14.4K cross-identity video pairs, and the MotionTwin-Bench evaluation set. The validation includes quantitative comparisons and ablations on MotionTwin-Bench, automatic metrics and qualitative comparisons on real-world inputs, and a human preference study.

Critical strengths
  1. The author proposes a new way for constructing cross identity video pair dataset and build the large-scale MotionTwin dataset which contains both single-person and multi-person pairs, providing direct supervision for effective motion transfer and identity-motion binding. It supports the training and evaluation of the proposed method and could also provide a valuable resource for future research.
  2. The author proposes novel improvements, particularly IBS, which effectively enhances the model’s ability to preserve information about each subject, and conduct extensive ablation studies on the proposed components.
Critical weaknesses

The paper lacks powerful automatic metrics for identity binding. IAA uses color features to match subjects. It may confuse people wearing similar-colored clothes or fail to detect identity swaps. Ground truth masks may not align with generated subjects if their body shapes, poses, or positions are different. In those cases, the score may be affected even when identities are correct.

Questions for authors

None.

5 comments
AuthorsOpenReview ↗

Official Response to Reviewer hoKM

W1. Reliability of Identity Assignment Metrics

The paper lacks powerful automatic metrics for identity binding. IAA uses color features to match subjects. It may confuse people wearing similar-colored clothes or fail to detect identity swaps. Ground truth masks may not align with generated subjects if their body shapes, poses, or positions are different. In those cases, the score may be affected even when identities are correct.

We appreciate the reviewer's concern about the limitations of color-based identity evaluation.

To complement the original color-based IAA, we evaluate identity binding on all 225 multi-person pairs using CLIP-ReID and Gemini 3.8 Flash. CLIP-ReID uses identity embeddings for assignment, while Gemini serves as a VLM judge [1], comparing full generated frames with paired GT frames using its own visual grounding.

MethodOriginal Color IAA ↑CLIP-ReID IAA ↑Gemini 3.8 Flash IAA ↑
MultiAnimate0.83600.88640.7089
Wan-Animate-20.60350.66270.8504
SCAIL0.85140.88150.8496
SCAIL-20.84060.89460.8533
WL2P (Ours)0.95690.98450.9807

Spatial misalignment. Gemini assesses identities using global scene context, without restricting generated subjects to fixed GT-mask regions. Baseline scores and rankings differ between the region-based and full-frame evaluators, underscoring the value of assessing identity binding beyond localized crops. Especially, we observe that Wan-Animate-2 tends to render subjects larger than their GT counterparts, which can adversely affect identity matching within fixed GT-mask regions. WL2P ranks first under both, supporting its advantage when the evaluation does not require exact alignment with GT masks.

Similar reference colors. On the 57 pairs with the most similar reference colors, WL2P achieves the highest score under all three evaluators.

MethodOriginal Color IAA ↑CLIP-ReID IAA ↑Gemini 3.8 Flash IAA ↑
MultiAnimate0.89160.92210.8889
Wan-Animate-20.55900.63980.8421
SCAIL0.87360.91850.8977
SCAIL-20.84140.94090.9123
WL2P (Ours)0.92210.97850.9591

Together, these results support WL2P's advantage beyond color-based matching and fixed GT-mask crops. We welcome suggestions for additional identity-aware metrics and would be happy to incorporate them into our evaluation during the remaining discussion period.

[1] Cong et al. OmniXRef: A Unified Framework for Multimodal Multi-Reference Video Editing. 2026. arXiv:2610.22744.

Thank you for the additional evaluations. They substantially alleviate my main concern. However, the metrics are not yet fully validated. WL2P ranking first across evaluators supports its advantage, but does not establish how accurately these metrics detect identity-binding errors. Agreement with human judgments would provide stronger evidence. But overall,given the creativity of the work and the thorough, thoughtful rebuttal, I am raising my score from 3 to 4.

Thanks for the response. I will maintain my rating.

Reviewer_Ex2jOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

WeLike2Party animates the people in a reference image using a driving video, with an emphasis on maintaining identity-to-motion correspondence during multi-person interactions. It conditions a Wan diffusion transformer directly on reference-image and driver-video tokens, avoiding external pose/mesh estimation and binding masks at inference. Reference Asymmetric RoPE Conditioning maps a higher-resolution reference grid into the target grid's coordinate range; Identity Binding Supervision uses synthetic instance labels to supervise target-to-reference attention during high-noise training. The paper also introduces MotionTwin: 14.4K paired cross-identity examples totaling 84.3 hours of rendered video, including groups of up to seven people, and a 300-pair benchmark. Experiments compare recent animation methods on synthetic reconstruction/subject-fidelity and color-signature assignment metrics, evaluate 161 real-world pairs with VBench variants, and conduct a 20-participant study on 20 real-world clips. Ablations support both proposed components. Inference remains expensive: approximately 6.5 minutes for 81 frames at 480p on one B200.

Critical strengths
  1. This paper presents a well-targeted solution to multi-person identity-motion binding with component-level evidence. IBS supervises which reference person's tokens a target subject attends to, without requiring test-time instance masks. RARC has a controlled high-resolution-reference comparison rather than attributing every gain to additional pixels. Table 3 shows that both components improve over a vanilla in-context model, with their combination increasing IAAcross from 0.8891 to 0.9496. This is consequential because the paper identifies a failure mode that ordinary reconstruction or global video-quality metrics can miss.

  2. This paper presents a useful paired synthetic resource and complementary real-input validation. MotionTwin provides cross-identity videos with matched motion/camera trajectories and instance correspondence, including multi-human interactions that are difficult to collect as paired real data. The paper supplements same-pipeline benchmark results with real-world comparisons and a randomized, anonymized user study. Together these provide a credible initial case that the training construction is useful beyond its renderer.

Critical weaknesses

The scope of generalization is not sufficiently characterized. Appendix C states that benchmark clips are held out, but does not specify disjointness at the source-recording/choreography, identity/garment asset, and environment levels; the test set shares the training rendering pipeline.

Real-world aggregate VBench scores do not directly measure identity assignment, and the dedicated binding preference study covers only 20 clips. Performance by person count, appearance similarity, and occlusion severity can be used to substantiate the broader multi-human claims.

Questions for authors

See weaknesses.

1 comment
AuthorsOpenReview ↗

Official Response to Reviewer Ex2j

W1. Train–Test Separation and Generalization

The scope of generalization is not sufficiently characterized. Appendix C states that benchmark clips are held out, but does not specify disjointness at the source-recording/choreography, identity/garment asset, and environment levels; the test set shares the training rendering pipeline.

We appreciate the request to clarify train–test separation and assess generalization.

We rechecked the split and confirm that training and test sets are disjoint at the source-recording/choreography, identity/garment-asset, and environment levels. The rendering pipeline is shared, while the underlying motions and assets are fully held out.

Real-world identity-binding evaluation. Gemini receives the reference image and a driving frame with corresponding subject numbers, together with the full generated frame. These numbers specify the intended identity–motion correspondence without a GT target video. In particular, a frame is correct only when every evaluated subject matches its assigned reference identity.

MethodGemini 3.8 Flash IAA ↑
MultiAnimate0.5125
Wan-Animate-20.5925
SCAIL0.7024
SCAIL-20.6951
WL2P (Ours)0.8292
VLM-based evaluation is also used in OmniXRef [1], which employs Gemini 3.8 Flash to assess reference adherence in video editing.

WL2P achieves the highest Gemini IAA on real-world inputs, providing further evidence of generalization beyond the shared rendering pipeline. We have clarified the split criteria in Appendix C and added the real-world evaluation protocol and results in Appendix D, with the complete VLM prompts and scoring rules provided in Appendix G.

W2. Real-World Breakdown Analysis and Binding Preference

Real-world aggregate VBench scores do not directly measure identity assignment, and the dedicated binding preference study covers only 20 clips. Performance by person count, appearance similarity, and occlusion severity can be used to substantiate the broader multi-human claims.

We thank the reviewer for suggesting a more detailed real-world evaluation and a larger binding-preference study.

Following the reviewer's suggestion, we analyze real-world identity binding across person count, reference appearance similarity, and occlusion duration. WL2P achieves the highest Gemini IAA in every evaluated group, supporting its consistent advantage in preserving identity–motion correspondence across diverse real-world conditions.

Each entry is WL2P / strongest baseline, with the strongest baseline selected separately for each group based on Gemini scores.

GroupGemini 3.8 Flash IAA ↑
2 subjects0.9811 / 0.8912 (SCAIL)
3–4 subjects0.8160 / 0.6853 (SCAIL)
5–7 subjects0.7292 / 0.5970 (SCAIL-2)
Distinct appearance0.8590 / 0.7833 (SCAIL)
Intermediate appearance0.7949 / 0.6795 (SCAIL-2)
Similar appearance0.8333 / 0.7500 (SCAIL-2)
Occlusion <0.5 s0.8898 / 0.7175 (SCAIL)
Occlusion 0.5–1 s0.6615 / 0.6567 (SCAIL-2)
Occlusion ≥1 s0.8633 / 0.8485 (SCAIL-2)

Expanded binding preference study. We expanded the study to 40 real-world clips evaluated by 20 participants, who rank anonymized outputs by identity–motion binding.

MethodIdentity–motion binding score ↑ (mean ± SD)
MultiAnimate2.429 ± 1.315
Wan-Animate-22.419 ± 1.147
SCAIL3.006 ± 1.408
SCAIL-23.081 ± 1.289
WL2P (Ours)4.065 ± 1.231

WL2P also receives the highest mean binding-preference score, providing complementary support from human judgments.

[1] Cong et al. OmniXRef: A Unified Framework for Multimodal Multi-Reference Video Editing. 2026. arXiv:2610.22744.

Reviewer_SAjEOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes WeLike2Party (WL2P), an in-context motion transfer framework for multi-human image animation that aims to preserve subject appearance and identity-motion correspondence, particularly in challenging scenarios involving inter-person occlusion. Unlike conventional pose- or mesh-based approaches, WL2P directly conditions video generation on a reference image and a driving video without requiring explicit motion representations at inference time. The framework introduces two key components: Reference Asymmetric RoPE Conditioning (RARC), which preserves fine-grained appearance details through high-resolution reference encoding and spatial positional rescaling, and Identity Binding Supervision (IBS), which encourages correct associations between reference identities and target motion trajectories through attention supervision. The authors further construct MotionTwin, a synthetic dataset containing 14.4K cross-identity video pairs, and MotionTwin-Bench, a benchmark comprising 300 pairs for evaluating visual fidelity and identity-motion binding. Experiments on MotionTwin-Bench and real-world videos demonstrate improvements over recent state-of-the-art methods in visual quality, subject-level fidelity, and identity-motion correspondence.

Critical strengths

None

Critical weaknesses

None

Questions for authors

Q1. Reliability of Identity Assignment Metrics. IAA and IAAcross rely on ground-truth target instance masks and simple color-based appearance descriptors. How reliable are these metrics when generated subjects deviate from the ground-truth spatial regions, or when subjects have similar clothing colors but distinct facial identities? Could the authors validate these metrics against human identity-matching judgments or stronger identity-aware representations?

Q2. Identity Consistency Under Prolonged Occlusion. IBS supervises target-to-reference attention but does not explicitly enforce temporal identity consistency across frames. How does the method perform when subjects undergo prolonged or complete occlusions and subsequently reappear? Could the authors report identity-switching rates under different occlusion durations and compare the results with and without IBS?

Q3. Scalability to Larger Groups. MotionTwin contains interactions involving up to seven subjects, whereas the identity assignment procedure described in Appendix C specifies (P \leq 4). Are scenes containing five to seven subjects included in the quantitative identity-binding evaluation? If not, could the authors report performance separately by the number of subjects, particularly for larger groups with frequent occlusions and interactions?

4 comments
AuthorsOpenReview ↗

Official Response to Reviewer SAjE

Q1. Reliability of Identity Assignment Metrics

IAA and IAAcross rely on ground-truth target instance masks and simple color-based appearance descriptors. How reliable are these metrics when generated subjects deviate from the ground-truth spatial regions, or when subjects have similar clothing colors but distinct facial identities? Could the authors validate these metrics against human identity-matching judgments or stronger identity-aware representations?

We thank the reviewer for raising these concerns about metric reliability.

We complement the original color-based IAA with CLIP-ReID and Gemini 3.8 Flash on all 225 multi-person pairs. CLIP-ReID uses identity embeddings for assignment, while Gemini evaluates full generated frames against paired GT frames using its own visual grounding.

MethodOriginal Color IAA ↑CLIP-ReID IAA ↑Gemini 3.8 Flash IAA ↑
MultiAnimate0.83600.88640.7089
Wan-Animate-20.60350.66270.8504
SCAIL0.85140.88150.8496
SCAIL-20.84060.89460.8533
WL2P (Ours)0.95690.98450.9807

VLM-based evaluation is also used in OmniXRef [1], which employs Gemini 3.8 Flash to assess reference adherence in video editing.

Spatial misalignment. Gemini assesses identities using global scene context, without restricting generated subjects to fixed GT-mask regions. Baseline scores and rankings differ between the region-based and full-frame evaluators, underscoring the value of assessing identity binding beyond localized crops. In particular, we observe that Wan-Animate-2 tends to render subjects larger than their GT counterparts, which can adversely affect identity matching within fixed GT-mask regions. WL2P ranks first under both, supporting its advantage when the evaluation does not require exact alignment with GT masks.

Similar reference colors. On the 57 pairs with the most similar reference colors, WL2P achieves the highest score under all three evaluators.

MethodOriginal Color IAA ↑CLIP-ReID IAA ↑Gemini 3.8 Flash IAA ↑
MultiAnimate0.89160.92210.8889
Wan-Animate-20.55900.63980.8421
SCAIL0.87360.91850.8977
SCAIL-20.84140.94090.9123
WL2P (Ours)0.92210.97850.9591

Together, these results support WL2P's advantage beyond color-based matching and fixed GT-mask crops. We welcome suggestions for additional identity-aware metrics and would be happy to incorporate them into our evaluation during the remaining discussion period.

[1] Cong et al. OmniXRef: A Unified Framework for Multimodal Multi-Reference Video Editing. 2026. arXiv:2610.22744.

Q2. Identity Consistency Under Prolonged Occlusion

IBS supervises target-to-reference attention but does not explicitly enforce temporal identity consistency across frames. How does the method perform when subjects undergo prolonged or complete occlusions and subsequently reappear? Could the authors report identity-switching rates under different occlusion durations and compare the results with and without IBS?

We appreciate the suggestion to evaluate identity consistency across occlusion durations.

We compare the full model(with IBS) and the variant without IBS on 75 additional MotionTwin-Bench pairs with five to seven subjects and frequent inter-person occlusions.

Performance by occlusion duration. We evaluate the occluded subject's identity accuracy after reappearance, grouped by occlusion duration. Each entry is full / w/o IBS.

Occlusion durationCLIP-ReID accuracy ↑Gemini 3.8 Flash accuracy ↑
<0.5 s0.8284 / 0.79490.8496 / 0.8015
0.5–1 s0.9716 / 0.91980.9470 / 0.8933
≥1 s0.8750 / 0.82350.8974 / 0.8492

Performance across IoU groups. We additionally evaluate identity accuracy in two groups defined by IoU, using 0.879 as the threshold.

IoU groupCLIP-ReID accuracy ↑Gemini 3.8 Flash accuracy ↑
Lower IoU (<0.879)0.8535 / 0.80310.9189 / 0.8354
Higher IoU (≥0.879)0.8348 / 0.78960.8991 / 0.8098

The full model achieves higher identity accuracy in every duration group and both IoU groups under both evaluators, supporting the contribution of IBS to identity preservation after occlusion in crowded scenes.

We also provide qualitative comparisons of the full model and the variant without IBS on the revised paper and our project page.

AuthorsOpenReview ↗

Official Response to Reviewer SAjE

Q3. Scalability to Larger Groups

MotionTwin contains interactions involving up to seven subjects, whereas the identity assignment procedure described in Appendix C specifies (P ≤ 4). Are scenes containing five to seven subjects included in the quantitative identity-binding evaluation? If not, could the authors report performance separately by the number of subjects, particularly for larger groups with frequent occlusions and interactions?

We thank the reviewer for highlighting the need to evaluate larger groups.

The original quantitative evaluation covered two to four subjects. We now extend it to 75 additional pairs containing five to seven subjects, including crowded interactions with frequent occlusions.

MotionTwin-Bench results on the additional 75 pairs.

MethodPSNR ↑LPIPS ↓FVD ↓mLPIPS ↓IAAcross ↑
MultiAnimate19.7300.1640201.930.12490.5895
Wan-Animate-216.9670.2985592.940.19770.2864
SCAIL17.1280.2446256.370.16200.5970
SCAIL-217.5040.2330198.160.15990.6019
WL2P (Ours)24.8070.088467.710.06980.8312

Binding performance by person count. Each entry is WL2P / strongest baseline, with the strongest baseline selected separately for each metric and person count.

Person countOriginal Color IAA ↑CLIP-ReID IAA ↑Gemini 3.8 Flash IAA ↑
50.8751 / 0.71360.9033 / 0.76410.9136 / 0.8148
60.8049 / 0.53480.8046 / 0.59850.8986 / 0.8551
70.8138 / 0.63160.8238 / 0.70840.9200 / 0.8733

WL2P maintains strong visual fidelity and identity binding in larger groups with frequent inter-person occlusions, achieving the highest score under all three identity evaluators at each person count.

Thank you for the authors' response. I will maintain my positive rating.

AuthorsOpenReview ↗

Official Response to Reviewer SAjE

Thank you very much for your careful review and for maintaining your positive rating! Your constructive comments helped us strengthen the evaluation of identity binding and clarify the effectiveness of IBS. We sincerely appreciate your time and thoughtful feedback.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary of revisions

We thank the reviewers for their constructive feedback. We have incorporated the following evaluations and clarifications into the revised manuscript, with the corresponding additions highlighted in color.

  1. Reliability of identity-binding evaluation (SAjE Q1; hoKM W1). We added complementary CLIP-ReID and full-frame Gemini 3.8 Flash evaluations on all 225 original multi-person pairs, together with an analysis of 57 pairs with similar reference colors (Appendix D, Tables 5–6).

  2. Identity preservation under occlusion and the contribution of IBS (SAjE Q2). We added full-model versus w/o-IBS comparisons of post-reappearance subject identity accuracy across occlusion durations. In addition, we report identity accuracy across overlap groups on the 75-pair extension. We also added qualitative comparisons under inter-person occlusion (Appendix D, Table 7 and Figure 7).

  3. Scalability to larger groups (SAjE Q3). We extended MotionTwin-Bench with 75 additional pairs containing five to seven subjects and frequent inter-person occlusions. We report visual-fidelity results and identity-binding performance by person count under all three evaluators (Appendix D, Tables 8–9).

  4. Train–test separation and real-world generalization (Ex2j W1). We clarified that the training and benchmark sets use disjoint source recordings and choreographies, identity and garment assets, and background HDRIs, while sharing the rendering pipeline (Appendix C). We also added real-world identity-binding evaluation using reference–driving subject correspondence without GT target videos (Appendix D, Table 10; Appendix G.2).

  5. Real-world breakdowns and human evaluation (Ex2j W2). We added analyses by person count, reference appearance similarity, and occlusion duration (Appendix D, Table 11). We also expanded the identity–motion binding preference study to 40 real-world clips evaluated by 20 participants, reporting the mean and standard deviation of the preference scores (Appendix D, Table 12).

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.