acceptodds
Under review as a conference paper at ICLR 2027

CrossPortrait: Cross-Species Portrait Animation via Adversarial Motion Alignment

Abstract

Cross-species portrait animation, which aims to animate a source portrait of species A using a driving video of species B, is particularly challenging due to substantial anatomical differences in facial structures across species. Previous works typically fine-tune human portrait animation models on animal video datasets, but often suffer from driving-species leakage. To address this, we propose a novel framework for expressive cross-species portrait animation, supporting diverse scenarios including human-to-animal, animal-to-human, and cross-species animal-to-animal (e.g., cat-to-dog) animation. Specifically, we decompose the portrait motion into rigid head poses and expression keypoints. To accommodate species-specific facial structures while facilitating cross-species expression alignment, we introduce species-routed expression encoders that incorporate species-specific LoRAs into a unified expression encoder. Furthermore, we introduce expression-level adversarial domain adaptation (ADA) to suppress driving-species-specific information in the expression representations, thereby mitigating species leakage and promoting a shared expression space across species. Extensive experiments demonstrate that our method enables expressive and identity-preserving portrait animation across diverse species combinations, while effectively reducing driving-species leakage.

Primary area:
generative models
Keywords:
Computer Vision、Generative Models、Diffusion Models、Portrait Animation、Cross-Domain Transfer

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_mVsEOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper proposes a novel framework for cross-species portrait animation, to metigate the leakage of species-specific feature during motion and expression transferring. There are three major contributions:

  1. A cross-species portrait-motion alignment framework which disentangles portrait features into rigid head motion represented with dense pose-flow, and facial deformation represented with shared implicit-keypoint tokens.
  2. A species-routed expression encoder, preserving capacity for anatomical differences while providing a shared expression-conditioning interface by species-specific LoRA adapters.
  3. An expression-level adversarial domain adaptation (Ex-ADA) to mitigate species leakage.

The authors conduct major experiments on datasets containing three different species (human, cat and dog) and visualize some results on unseen species. Qualitative/quantitative evaluation and a user study demonstrate better performance over previous methods.

Critical strengths
  1. The proposed method is technically sound. How to disentangle motion, expression and identity-specific feature is one of the biggest challenges in motion/expression transferring for generation. The integration of species-specific LoRA adapters and adversarial expression alignment provides a reasonable approach under the settings considered in this paper.

  2. The qualitative and quantitative results are promising.

Critical weaknesses

My main concerns are related to the generalizability, scalability and architectural limitation, most of which the authors have acknowledged in the limitations section. Although in the appendix the authors show results with unseen species as the source, it seems that framework cannot directly handle unseen species as driving inputs, since no corresponding adapter exists for an unseen species label and the classifier could be incompatible. Furthermore, the experiments are conducted on only three species, which is a relatively limited number. Considering the design of LoRA tuning and classifier are directly related to the number of labels, it remains unclear whether the proposed method can maintain its performance when trained on a larger and more diverse set of species.

A minor concern is about the quantitative evaluation. For A to H, the authors calculated AED of SMIRK features, APD of SMIRK features and CSIM of ArcFace features for expression, pose and identity, respectively. I understand that it is difficult to directly evaluate the results regarding these similarities since there is no ground truth nor golden standards for evaluation for generation tasks. And I agree that measuring feature distances using relevant pretrained feature extractors is a reasonable approach. However, it would be more convincing if the authors could report results obtained using multiple different feature extractors for each evaluation aspect. This would help mitigate potential biases introduced by relying on a single pre-trained extractor for each metric. Some candidates for consideration: AdaFace, 3DDFA, EMOCA, Action Units, etc.

Questions for authors

Please see the weaknesses

Reviewer_92YvOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper aims at the cross-species portrait animation, where a source portrait from one species is animated using a driving video from a potentially different species, while preserving the source’s identity and species-specific morphology. The authors propose CrossPortrait, a two-stage framework built on the LivePortrait pipeline. In Stage 1, the method decomposes portrait motion into rigid head pose and facial expression. In Stage 2, pose and expression features extracted from the frozen encoders are injected into a DiT-based video diffusion generator to produce the final animated video. Experiments evaluate three transfer directions. Results on same-species self-reenactment show improvements over baselines.

Critical strengths
  1. Cross-species portrait animation is a compelling and under-explored task that sits at the intersection of facial animation and domain adaptation. The paper correctly identifies species leakage as the central failure mode of existing methods when applied across species, and the decomposition of motion into rigid pose and deformable expression is a sensible and standard formulation that aligns with prior portrait animation literature.

  2. The decision to apply adversarial domain adaptation only to expression features is methodologically sound. Pose can be represented as a universal geometric transformation across species, while source appearance must be preserved as-is. Restricting the adversarial loss to the expression latent space strikes a good balance between aligning semantic expression content and preserving identity and geometry.

  3. The paper systematically evaluates three transfer directions and provides clean component ablations that isolate the contribution of each proposed module.

  4. The causal temporal refiner for expression tokens is a nice detail that improves temporal consistency of fine-grained facial motions. The ablation clearly shows that removing it degrades expression and overall preference, supporting its value for coherent video animation.

Critical weaknesses
  1. The paper only evaluates against a small set of general portrait animation methods, most of which are not designed for cross-species animation and have limited or no official support for animal driving inputs. Strikingly, the paper entirely omits comparisons with specialized open-source methods explicitly designed for cross-species or cross-domain motion transfer, such as AnimateZoo, and recent open-source expression retargeting methods that support cross-identity and cross-species transfer.

Because the paper only compares against generic portrait models with ad-hoc animal fine-tuning, rather than against purpose-built cross-species animation methods, it cannot demonstrate that CrossPortrait offers any meaningful advantage over the current state of the art. Many of the baseline methods are also functionally incomplete on the evaluation tasks, further reducing the informativeness of the comparisons. This severely weakens the paper’s claim of state-of-the-art performance.

  1. All core cross-species results rely exclusively on pairwise user preference studies. While user studies have value, they are subjective, have limited statistical power, and provide only relative ranking rather than absolute measurement of quality. The paper does include quantitative metrics (AED, APD) based on SMIRK facial analysis, but SMIRK is trained on human faces and its accuracy and validity on animal faces are not established. There are no objective, cross-species-validated metrics for expression fidelity or identity preservation, making it difficult to independently verify the magnitude of improvements or compare results across papers.

  2. The core technical components are largely established techniques adapted to the cross-species setting. Species-routed LoRA adapters are a standard approach for domain-specific capacity around a shared backbone, and adversarial domain adaptation with gradient reversal is a classic technique dating back over a decade. The overall pipeline follows the same high-level architecture as numerous recent portrait animation methods. The paper does not introduce fundamentally new algorithms or representations; the contribution is primarily integration and application of existing components to the cross-species task.

  3. The entire study is restricted to only three species: human, cat, and dog. Cross-species animal experiments are limited exclusively to cat↔dog transfer. While the paper shows qualitative examples of unseen source species driven by human motion, it does not test unseen driving species, which is arguably the harder and more practically relevant generalization setting. The method also has not been evaluated on more diverse facial structures (e.g., birds, rodents, horses) or non-mammalian species, so the generalizability of the approach remains largely untested.

  4. The cat and dog training datasets are collected internally and are not publicly available. The paper provides only very high-level descriptions of the data without details on diversity, capture conditions, pose/expression distribution, or annotation quality. Since animal data quality and diversity are critical drivers of cross-species performance, the unavailability of the training data makes it difficult to reproduce results or disentangle data effects from architectural contributions.

Questions for authors

see above.

Reviewer_eLGJOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This work transfers facial expressions and head movements among humans, cats, and dogs to animate a single portrait image. For example, given a cat image and a video of a person opening their mouth and turning their head, the method generates a video of the cat performing corresponding movements. The work focuses on preventing appearance characteristics of the driving species from being transferred along with the motion.

Critical strengths
  1. The paper demonstrates the model’s effectiveness across diverse evaluation settings, including animal self-reenactment, human-to-animal, animal-to-human, and cat-to-dog and dog-to-cat motion transfer.
  2. The video examples provide detailed demonstrations of effective portrait animation driven by motion from a different species.
Critical weaknesses
  1. Although Ex-ADA makes it harder to distinguish humans, cats, and dogs from the expression features, this does not guarantee that corresponding actions, such as mouth opening, map to a shared representation across species. The approach therefore does not appear to directly supervise cross-species correspondences. While this does not establish that alignment fails, I believe the claimed alignment has not been sufficiently verified.
  2. While the user study demonstrates a preference for the proposed method, it provides limited evidence about precise motion accuracy, such as the degree of mouth opening or the timing of eye closure.
  3. The No-Alignment baseline removes LoRA and Ex-ADA while also combining the pose and expression encoders into a single encoder. It therefore changes both the alignment mechanism and the encoder architecture, making it difficult to isolate the contribution of alignment.
  4. Evidence for generalization to unseen species is limited. For example, it is unclear whether the method would extend to motion transfer between humans and crocodiles. I consider this a minor limitation.
Questions for authors

Please refer to the weaknesses above. I am satisfied with what this work has achieved, and I believe it would be a strong contribution if these weaknesses were adequately addressed.

Comments on the paper

2 comments
AuthorsOpenReview ↗

Follow-up on Author Responses

Dear Reviewers,

Thank you again for your time and thoughtful feedback. We would appreciate your feedback on whether our responses address your concerns. If any questions remain unresolved, we would be happy to provide further clarification.

If our responses have resolved your concerns, we would be grateful if you could consider updating your assessment and score accordingly.

Thank you for your consideration.

Best regards,
The Authors

Dear reviewers,

As the rebuttal period is coming to an end, I kindly remind you to check the authors' responses. Please indicate whether your concerns have been adequately addressed. Many thanks!

Best, AC

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.