acceptodds
Under review as a conference paper at ICLR 2027

MoZoo:Bringing Animal Meshes to Life with Video Diffusion

Abstract

Creating cinematic-quality animal effects is highly valuable but remains labor-intensive, as producing realistic animal appearance and motion still relies heavily on manual animation. Recent advances in video generation offer a promising path toward automating this process, yet existing models provide limited structural control over animal motion and geometry. Progress toward such controllable generation is further hindered by the lack of large-scale data pairing animated meshes with realistic RGB videos. We introduce mesh-guided animal appearance synthesis, a new formulation in which an animated mesh specifies geometry and motion, while a text, image, or video reference specifies appearance. To enable learning under this formulation, we construct MoZoo-Data, a 62K-clip synthetic-to-real dataset that combines aligned rendered pairs with filtered pseudo-pairs recovered from real videos via inverse mesh generation. Building on this data, we propose MoZoo, an in-context video diffusion model that synthesizes realistic animal videos while following mesh-defined motion and preserving reference appearance. MoZoo introduces RAR-RoPE to establish correspondence between mesh and target representations while handling temporally asynchronous references, and ADA to incorporate structural and appearance conditions without allowing noisy target features to contaminate the conditioning streams. We further introduce MoZooBench, a 120-case benchmark for evaluating structural consistency and appearance fidelity. Experiments show that MoZoo outperforms baselines trained on the same data and generalizes across reference modalities, geometries, and motion patterns.

Primary area:
generative models
Keywords:
video generation, Diffusion model

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_AFjqOpenReview ↗
Rating
1(−1)
Confidence
4
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper introduces MoZoo, a video diffusion framework for mesh-guided animal appearance synthesis, generating photorealistic animal videos by combining the motion of an animated mesh with an appearance specified by text, image, or video reference. To address the scarcity of paired training data, the authors build MoZoo-Data, a 62K-clip dataset combining UE5-rendered synthetic pairs and real animal videos processed via inverse mesh generation. The model uses Role-Aware RoPE (RAR-RoPE) for temporal alignment and Asymmetric Decoupled Attention (ADA) for information routing, and is evaluated on a 120-case MoZooBench.

Critical strengths
  1. The problem of mesh-guided animal appearance synthesis is well-motivated.
  2. MoZoo-Data and especially the inverse-mesh pipeline that extracts pseudo-pairs from real videos are valuable contributions.
  3. RAR-RoPE cleanly handles the temporal misalignment between reference and target/mesh tokens, and its ablation shows clear failure modes when removed.
  4. The model shows impressive cross-species texture transfer and multi-entity handling.
Critical weaknesses
  1. The model's behavior resembles texture transfer onto a pre-defined mesh video rather than genuine mesh-guided appearance synthesis. It fails to truly decouple motion from appearance; the generated video heavily retains the overall body shape and proportions of the input mesh video. The mesh already determines the full layout, so the network's role reduces to painting textures onto a known proxy, which limits its capability to synthesize vivid video.
  2. Because the overall body shape is largely determined by the input mesh video, the showcased results are dominated by animals of similar stature. I wonder whether the model can generalize when the reference image is an AIGC-generated sample with unusual appearances rather than a real photo.
  3. There are many existing models that take a coarse render or layout as the condition. For example, LTX-2.5-22b-IC-LoRA-Layout-To-Render accepts a rough white-model/layout render and synthesizes the final video. I think there should be a comparison.
  4. The methodological contribution is relatively incremental. ADA is a standard masked attention pattern that decouples condition streams. The RAR-RoPE is also a minor modification .
Questions for authors
  1. Have you compared RAR-RoPE against simpler alternatives such as adding explicit modality/role embeddings (e.g., learnable mesh/reference/target tokens) ? Without this, the necessity of the custom RoPE scheme is unclear. 

  2. The pipeline and data are built around quadruped animals. Can this method and pipeline be extended to non-quadruped categories (e.g., birds, fish, or humans).

9 comments
AuthorsOpenReview ↗

Response to Reviewer AFjq(part1/2)

Thank you for your positive evaluation of our work and for the valuable suggestions provided. Below, we address your concerns point-by-point.

W1

We would like to clarify that MoZoo goes beyond texture transfer. Our design follows the workflow of animators, who use mesh videos to define structure and motion before grooming and rendering fur, a process that is often time-consuming and repetitive. MoZoo generates realistic animal videos by adapting fur characteristics from a reference animal to a target mesh, even when their body shapes differ.

W2 & Q2 — Cross-shape, AIGC, and non-quadrupedal examples

We thank the reviewer for the suggestion. We added two sets of examples to evaluate generalization.

  1. Appearance transfer across body shapes and with AIGC references. MoZoo transfers appearance when the reference animal and the input coarse mesh differ in body shape, or when the reference image is AI-generated and depicts an unusual appearance. The results are available here.

  2. Non-quadrupedal animals. We added examples of birds and fish. Our data construction pipeline produced the coarse mesh videos, and MoZoo generated the corresponding output videos. The results are available here.

Our current training data provide limited coverage of non-quadrupedal animals and unusual appearances. We expect that broader coverage would make MoZoo’s results more consistent in these settings.

W3 — Comparison with coarse-render baselines

Thank you for raising this question. We added quantitative and qualitative comparisons with LTX-2.5-22b-IC-LoRA-Layout-To-Render (released on October 1, 2026, after the submission deadline). LTX supports both image and text guidance. Under image guidance, it struggles to preserve the reference appearance throughout the video and shows limited cross-species texture transfer in our evaluation. The video comparisons are available here.

Table R10. LTX comparison on the expanded real-world test set.

MethodVideo QualityPaired-Target Fidelity
Subject Consist. ↑Background Consist. ↑Motion Smooth. ↑Imaging Qual. ↑Aesthetic Qual. ↑PSNR ↑LPIPS ↓SSIM ↑
LTX-2.5-22b-IC-LoRA-Layout-To-Render96.4292.3296.7763.4652.4318.160.0840.887
MoZoo (Image)97.8398.8099.3769.4556.6522.160.0750.920
MoZoo (Video)97.9898.9299.4569.5156.9122.390.0700.925

Table R11. LTX comparison on the rendered mesh test set.

MethodVideo QualityPaired-Target Fidelity
Subject Consist. ↑Background Consist. ↑Motion Smooth. ↑Imaging Qual. ↑Aesthetic Qual. ↑PSNR ↑LPIPS ↓SSIM ↑
LTX-2.5-22b-IC-LoRA-Layout-To-Render95.9792.1396.3162.8952.1319.320.0870.893
MoZoo (Image)97.6698.8299.2469.3660.9821.340.0740.913
MoZoo (Video)97.8698.9499.3369.4061.0221.540.0690.920
AuthorsOpenReview ↗

Response to Reviewer AFjq(part2/2)

w4 Methodological contribution

As reviewers BUyi and No9k recognize, MoZoo addresses a practical problem in controllable animal video generation. Our contribution is a complete pipeline, from training data construction to model design, for rendering mesh-defined animal animations with independently specified appearance.

1. A unified conditioning framework. MoZoo takes a rendered mesh animation and an appearance reference in text, image, or video form, and generates a realistic animal video that follows the mesh-defined geometry and motion while matching the reference appearance. Within this framework, RAR-RoPE encodes the correspondence between synchronized mesh and target frames while accommodating asynchronous references. ADA allows target tokens to access both structural and appearance information while protecting the conditioning streams from feedback from noisy target tokens.

2. A scalable data-construction pipeline and MoZoo-Data. Real animal videos provide diverse appearances but generally lack corresponding mesh animations. We address this bottleneck by training an inverse generation model on paired synthetic renders and applying it to real videos to obtain coarse mesh proxies. MoZoo-Data combines these real-video pseudo-pairs with aligned synthetic renders, providing approximately 62K training clips. This enables training on diverse real animal footage for mesh-guided appearance synthesis without requiring explicit 3D reconstruction of each animal.

Realizing this task requires the data pipeline and conditioning framework to work together. Our same-data comparisons and component ablations support their respective contributions. MoZoo’s contribution therefore extends beyond an individual positional encoding design: it establishes a task formulation and a working data-to-model pipeline for combining frame-aligned structural guidance with independent appearance control.

Q1

We added a comparison with role embeddings. As shown in the table below, the two approaches perform similarly with image references, while RAR-RoPE performs better with video references. Our contribution is to provide a practical framework for mesh-guided animal appearance synthesis, with conditioning modules that can be replaced by or combined with role embeddings or more effective methods in the future. In our evaluation, the full framework substantially outperforms the channel-concatenation and VACE baselines, while RAR-RoPE better preserves appearance details than role embeddings in complex-texture transfer cases.

Table R12. Role embeddings on the expanded real-world test set.

MethodVideo QualityPaired-Target Fidelity
Subject Consist. ↑Background Consist. ↑Motion Smooth. ↑Imaging Qual. ↑Aesthetic Qual. ↑PSNR ↑LPIPS ↓SSIM ↑
Role Embeddings (Image)97.7698.6499.3269.2756.3122.110.0780.912
Role Embeddings (Video)97.6399.1299.3069.3556.5822.020.0800.910
MoZoo (Image)97.8398.8099.3769.4556.6522.160.0750.920
MoZoo (Video)97.9898.9299.4569.5156.9122.390.0700.925

Table R13. Role embeddings on the rendered mesh test set.

MethodVideo QualityPaired-Target Fidelity
Subject Consist. ↑Background Consist. ↑Motion Smooth. ↑Imaging Qual. ↑Aesthetic Qual. ↑PSNR ↑LPIPS ↓SSIM ↑
Role Embeddings (Image)97.5798.6599.2069.2260.7121.280.0770.906
Role Embeddings (Video)97.5998.8599.2669.3460.8321.170.0790.908
MoZoo (Image)97.6698.8299.2469.3660.9821.340.0740.913
MoZoo (Video)97.8698.9499.3369.4061.0221.540.0690.920

We sincerely thank you for your valuable suggestions, which have helped us refine and strengthen the paper. If you have any additional questions or concerns, please do not hesitate to let us know.

Thank you for your detailed rebuttal and additional experiments. However, i concern with your results.

  1. The motion following in the video demo_comparison_reference.mp4 (The tiger) is bad could you provide an explanation.
  2. I still holds that the demo is similar to texture transfer. I admit that the data construction pipeline is valuable in this specific task and domain, but it is not essential in current video generation area. For example, LTX exhibit more photo-realistic results even they do not specifically train on this domain. In contrast, your results are sometimes oversaturated. I think scaling LTX and its data construction pipeline is a more promising future.
Reviewer_BUyiOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

MoZoo addresses mesh-guided animal video generation: given a rendered mesh animation and an appearance reference in text, image, or video form, it generates a realistic animal video intended to preserve the input motion and geometry. Built on Wan2.1, the method introduces RAR-RoPE to distinguish temporally aligned mesh conditions from independent appearance references, and ADA to separate conditioning streams while allowing target frames to access the relevant motion and appearance information.

The paper also constructs MoZooData, comprising approximately 62K synthetic and real-video-derived training pairs. Validation includes comparisons with existing video-editing methods and adapted baselines on the 120-case MoZooBench, quantitative and qualitative evaluations, a user study, and component ablations. The reported results show improvements in appearance fidelity and video consistency, with ablations supporting the proposed positional encoding and attention design.

Critical strengths
  1. A useful data-construction pipeline for controllable animal video generation. The paper combines aligned UE5 renders with real-video pseudo-pairs produced by an inverse video-editing model, yielding approximately 62K training pairs. This addresses an important bottleneck: real animal videos offer diverse appearances and motions but generally lack corresponding mesh animations. The pipeline provides a practical way to incorporate these videos into training without requiring explicit 3D reconstruction of each animal.

  2. The conditioning design is supported by meaningful experimental evidence. The method explicitly distinguishes synchronized mesh guidance from temporally independent appearance references. RAR-RoPE and ADA implement this distinction through positional encoding and attention connectivity. Under the video-reference setting, removing these components reduces PSNR from 20.75 to 18.44 and 18.67, respectively. Comparisons with same-data-trained baselines, qualitative results, and the user study provide additional support. These results make the proposed design more convincing than a straightforward application of an existing video generator.

Critical weaknesses
  1. The methodological novelty beyond existing conditioning techniques is insufficiently established. RAR-RoPE extends the shared-versus-shifted positional indexing used in OminiControl to temporal alignment, while condition isolation and blocking attention from conditions to noisy targets have precedents in EasyControl. Likewise, Refaçade already trains a texture-removal model on textured/untextured renders to produce appearance-free video representations. MoZoo’s combination and video-specific adaptations are useful, but the paper does not sufficiently establish the technical advance beyond these existing ideas. This limits the strength of the claimed methodological contribution.

  2. Limited diversity and representativeness of the evaluation benchmark. MoZooBench contains only 120 cases retained from 400 candidates through the same inverse-generation and filtering pipeline used for the real training pairs, and 87.5% of the cases are quadrupedal mammals. Although training and evaluation source videos are disjoint, this filtering may favor cases that the inverse generator handles well, leaving challenging motions, occlusions, and less common animal shapes underrepresented. Consequently, the reported results establish performance on a narrowly curated distribution but provide limited evidence of generalization to independently authored mesh animations. This weakens the support for the paper’s broader claims of generalization across animal geometries and motion patterns.

Questions for authors
  1. Only 120 of the 400 candidate cases are retained in MoZooBench, with quadrupedal mammals accounting for 87.5% of the final benchmark. Why is the retention rate so low, and what proportion of the excluded cases fails each filtering criterion? Does this primarily reflect limitations of the inverse-generation pipeline, and does the filtering disproportionately remove challenging motions, occlusions, or animal shapes? Given the resulting small and heavily filtered test set, how do the authors establish that it provides a sufficiently representative and reliable comparison, rather than favoring cases that their data-generation pipeline handles well?

  2. The external comparisons are limited to VACE and Refaçade. UniVideo also supports reference-image-guided video editing, while methods such as AnyV2V and Zero-to-Hero offer an alternative based on editing an anchor frame and propagating its appearance through the video. Could the authors include a representative comparison from these approaches, or explain why their input requirements make them unsuitable? Such comparisons would help establish whether MoZoo offers advantages over broader reference-guided editing and appearance-propagation approaches.

  3. Which part of ADA is essential? ADA combines several restrictions: blocking feedback from noisy target tokens, isolating the conditioning streams from one another, and restricting direct mesh access to the corresponding frame. Although the paper includes an overall ADA ablation and a frame-alignment study, do the existing results distinguish the benefit of blocking target-to-condition feedback from that of isolating the condition streams? In particular, why is preventing mesh–reference interaction necessary, rather than only preventing noisy-target feedback?

  4. The intended application takes rendered mesh animations as input, whereas the real-data pipeline creates gray-model proxies through video editing. Could the authors clarify how adding these pseudo-pairs improves performance on actual rendered mesh inputs? In particular, does the benefit of real-video training data demonstrated in Appendix F also hold for independently created mesh animations, rather than primarily for inverse-generated proxies?

4 comments
AuthorsOpenReview ↗

Response to Reviewer BUyi(part1/2)

Thank you for your positive evaluation of our work and for the valuable suggestions provided. Below, we address your concerns point-by-point.

W1 — Methodological novelty

Thank you for pointing out MoZoo’s connections to prior work and recognizing its practical value for controllable animal video generation. Our contribution is a complete data-to-model pipeline for generating realistic animal appearance from mesh animations and independent appearance references.

1. Data construction. Real animal videos offer diverse appearances but generally lack corresponding mesh animations. We train an inversion model on paired synthetic renders and use it to create coarse mesh proxies for real clips. Combining these pseudo-pairs with aligned synthetic renders provides a training-ready dataset for mesh-guided animal appearance synthesis. In the examples shown, our pipeline leaves less residual texture and preserves the animals’ 3D shape more clearly than Refaçade. The video comparisons are available here.

2. Tailored conditioning design. MoZoo jointly handles frame-aligned mesh guidance and independent appearance references. RAR-RoPE encodes their different temporal relationships, while ADA controls access to structural and appearance information and blocks feedback from noisy target tokens into the conditioning streams. Same-data comparisons and component ablations support the effectiveness of these designs relative to the evaluated alternatives.

MoZoo’s contribution lies in formulating the task, introducing a new approach to generation, and developing the data pipeline and conditioning framework needed to support it. Ablation studies support the contributions of the data pipeline and conditioning framework. We therefore believe that MoZoo makes a substantive contribution that is not limited to a single positional encoding or attention mechanism.

W2 & Q1 — Benchmark representativeness

We thank the reviewer for raising this important point, and we will address it in three parts.

1.Filtering criteria. We check whether the reconstructed mesh follows the motion in the corresponding RGB video, remains stable across frames, and is free of residual fur or mesh collapse. We first use Qwen3-VL to flag clear failures, then manually review the remaining clips.

2.Reasons for the low retention rate. Of the 400 candidate clips, 230 featured non-mammalian animals, while the Unreal Engine (UE) renders used to train our mesh inversion model primarily depicted mammals. The model generalizes to some non-mammalian animals, but limited coverage during training made it harder to consistently produce stable mesh videos that followed the motion in the source clips. This reduced the number of clips suitable for paired evaluation. We have expanded the training set for the inversion model by incorporating both rendered and real-world samples, and have observed an improvement in the quality of the mesh reconstruction.

3.Fairness of evaluation. We applied the same filtering criteria to all candidates and evaluated all methods on the same retained clips. To address the reviewer’s concern about coverage, we added 60 clips featuring non-mammalian animals and 40 clips with rapid motion, then recomputed all metrics on the expanded test set. We also added a separate test set of 50 rendered clips.

Table R6. Benchmark comparison on the expanded real-world test set.

MethodVideo QualityPaired-Target Fidelity
Subject Consist. ↑Background Consist. ↑Motion Smooth. ↑Imaging Qual. ↑Aesthetic Qual. ↑PSNR ↑LPIPS ↓SSIM ↑
VACE92.8395.3098.7764.3545.6517.420.1010.897
VACE+Data95.0897.7199.1269.3254.0420.080.0830.907
Refaçade94.9294.8199.1867.2152.3119.820.0880.903
Channel concat. (Image)95.3197.1799.0465.6853.2220.120.0920.893
Channel concat. (Video)95.6997.5299.1565.9753.6520.830.0850.901
MoZoo (Image)97.8398.8099.3769.4556.6522.160.0750.920
MoZoo (Video)97.9898.9299.4569.5156.9122.390.0700.925
AuthorsOpenReview ↗

Response to Reviewer BUyi(part2/2)

Table R7. Benchmark comparison on the rendered mesh test set.

MethodVideo QualityPaired-Target Fidelity
Subject Consist. ↑Background Consist. ↑Motion Smooth. ↑Imaging Qual. ↑Aesthetic Qual. ↑PSNR ↑LPIPS ↓SSIM ↑
VACE93.8396.3099.1769.3558.9116.760.0960.903
VACE+Data95.9297.6299.1969.3959.0219.320.0780.908
Refaçade95.6494.6299.2267.8656.9719.170.0830.902
Channel concat. (Image)95.6797.3599.1265.3757.1319.320.0890.897
Channel concat. (Video)95.8297.4499.1765.7257.3720.070.0830.894
MoZoo (Image)97.6698.8299.2469.3660.9821.340.0740.913
MoZoo (Video)97.8698.9499.3369.4061.0221.540.0690.920

We provide video results for non-mammalian animals and clips with rapid motion.

Q2 — External comparisons

We thank the reviewer for suggesting these comparisons. We evaluated MoZoo, UniVideo, AnyV2V, and Zero2Hero on the expanded test set.

Table R8. External baselines on the expanded real-world test set.

MethodVideo QualityPaired-Target Fidelity
Subject Consist. ↑Background Consist. ↑Motion Smooth. ↑Imaging Qual. ↑Aesthetic Qual. ↑PSNR ↑LPIPS ↓SSIM ↑
Univideo96.4596.3297.3366.4354.4619.340.0800.894
Anyv2v94.3795.6793.4765.3153.3918.570.0850.832
Zero2Hero92.7894.3592.3765.5352.3417.160.0960.784
MoZoo (Image)97.8398.8099.3769.4556.6522.160.0750.920
MoZoo (Video)97.9898.9299.4569.5156.9122.390.0700.925

Table R9. External baselines on the rendered mesh test set.

MethodVideo QualityPaired-Target Fidelity
Subject Consist. ↑Background Consist. ↑Motion Smooth. ↑Imaging Qual. ↑Aesthetic Qual. ↑PSNR ↑LPIPS ↓SSIM ↑
Univideo96.3196.4797.1466.5158.6418.620.0810.889
Anyv2v94.1295.5693.6165.1857.8317.680.0860.829
Zero2Hero92.7494.4892.0565.2756.5116.470.0930.773
MoZoo (Image)97.6698.8299.2469.3660.9821.340.0740.913
MoZoo (Video)97.8698.9499.3369.4061.0221.540.0690.920

The video comparisons are available for reference guidance and text guidance. UniVideo transfers some appearance features and tends to retain the reference pose when it differs substantially from the pose of the input mesh. AnyV2V captures the reference appearance in the edited first frame, though some features fade in later frames. Zero2Hero produces blurred fur details in some cases.

Q3:

We thank the reviewer for raising this question. To preserve generation quality, ADA blocks feedback from noisy target tokens to the conditioning streams, keeping the mesh and appearance reference representations independent of the current denoising state.

To reduce unnecessary attention interactions, ADA masks direct attention between the conditioning streams and lets target tokens read only the corresponding mesh frame. This keeps the reference pose out of the mesh representation while allowing target tokens to combine frame-aligned mesh cues with reference appearance.

Q4:

We understand the reviewer’s concern. To assess the benefit of real-video training data, we evaluated the synthetic-only and mixed-data models on the same set of independently created Blender mesh animations, using identical text prompts and reference images for both models.

  1. Text guidance. The mixed-data model generates finer, fluffier fur than the synthetic-only model. The comparison is available here.

  2. Image guidance. With photographs of real animals as references, the mixed-data model produces more realistic results than the synthetic-only model. here.

AuthorsOpenReview ↗

Response to Reviewer BUyi

We sincerely thank you for your valuable suggestions, which have helped us refine and strengthen the paper. If you have any additional questions or concerns, please do not hesitate to let us know.

AuthorsOpenReview ↗

Response to Reviewer BUy

Dear Reviewers,We sincerely appreciate your professional comments on our manuscript. We have carefully revised our paper and responded to your comments point by point. Your valuable feedback has significantly improved the quality of our manuscript. We welcome further feedback if you have any remaining concerns and are ready to do our best to address them.

Reviewer_No9kOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces mesh-guided animal appearance synthesis, where a coarse animal mesh video specifies geometry and motion while text, an image, or an asynchronous reference video specifies appearance. The authors construct MoZoo-Data from approximately 30K synthetic paired clips and 32K real-video pseudo-pairs, together with the 120-case MoZooBench. The model uses role-aware positional encoding to distinguish synchronized structural conditions from asynchronous appearance references, and asymmetric decoupled attention to control information flow among the streams.

Critical strengths

1、The dataset construction addresses a concrete bottleneck in controllable animal video generation. Synthetic rendering provides aligned structural supervision, while real-video pseudo-pairs expand appearance diversity. Fine-tuning VACE on the proposed data increases PSNR from 15.63 to 18.72 and reduces LPIPS from 0.098 to 0.075, demonstrating the utility of the dataset. The appendix also checks training–evaluation separation using original source-video identifiers.

2、The conditioning design matches the relationships among the inputs and is supported by controlled comparisons. Mesh and target tokens share temporal coordinates, while asynchronous references occupy a separate temporal range. Target queries access frame-aligned mesh features and global appearance evidence. The experiments include matched reference images, same-data channel-concatenation baselines, and component removals. The user study directly evaluates reference appearance similarity and agreement with mesh motion.The image-reference comparison provides a concrete measure of the architectural gain. With the same training data and reference image, MoZoo improves PSNR from 17.89 to 19.61 and reduces LPIPS from 0.090 to 0.075 relative to channel concatenation. Together with the VACE fine-tuning results, this comparison helps distinguish the benefits of the dataset from those of the conditioning design.

Critical weaknesses

1、The reported automatic metrics do not separately establish reference-appearance fidelity and adherence to the input mesh. Within-video subject consistency does not establish whether the output preserves the reference appearance, and motion smoothness does not establish whether it follows the input mesh. Full-frame PSNR, SSIM, and LPIPS combine foreground and background errors, obscuring the contributions of motion control and appearance transfer. For example, under image conditioning, MoZoo improves PSNR over VACE+Data from 18.72 to 19.61, while the reported LPIPS and SSIM are unchanged at 0.075 and 0.915. A foreground/background breakdown would help identify where the additional PSNR gain arises and whether it corresponds to better animal appearance or motion alignment.Table 2 states that paired-target metrics are computed where synchronized targets are available but does not specify the number of evaluated cases. The user study directly evaluates both objectives, with 25 volunteers assessing 20 cases. The paper does not specify whether MoZoo uses an image or a video reference in this study, or whether output positions were randomized. The reference modality matters because a video can provide appearance information from several views. Per-case preference rates would also show how the advantage varies across the 20 examples.

2、Benchmark filtering and morphology imbalance limit the generalization claims. MoZooBench retains 120 of 400 candidate cases after removing unstable meshes and some failures involving large or fast motions. Of the retained cases, 105 are quadrupedal mammals, accounting for 87.5% of the benchmark. Appendix C already identifies these limitations. The results mainly establish performance on curated mesh inputs and quadrupedal mammals. Species-level generalization also depends on which test species appear in training; the source-video overlap check addresses separation at the video level and does not establish a species-held-out evaluation. The appendix includes qualitative synthetic-only results, but does not report a matched-budget quantitative comparison with mixed-data training. The examples in Appendix F suggest that synthetic training transfers more readily to short, sleek fur than to fine or voluminous fur. Quantifying this difference would clarify which appearance characteristics benefit most from real-video pseudo-pairs and how much they contribute to the final results.

Questions for authors

1、How many test cases are included in the paired-target metrics in Table 2? Does the method ranking remain the same when evaluation is restricted to animal foregrounds and includes silhouette or keypoint motion errors and reference-to-output appearance similarity?

2、Does MoZoo use an image or a video reference in the user study? Do all methods receive equivalent appearance information, and is output presentation randomized? Please provide case-level preference statistics or uncertainty intervals.

3、How does performance vary with whether the species appears in training, motion difficulty, and mesh quality? Under the same training budget, what are the quantitative results for synthetic-only, real-pseudo-pair-only, and mixed-data training?

4 comments
AuthorsOpenReview ↗

Response to Reviewer No9k (part1/3)

Thank you for your positive evaluation of our work and for the valuable suggestions provided. Below, we address your concerns point-by-point.

W1 & Q1 — Evaluation metrics

We thank the reviewer for their feedback on our evaluation. We clarify how we compute the metrics in Table 2 and report additional metrics to assess mesh adherence and appearance fidelity separately.

1. Evaluation details. Table 2 reports paired-target metrics for all 120 test cases in MoZooBench. Each RGB target video depicts the reference appearance and is synchronized with the input mesh motion. Because PSNR, SSIM, and LPIPS compare corresponding frames, these scores reflect both appearance differences and motion misalignment.

2. Additional metrics. Following the reviewer’s suggestion, we recomputed PSNR, SSIM, and LPIPS on the animal foreground, as shown in Tables R2 and R3. We measure mesh adherence using framewise silhouette IoU between the generated animal and the input mesh, and appearance fidelity using DINO similarity between the reference and generated animal foregrounds. MoZoo ranks first or ties for first on all three foreground metrics on both test sets.

W1 & Q2 — User study

We understand the reviewer’s concern. In the user study, all methods received the same reference image for each case, and we randomized the order in which the candidate videos were displayed. The table below reports the overall preference rates.

Table R1. User-study preferences.

MoZoo preferenceC1C2C3C4C5C6C7C8C9C10C11C12C13C14C15C16C17C18C19C20All cases
Appearance1822211923201724202223192022182124172120411/500 (82.2%)
Motion1419161320171518161919141720131618151715331/500 (66.2%)
Overall2023221924212024212323202125202324202321437/500 (87.4%)

Note: C1–C20 report the observed preference counts out of 25 responses per case. The “All cases” column reports the aggregate preference counts and percentages across all 20 cases.

W2 — Benchmark filtering

We thank the reviewer for raising this important point, and we will address it in three parts.

  1. Filtering criteria. We check whether the reconstructed mesh follows the motion in the corresponding RGB video, remains stable across frames, and is free of residual fur or mesh collapse. We first use Qwen3-VL to flag clear failures, then manually review the remaining clips.

  2. Reasons for the low retention rate. Of the 400 candidate clips, 230 featured non-mammalian animals, while the Unreal Engine (UE) renders used to train our mesh inversion model primarily depicted mammals. The model generalizes to some non-mammalian animals, but limited coverage during training made it harder to consistently produce stable mesh videos that followed the motion in the source clips. This reduced the number of clips suitable for paired evaluation. We have expanded the training set for the inversion model by incorporating both rendered and real-world samples, and have observed an improvement in the quality of the mesh reconstruction.

  3. Fairness of evaluation. We applied the same filtering criteria to all candidates and evaluated all methods on the same retained clips. To address the reviewer’s concern about coverage, we added 60 clips featuring non-mammalian animals and 40 clips with rapid motion, then recomputed all metrics on the expanded test set. We also added a separate test set of 50 rendered clips.

AuthorsOpenReview ↗

Response to Reviewer No9k (part2/3)

W2 & Q3 — Generalization and training data

We understand the reviewer’s concern about generalization. We expanded the test set, evaluated MoZoo under different mesh input conditions, and compared three training data settings.

  1. Expanded test set. We added 60 clips featuring non-mammalian animals and 40 clips with rapid motion, then recomputed all metrics on the expanded test set. We also added a separate test set of 50 rendered clips and provide videos showing the results on the additional clips.

Table R2. Expanded real-world test set.

MethodSubject Consist. ↑Background Consist. ↑Motion Smooth. ↑Imaging Qual. ↑Aesthetic Qual. ↑PSNR ↑LPIPS ↓SSIM ↑FG-PSNR ↑FG-LPIPS ↓FG-SSIM ↑Silhouette IoU ↑Ref–Output DINO ↑
VACE92.8395.3098.7764.3545.6517.420.1010.89716.310.1050.8870.8730.735
VACE+Data95.0897.7199.1269.3254.0420.080.0830.90719.580.0850.9060.8970.816
Refaçade94.9294.8199.1867.2152.3119.820.0880.90319.240.0890.9010.8910.801
Channel concat. (Image)95.3197.1799.0465.6853.2220.120.0920.89319.490.0940.8710.8900.840
Channel concat. (Video)95.6997.5299.1565.9753.6520.830.0850.90120.100.0880.8900.9000.856
MoZoo (Image)97.8398.8099.3769.4556.6522.160.0750.92021.500.0770.9150.8970.906
MoZoo (Video)97.9898.9299.4569.5156.9122.390.0700.92521.610.0740.9200.9100.913

Note: The table compares MoZoo with baseline methods on the expanded real-world test set, with image and video guidance.

Table R3. Rendered mesh test set.

MethodSubject Consist. ↑Background Consist. ↑Motion Smooth. ↑Imaging Qual. ↑Aesthetic Qual. ↑PSNR ↑LPIPS ↓SSIM ↑FG-PSNR ↑FG-LPIPS ↓FG-SSIM ↑Silhouette IoU ↑Ref–Output DINO ↑
VACE93.8396.3099.1769.3558.9116.760.0960.90315.640.0940.8980.8620.690
VACE+Data95.9297.6299.1969.3959.0219.320.0780.90818.470.0800.9070.8830.802
Refaçade95.6494.6299.2267.8656.9719.170.0830.90218.340.0830.9030.8740.793
Channel concat. (Image)95.6797.3599.1265.3757.1319.320.0890.89718.430.0900.8830.8740.829
Channel concat. (Video)95.8297.4499.1765.7257.3720.070.0830.89419.130.0850.8900.8830.842
MoZoo (Image)97.6698.8299.2469.3660.9821.340.0740.91320.390.0750.9070.9020.897
MoZoo (Video)97.8698.9499.3369.4061.0221.540.0690.92020.440.0710.9110.9060.903

Note: The same metrics are reported on the 50 rendered mesh clips.

  1. Generalization across mesh inputs. We evaluated MoZoo on mesh videos that vary in motion complexity and mesh quality. We provide video results for non-mammalian animals and clips with rapid motion.
AuthorsOpenReview ↗

Response to Reviewer No9k (part3/3)

  1. Training data comparison. Under the same training budget, we trained three models using synthetic data only, real-video pseudo-pairs only, or a mixture of the two. We evaluated all three models on the same test set and report the results below.

Table R4. Training data ablation on the expanded real-world test set.

MethodSubject Consist. ↑Background Consist. ↑Motion Smooth. ↑Imaging Qual. ↑Aesthetic Qual. ↑PSNR ↑LPIPS ↓SSIM ↑FG-PSNR ↑FG-LPIPS ↓FG-SSIM ↑Silhouette IoU ↑Ref–Output DINO ↑
Synthetic only (Image)96.4897.6499.2769.1855.8618.290.0810.91517.650.0780.9060.8840.876
Synthetic only (Video)96.6797.7899.3469.2556.1318.600.0750.92217.830.0740.9100.8960.887
Real-video pseudo-pairs only (Image)96.8198.4099.3169.3255.9221.280.0740.92320.940.0750.9130.8700.894
Real-video pseudo-pairs only (Video)96.9498.5999.4069.4156.1621.500.0700.92921.070.0730.9190.8860.902
MoZoo (Image)97.8398.8099.3769.4556.6522.160.0750.92021.500.0770.9150.8970.906
MoZoo (Video)97.9898.9299.4569.5156.9122.390.0700.92521.610.0740.9200.9100.913

Note: The table compares synthetic-only, real-video pseudo-pair-only, and mixed training under the same training budget on the expanded real-world test set, with image and video guidance.

Table R5. Training data ablation on the rendered mesh test set.

MethodSubject Consist. ↑Background Consist. ↑Motion Smooth. ↑Imaging Qual. ↑Aesthetic Qual. ↑PSNR ↑LPIPS ↓SSIM ↑FG-PSNR ↑FG-LPIPS ↓FG-SSIM ↑Silhouette IoU ↑Ref–Output DINO ↑
Synthetic only (Image)96.3797.6399.1869.1460.1717.530.0800.90816.610.0760.9000.8870.871
Synthetic only (Video)96.5897.7299.2369.1160.3017.680.0760.91116.900.0730.9040.8960.875
Real-video pseudo-pairs only (Image)96.6898.3999.2169.2360.3420.460.0760.91119.780.0750.9060.8760.883
Real-video pseudo-pairs only (Video)96.7698.6199.1969.3160.3220.830.0720.91719.940.0730.9080.8810.892
MoZoo (Image)97.6698.8299.2469.3660.9821.340.0740.91320.390.0750.9070.9020.897
MoZoo (Video)97.8698.9499.3369.4061.0221.540.0690.92020.440.0710.9110.9060.903

Note: The table compares the same three training settings under the same training budget on the 50-clip rendered mesh test set, with image and video guidance.

key findings: Mixed training achieves the best overall balance between mesh adherence and appearance fidelity.


We sincerely thank you for your valuable suggestions, which have helped us refine and strengthen the paper. If you have any additional questions or concerns, please do not hesitate to let us know.

AuthorsOpenReview ↗

Response to Reviewer No9k

Dear reviewers, we sincerely appreciate for your professional comments on our manuscript. We have carefully revised our paper and responded to your comments one-by-one. Your valuable feedback has significantly improved the quality of our manuscript. We kindly invite you to provide further feedback if you have any concerns, and we are ready to do our best to address any issues you may have.

Comments on the paper

2 comments
AuthorsOpenReview ↗

Global Response

Dear Area Chair and Program Chairs, Thank you for overseeing the review of our submission. We sincerely appreciate the time and effort of all reviewers and the AC. MoZoo contributes a complete workflow for mesh-guided animal appearance synthesis, from data construction to model design:

  1. A practical data-construction pipeline. MoZoo-Data alleviates paired-data scarcity and offers a new way to pair real animal videos with coarse mesh videos. This enables paired training on real footage without explicit 3D reconstruction of each animal.

  2. A practical framework for controllable animal rendering. MoZoo allows artists to retain mesh-defined geometry and motion while independently specifying appearance through text, images, or videos, providing an alternative to labor-intensive animal appearance production.


We are pleased that the reviewers recognize the following strengths:

  1. “The dataset construction addresses a concrete bottleneck…” [No9k] “…without requiring explicit 3D reconstruction of each animal.” [BUyi] “MoZoo-Data and especially the inverse-mesh pipeline…are valuable contributions.” [AFjq]

  2. “The conditioning design matches the relationships among the inputs and is supported by controlled comparisons.” [No9k] “…more convincing than a straightforward application of an existing video generator.” [BUyi]

  3. “The model shows impressive cross-species texture transfer and multi-entity handling.” [AFjq]

We believe this workflow can contribute useful training resources and tested design choices to the community, supporting further research on rendering existing animations and extending structural and appearance control to larger video models.


We have addressed the reviewers’ questions point by point. Bellow, we summarize the additional experiments and clarifications addressing their main concerns. The corresponding revisions are marked in red in the manuscript.

✅ 1. Reviewers No9k and BUyi: Evaluation Coverage and Metric Concerns Addressed

The reviewers questioned benchmark coverage and whether full-frame metrics obscure foreground performance.

  • (1) Broader evaluation: As shown in Appendix Tables S1–S2, MoZoo (Video) ranks first on 12 of 13 metrics on the expanded real-world set and all 13 on the rendered mesh set, extending the evidence beyond the original filtered benchmark.
  • (2) Foreground assessment: The foreground metrics, silhouette IoU, and DINO similarity in Appendix Tables S1–S2 show that the gains include animal appearance fidelity and mesh adherence, rather than background reconstruction alone.

✅ 2. Reviewers BUyi and AFjq: Baseline Comparisons Substantially Supplemented

The reviewers requested broader baselines and comparisons with simpler conditioning alternatives.

  • (1) Additional methods: As shown in Appendix Tables S1–S2, comparisons with UniVideo, AnyV2V, Zero2Hero, and LTX support MoZoo’s advantages in jointly preserving reference appearance and mesh-defined structure. We also provide video comparisons.
  • (2) Conditioning alternatives: Role embeddings achieve comparable aggregate scores in Appendix Tables S1–S2, while the complex-texture comparisons show better detail preservation with RAR-RoPE. Table 3 further examines temporal indexing and attention visibility, supporting the selected conditioning design.

✅ 3. Reviewers No9k and BUyi: Training Data Contribution Questions Answered

The reviewers asked whether real-video pseudo-pairs improve performance under a matched training budget and whether their benefits extend to rendered animations.

  • (1) Data complementarity: As shown in Appendix Tables S3–S4, synthetic-only training provides stronger mesh adherence than real-video pseudo-pair-only training, while the latter improves appearance matching. Mixed training achieves higher IoU, DINO similarity, PSNR, and foreground PSNR than either source alone on both test sets.
  • (2) Independent animations: Comparisons on independently authored Blender animations show improved fur detail under text guidance and more realistic appearance under image guidance, illustrating benefits beyond inverse-generated proxies.
AuthorsOpenReview ↗

Global Response

✅ 4. Reviewers No9k, BUyi, and AFjq: Generalization Evidence Substantially Supplemented and Evaluation Fairness Clarified

The reviewers questioned generalization beyond quadrupedal mammals and requested clearer evaluation details.

  • (1) Generalization: The non-mammalian examples, rapid-motion examples, and cross-body-shape and AI-generated-reference examples demonstrate applicability across a broader range of animal shapes, motions, and appearances.
  • (2) Evaluation protocol: We clarify that Table 2 includes all 120 original cases. Section 5.5 and Table 4 describe the user study, which uses identical reference images across methods and randomized output order, addressing questions about appearance information and presentation fairness.

✅ 5. Reviewers AFjq: Scope of the Revisions

The revisions add baseline comparisons, ablation studies, and generalization evaluations in response to the reviewers’ suggestions, without changing the proposed framework.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.