- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper adapts WAN-Animate into a single-image avatar model with controllable expression, pose, and viewpoint. It introduces DISTRACT to diagnose expression-code entanglement, combines learned expression conditioning with a pose-aware proxy mesh, and trains using multi-camera triplets. Reenactment benchmarks and a static 3DGS reconstruction probe demonstrate strong visual quality, quantitative performance, and multi-view consistency.
Critical strengths
- The DISTRACT benchmark offers useful insights into viewpoint and identity entanglement in existing facial expression encoders.
- The method achieves strong visual quality and state-of-the-art quantitative results on the evaluated reenactment benchmarks.
Critical weaknesses
Table 1 evaluates a separate MLP probe, but its improvements do not directly establish the effectiveness of the disentanglement modules used in VDA. The deployed modules are subsequently fine-tuned with only the flow-matching loss, leaving it unclear whether the invariance properties learned during disentanglement pretraining are preserved. The small gap between No disent. and Full in Table 5 also leaves the added benefit of explicit disentanglement unclear.
Please report DISTRACT results (viewpoint and identity invariance) for , the re-entangled codes, and the outputs before and after VDA fine-tuning, with No disent. as a baseline where applicable. For , please evaluate whether samples from the same camera but different identities and expressions are closer than samples from different cameras. To clarify the role of pretraining, it would also help to compare the same architecture with (1) pretrained disentanglement modules kept frozen, (2) pretrained modules fine-tuned with VDA, as in the current method, and (3) modules trained jointly with VDA without separate pretraining.
Questions for authors
My questions are mainly about ablations in Table 5:
-
Could the authors provide 2 additional ablations, one replacing Ours (full) X-NeMo with WAN-Animate’s native expression encoder only, and one freezing the face encoder only? These ablations would help isolate the effects of encoder choice and face-encoder fine-tuning, as the existing comparisons change multiple components simultaneously.
-
How much do ray-direction coloring and separate head/torso coordinate frames contribute beyond the proxy mesh’s spatial cues? The no-proxy-mesh ablation does not isolate these effects. Two additional comparisons would help: (a) the same mesh rendered with Lambertian shading only, and (b) the same ray-direction coloring using a shared torso coordinate frame for both head and torso. These would isolate the benefits of directional coloring and part-specific coordinate frames, respectively.
3 comments
Part 2/2
Additional Disentangler Ablations
As suggested, we trained Ours with a frozen disentangler, as well as with a disentangler trained jointly from random initialization:
| Variant | PSNR ↑ | LPIPS ↓ | JOD ↑ | ID ↑ | Expr. ↑ | AKD ↓ | APD ↓ |
|---|---|---|---|---|---|---|---|
| Frozen Disentangler (pretrained) | 22.12 | 0.2090 | 4.628 | 0.6958 | 0.9121 | 0.0575 | 4.16 |
| Disentangler trained jointly (not pretrained) | 22.42 | 0.2095 | 4.626 | 0.7009 | 0.9141 | 0.0532 | 4.18 |
| Ours (pretrained, fine-tuned) | 22.43 | 0.2021 | 4.638 | 0.6989 | 0.9201 | 0.0549 | 3.97 |
- Frozen vs. fine-tuned. Fine-tuning the pretrained disentangler improves performance slightly.
- From random initialization. For this version, we added the disentangler losses alongside the flow-matching criterion. Its performance is very close to that of the pretrained disentangler.
Question 1: Why we are using X-NeMo codes?
Our method is not tied to X-NeMo. We chose it because it showed the strongest separable expression information on DISTRACT (96% vs. 92.5%). The table below crosses the expression code with whether WAN-Animate's face encoder is trained:
| Expression code | Face encoder | PSNR ↑ | LPIPS ↓ | JOD ↑ | ID ↑ | Expr. ↑ | AKD ↓ | APD ↓ |
|---|---|---|---|---|---|---|---|---|
| WAN-Animate | frozen | 21.52 | 0.2098 | 4.438 | 0.7012 | 0.9115 | 0.0532 | 4.06 |
| WAN-Animate | trained | 21.14 | 0.2015 | 4.639 | 0.7042 | 0.9231 | 0.0536 | 4.11 |
| X-NeMo | frozen | 22.11 | 0.2087 | 4.658 | 0.6977 | 0.9223 | 0.0549 | 3.96 |
| X-NeMo (ours) | trained | 22.43 | 0.2021 | 4.638 | 0.6989 | 0.9201 | 0.0549 | 3.97 |
While X-NeMo tends to give higher PSNR, most metrics are roughly equal. The method therefore works with both latent spaces.
Question 2: Proxy-mesh shading style
We did not originally ablate this design, since we do not consider it a central contribution, but we agree that the ablation strengthens the paper. We trained three variants that change only how the proxy mesh is shaded: (i) Plücker coloring in a single coordinate frame (torso centric), (ii) Lambertian gray shading, and (iii) camera-space surface normals. We train Ours with these alternative proxy mesh styles and are evaluated on BecomingLit self-reenactment:
| Proxy-mesh shading | PSNR ↑ | LPIPS ↓ | JOD ↑ | ID ↑ | Expr. ↑ | AKD ↓ |
|---|---|---|---|---|---|---|
| Part-local Plücker (ours) | 22.43 | 0.202 | 3.99 | 0.699 | 0.920 | 0.055 |
| Global Plücker (torso frame) | 21.70 | 0.204 | 3.94 | 0.704 | 0.914 | 0.055 |
| Lambertian | 21.36 | 0.204 | 3.96 | 0.708 | 0.923 | 0.056 |
| Surface normals | 21.32 | 0.204 | 3.95 | 0.699 | 0.923 | 0.053 |
The shading has only a small effect. We will add this ablation to the paper supplementary and reduce the emphasis on the part-local coloring accordingly.
response to authors
Thank you for the additional experiments and clarifications. The evaluations of the latent codes and the controlled ablations address my concerns and strengthen the empirical support for the method. I maintain my weak-accept recommendation.
Part 1/2
We thank the reviewer for the careful analysis and the precise questions. We address each point below.
Weaknesses
Tab. 1 and the DISTRACT benchmark merely serve as motivation, highlighting those expression spaces as promising candidates, if the view-bias is taken care of. We agree that these results are separate from the avatar use-case.
We now measure DISTRACT viewpoint invariance of the driving expression code at every stage of the pipeline, before and after joint fine-tuning. The stages are the raw X-NeMo code, the disentangled code, the code entering WAN-Animate's face encoder, and the face encoder's output tokens. We also report the camera-matching task described in Weakness 3 below, where a high score means camera information remains in the code. We perform the same measurement on the variant without disentangler, trained with camera triplets only:
For our model, the disentanglement is fully preserved.
We agree with the review, that there only is a small performance gap between Ours and "Ours without disentangler". The conclusion is that camera-triplet supervision alone makes the code inside the video model much more view-invariant (24% → 79%, as we reported in our supplementary (Figure 14)) and removes most camera information (43% → 6%). The cross-view training signal does most of the work by itself. Adding the disentangler still yields clearly stronger invariance at the code level (97% vs. 79%, with essentially no camera information left), and improved metrics, as shown in our orginal ablations.
Does zcam discriminate the view-point?
We designed an inverse version of DISTRACT to test exactly this. For each query image, the positive is an image from the same camera showing a different identity and expression. The four negatives come from other cameras, including the query's own identity and expression seen from a different camera. Rank-1 (%) is the share of queries for which the same-camera image is closest:
Raw X-NeMo codes are heavily influenced by the camer view, resulting in 58% Rank-1 performance . The disentangled expression code removes this almost completely (0.4%), while the camera code captures it (88%).
The additional ablations and our answers to the questions follow in Part 2/2.