- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper presents TaoHeadVerse, a feed-forward framework that reconstructs an animatable full-head avatar from one or a few portraits while jointly estimating per-frame motion. The predicted asset is explicit: a neutral mesh, mesh-bound Gaussians, identity-adapted mesh blendshapes (MBA) and Gaussian appearance bases (GAB), so animation requires only basis mixing, skinning and rasterization. Two regularizers target what the authors call representation confounding: Neutral Identity Anchoring (NIA) applies an ArcFace identity loss to zero-expression renderings, and Adversarial Domain Alignment (ADA) discourages canonical vertex features from predicting dataset-level acquisition attributes. Experiments on VFHQ-Test, AVA256 and RenderMe-360 cover self/cross-reenactment, sparse-view reenactment and single-image full-head completion, with a public-data-only variant, a six-pair matched ablation, and desktop/mobile throughput measurements.
Critical strengths
-
Extensive, protocol-matched and unusually transparent evaluation. Baselines are re-run under identical inputs, author-reported numbers are separated and excluded from ranking, raw and reference-relative AED are both reported, the registration step on RenderMe-360 is disclosed, and a public-data-only variant is trained with the same recipe. The public variant already improves VFHQ self-reenactment PSNR by 2.30 dB over FlexAvatar and raises cross-identity CSIM_Arc from 0.596 to 0.774 (Table 1), and it beats all baselines on every RenderMe-360 aggregate metric (Table 3). This substantially weakens the usual "gains come from private data" objection.
-
The explicit asset is a practically important design and the efficiency claims are carefully scoped. Animation costs 0.04 GFLOPs versus 182.51 for FlexAvatar (Table 5a); four concurrent avatars reach 84 and 68 FPS at 1024×1024 on two phones. The paper scopes these numbers honestly as animation-and-rendering throughput with precomputed controls.
Critical weaknesses
-
The qualitative evidence is thin relative to the extensive quantitative evaluation, and its baseline coverage does not match the tables. In the main paper, Fig. 3 shows two cross-reenactment examples per dataset, and the appendix adds two or three per figure (Figs. 7–9), so it is hard to judge at a glance how the methods differ. FastAvatar is reproduced in Tables 1, 2 and 5 and plotted in Fig. 1 but never appears in a qualitative figure; GAGAvatar appears in Fig. 7 but only with author-reported numbers in Table 1; LAM, the closest explicit-rig one-shot system (public code; reported AKD 2.06 and self CSIM 0.822), appears in neither the qualitative figures nor Table 5. On AVA256, FastAvatar's CSIM (0.170) and AKD (27.87 px), and UIKA's AKD (28.49) with visibly mis-posed, differently framed heads in Figs. 3(b) and 8, suggest that these pose-free methods may not be consuming the calibrated target cameras, in which case FlexAvatar is the main informative feed-forward competitor there and a footnote would help readers. More examples per comparison, FastAvatar outputs, GAGAvatar under the shared protocol, and, if feasible, LAM would bring the qualitative story in line with the quantitative one.
-
End-to-end real-time, video-driven animation on device is not demonstrated. The mobile numbers (84 / 68 FPS for four avatars) are animation-and-rendering throughput with precomputed driving controls, measured with up to eight frames in flight, and the paper states that its own expression extraction is not yet real time on mobile (Sec. 4.7). The telepresence scenario that motivates the paper is therefore only partly shown. Per-frame latency (the component times in Table 5(b) exceed the totals), the cost of motion estimation and asset generation, asset size and peak memory would let readers judge how far the system is from an on-device pipeline; a demonstration with platform-native controls (e.g. ARKit) would also close much of the gap.
Questions for authors
-
Attribution of the gains. In Table 4(a), the variant without NIA (A1) already reaches 24.79 dB / 0.095 LPIPS on VFHQ versus 24.98 / 0.093 for the full model, and removing ADA (A4) changes VFHQ by at most 0.06 dB, so most of the 2.5 dB gain over FlexAvatar seems to come from the base system (joint estimation, A0→A3: 20.54→24.72 dB; an 878M-parameter DINOv2-L backbone trained on sixteen 96 GB GPUs; the training corpus). NIA's measurable effect is cross-identity CSIM_Arc 0.680→0.777 with a small expression cost (cross-identity AED 0.230→0.240), and ADA's effect is confined to RenderMe-360. What are FlexAvatar's parameter count, training data and compute relative to yours, and what does the base architecture achieve before NIA, ADA, GAB and MBA are added? Do the NIA and ADA effects also hold in the Public setting, given that all ablations use Full? Will the public-data weights and code be released?
-
RenderMe-360 protocol and the ADA mechanism. Full-head completion uses head masks from four additional calibrated views to fit a per-identity similarity transform, and the 80 registration views are part of the 1,180-view aggregate (Appendix A.6.2); the protocol is applied consistently to all methods, but it is no longer strictly single-image and the transform depends on the predicted geometry. Could you report Tables 3 and 4(b) with those views excluded, and A4 vs A5 under an identical registration transform? ADA improves frontal LPIPS by 13.5% (0.171→0.148, Table 4b), as much as rear-side (10.3%) and strict-rear (11.4%); how do you interpret this, and what is the accuracy of an independently trained probe for the three acquisition attributes on frozen canonical features with and without ADA?
-
Expression fidelity. The full model has higher raw cross-identity AED (0.233 vs 0.221) and APD (0.029 vs 0.025) than FlexAvatar; Ref-AED removes per-trajectory constant offsets but would also hide a uniformly damped expression amplitude, which identity anchoring and identity-scaled blendshapes (MBA) could in principle induce. What is the ratio of predicted to driving expression magnitude with and without NIA and with and without MBA, and does the NIA identity gain persist under a non-face-recognition identity measure or a small user study, given that the training loss and the CSIM evaluators are all face-recognition embeddings? Per-identity variance for AVA256 (8 identities) and RenderMe-360 (20 identities) would also help.
3 comments
Response to Q3
Q3
Identity preservation beyond face-recognition embeddings
We added an independent geometry-based evaluation of 50 VFHQ cross-identity sequences using RealDenseFace (Li et al., 2026), a state-of-the-art monocular 3D face reconstruction method. We track each generated sequence independently with a shared FLAME shape and compare the generated and source neutral meshes over 1,787 corresponding facial vertices in the FLAME model coordinate system, with pose and expression set to zero.
In the matched A1–A3 comparison, NIA reduces neutral-face RMSE by 12.00%, with improvements in 39 of 50 sequences. This supports better preservation of source facial geometry beyond CSIM. We also added eight comparisons of predicted neutral heads (Fig. 13). Without NIA, the predicted neutral heads exhibit noticeable changes in face, mouth, and eye shape, suggesting that identity-specific features leak into the expression representation.
Expression and pose fidelity
Full has higher raw cross-identity AED and APD than FlexAvatar under the evaluation used in Table 1. Ref-AED removes constant per-sequence offsets and measures relative expression changes, but it does not establish absolute expression accuracy or rule out reduced expression amplitude.
We additionally evaluate expression and head-pose accuracy using parameters tracked by RealDenseFace. Full achieves lower FLAME-AED than FlexAvatar (0.275656 vs. 0.292315), while its pose error remains higher (Table 10). The two expression evaluators give different rankings, so the results do not support a consistent expression-accuracy advantage over FlexAvatar. We also measure expression magnitude directly to examine the amplitude concern, as detailed below.
Expression magnitude with NIA and MBA
In the matched A1–A3 comparison, raw AED increases from 0.230 to 0.240 in Table 4(a), while FLAME-AED remains nearly unchanged under RealDenseFace (0.276953 vs. 0.277396; Table 10). The predicted-to-driving expression magnitude ratio decreases from 0.935 to 0.894, corresponding to a 4.36% reduction in mean expression magnitude. The expanded A1–A3 qualitative comparisons (Fig. 12) help interpret this reduction: A1 can produce exaggerated and unnatural mouth shapes, which NIA helps reduce in these examples.
Adding MBA in the matched A3–A4 comparison changes the magnitude ratio from 0.894 to 0.888, a further 0.69% reduction. Tables 4(a), 10, and 11 report these complementary measurements.
Response to Weakness
Weakness 1
Expanded qualitative evidence
We have expanded the qualitative comparisons to seven VFHQ examples and eight AVA256 examples (Figs. 7–8). The VFHQ comparison now includes GAGAvatar, FastAvatar, and LAM, and the AVA256 comparison includes FastAvatar. We also expanded the NIA ablation to four examples, added eight comparisons of predicted neutral heads, and included four full-head animation examples from photographic and stylized inputs, shown from front, side, and rear viewpoints.
The originally submitted supplementary video provides additional visual results, including comparisons with other methods and ablation results.
UIKA camera and alignment checks
We ran UIKA using its released weights, official preprocessing, and inference code with AVA256’s calibrated cameras and VHAP-tracked FLAME motion. On 336 valid target frames at 512 × 512 resolution, the tracked FLAME projections have a 41-point 2D landmark error of 4.98 pixels over the eyes, nose, and mouth, without additional 2D alignment. We also manually inspected a subset of frames and found the tracked FLAME projections reasonably aligned with the target images.
With the target camera and motion fixed, UIKA’s single-image prediction for the blue-haired subject in Fig. 3(b) has a 26.4% larger canonical head width and a 29.8% larger head height than its four-image prediction. This enlargement contributes to the misalignment in this example. In our qualitative inspection, similar enlargement occurred more often for subjects with long, loose hair, as illustrated in the additional examples.
FastAvatar camera setup
We used AVA256’s calibrated cameras and the FLAME parameters obtained through FastAvatar’s official code. Manual inspection showed reasonable alignment between the tracked FLAME projections and target images, with a 41-point 2D landmark error of 5.69 pixels, measured over the eyes, nose, and mouth across 393 valid frames at 512 × 512 resolution.
After carefully inspecting the released code, we found that it transforms Gaussian centers with head motion but leaves their orientations unchanged. We therefore switched to head-centric rendering, which slightly improved the visual results. Despite these efforts, we were unable to obtain satisfactory results on AVA256: self-reenactment AKD remains high at 21.06 and 21.93 pixels for single- and four-image inputs, respectively. We include these outputs in the qualitative comparisons (Fig. 8).
GAGAvatar and LAM comparisons
We reevaluated GAGAvatar under the shared VFHQ protocol using 2,500 frames per reenactment setting and updated Table 1. Full improves self-reenactment PSNR by 2.34 dB and cross-identity CSIM_Arc from 0.600 to 0.766 over GAGAvatar.
We also added LAM’s official implementation to the efficiency comparison in Table 5(a). On an RTX 4090 at 512 × 512 resolution, LAM takes 4.41 ms for animation and rendering and achieves 210.10 FPS with precomputed driving controls.
Weakness 2: End-to-end animation and system costs
Live on-device driving
We have added live camera-driven animation using MediaPipe expression and head-pose tracking on the same device, reported under “Live on-device driving” in the main text’s “Efficiency and Mobile Deployment” subsection. On iPhone 17, one avatar at 512×512 achieves 43.71 Hz tracking and a 59.99 Hz rendering loop over three 30-second runs, each following a 5-second warm-up. Latency from browser input sampling to the corresponding GPU completion notification averages 44.46 ms (p95: 61 ms), excluding sensor exposure and physical display latency. This demonstrates live on-device animation with MediaPipe controls; our own motion extractor remains non-real-time on mobile, and its acceleration remains future work.
Additional system costs
We added system-cost measurements on an RTX 4090 in Appendix A.7 (Table 9):
| Component | Time | Peak GPU memory |
|---|---|---|
| Single-image avatar synthesis | 0.332 s | 4.65 GiB |
| Motion preprocessing | 16.18 ms/frame | 4.78 GiB (combined) |
| Motion network | 48.14 ms/frame | |
| Animation and rendering | — | 0.319 GiB |
The runtime avatar asset occupies 167.71 MB before compression and 15.77 MB after compression. Table 9 also reports synthesis costs for 1–32 input images.
Mobile timing protocol
Table 5(b) measures animation and rendering throughput using precomputed driving controls. Component and total times are measured separately and are not additive; the reported frame times are not single-frame latency.
Response to Q1, Q2
Thank you for the detailed review and for recognizing the breadth of our evaluation and the practical value of the explicit avatar representation.
Q1
Model scale and training resources
We acknowledge that model scale and training resources may contribute to the overall performance gap. Our model has 878M trainable parameters, compared with FlexAvatar’s 269M total and 182M trainable parameters. Training takes approximately two weeks on 16 H20 GPUs, compared with three weeks on one A100 for FlexAvatar. Public shares several major training datasets with FlexAvatar, while Full adds synthetic and studio data. We have added the resource and dataset comparison in Appendix A.5 (Table 7).
The larger model generates an explicit avatar for efficient real-time animation and rendering on mobile devices. FlexAvatar’s animation requires a Transformer-based decoder, resulting in higher runtime costs. We also report inference costs in Appendix A.7 (Table 9): single-image avatar generation takes only 0.332 s on an RTX 4090.
Attribution of the gains and ablation design
Much of the reconstruction improvement comes from joint avatar–motion learning. In the matched A0–A3 comparison, PSNR increases from 20.54 to 24.72 dB and LPIPS decreases from 0.134 to 0.097. NIA addresses identity preservation during cross-identity animation, where small changes in facial features and motion can noticeably affect perceived identity. In the matched A1–A3 comparison, NIA raises cross-identity CSIM_Arc from 0.680 to 0.777. ADA improves full-head reconstruction, reducing rear-side and strict-rear LPIPS by 10.3% and 11.3%. Our ablations isolate these factors through matched comparisons; an all-components-removed baseline is not included.
NIA and ADA in the Public setting
Public achieves broadly comparable performance to Full on VFHQ and AVA256 and outperforms all baselines on the aggregate RenderMe-360 metrics. These results support the overall Public model, although matched Public ablations for NIA and ADA have not been completed.
Code and model release
The appendix provides details of our architecture, training, preprocessing, and evaluation protocols to support reproducibility. The release of the code and Public model weights remains under consideration.
Q2
Evaluation without registration views
We have removed the four registration cameras (24, 26, 28, and 55) from evaluation and updated Tables 3 and 4(b) using the remaining 1,100 views across 20 subjects. Full remains best across all five view groups and three metrics, and Public outperforms all baselines on the aggregate metrics. The ADA ablation conclusions also hold. The four head masks remain part of registration, as clarified in Appendix A.6.2.
Registration and frontal-view improvements
A4 and A5 use the same fitting procedure, with a transform estimated for each predicted geometry. Our comparison therefore evaluates reconstruction quality after registration.
The frontal set includes oblique views and cameras above and below the subject. In A4, we observe forward-leaning neck geometry and insufficient shoulder coverage, which cause silhouette and alignment mismatches in these views. Improvements in these regions can therefore benefit frontal LPIPS as well as rear-view reconstruction. We have added this analysis and corresponding qualitative comparisons in Appendix A.8.2 (Fig. 11).
ADA objective and acquisition-attribute predictability
We have not conducted an independent probe evaluation. ADA is designed to reduce reliance on acquisition-specific cues in mixed-source training. Our matched ablations show reconstruction improvements across viewing regions, supporting ADA’s benefit for full-head reconstruction.