- Rating
- 3(+1)
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper presents StrainAvatar, a person-specific head avatar that reconstructs expression-dependent mesh wrinkles from multi-view RGB. It uses analytically computed strain to predict residual Jacobian corrections, integrates them through a Poisson solve, and transports Gaussians using the corrected mesh deformation with surface alignment. Experiments on nine NeRSemble subjects evaluate novel views and expressions, reporting competitive rendering quality, improved normal agreement, and reduced mesh depth error.
Critical strengths
- Strain-driven wrinkle modeling. Using surface strain to drive a person-specific wrinkle response is a reasonable design. It gives the network a meaningful local deformation signal for learning how facial wrinkles change with expression.
- Image-based geometry supervision and convincing mesh results. White-texture shading and stopped lighting gradients direct the wrinkle-band loss toward mesh deformation, which then drives Gaussian rendering. The qualitative mesh comparisons show clearer and finer wrinkle structures, particularly around the forehead and between the eyebrows. The reduced stereo-depth error also supports improved surface accuracy at a coarser facial-shape scale.
Critical weaknesses
The contribution of the proposed wrinkle modeling is not fully isolated. Removing the corrected mesh eliminates all learned geometric refinement, which may include personalized shape correction as well as wrinkle detail. The targeted geometry-stage ablations in Fig. 7, without the strain map or wrinkle-band loss, would help clarify this attribution, but are reported only qualitatively. Table 4 instead tests strain conditioning in the Gaussian decoders, a different component. Section 4.3 also states that the Fig. 7 variants increase stereo-depth error and refers to Appendix H, yet the corresponding numbers are not reported there. Since the depth and pseudo-normal references do not resolve individual wrinkles, quantitative results for these targeted ablations are particularly important for assessing the central contribution.
Questions for authors
- Quantitative results for the geometry ablations: Could the authors provide quantitative geometry and final rendering results for the variants without the strain map or wrinkle-band loss, while retaining geometry optimization and matched subsequent Gaussian training? Section 4.3 refers to increased stereo-depth errors in Appendix H, but these measurements do not appear to be listed there.
- Choice of deformation representation: Could the authors elaborate on the benefits of predicting Jacobian corrections followed by Poisson integration, compared with directly predicting vertex offsets or a UV displacement map? If available, a comparison using the same strain inputs and geometry supervision would help clarify the contribution of this design choice.
5 comments
Thank you for the additional experiments. Since the final rendering PSNR gains from strain conditioning and the wrinkle-band loss appear marginal, my assessment relies primarily on their geometric benefits. However, the Q1 geometry-stage ablations cover only subject 104 under novel-view synthesis, leaving their consistency across identities unclear. Could you repeat these two ablations on one additional original subject using the same evaluation protocol and matched geometry-training settings? Geometry metrics alone would suffice; appearance-stage retraining is unnecessary. Consistent improvements would strengthen the case for raising my rating to weak accept.
Official Comment by Authors
Thank you for this suggestion. We repeated both geometry-stage ablations on all subjects using the same evaluation settings as in Q1. We report mesh depth MAE (mm) as mean ± std across subjects.
| Surface | Novel View MAE ↓ | Novel Expression MAE ↓ |
|---|---|---|
| Tracked mesh | 1.89 ± 0.43 | 2.97 ± 0.86 |
| Corrected mesh (ours) | 1.17 ± 0.28 | 2.46 ± 0.72 |
| − strain map | 1.47 ± 0.29 | 2.63 ± 0.75 |
| − wrinkle-band loss | 1.84 ± 0.44 | 2.99 ± 0.82 |
On average, removing either the strain input to the corrective network or the wrinkle-band loss increases mesh depth MAE on both splits. These results extend the evidence for their geometric contributions beyond subject 104.
Official Comment by Authors
We thank the reviewer for recognizing the motivation of strain-driven wrinkle modeling and the qualitative geometric improvements. Below, we address the geometry-stage ablations and the choice of deformation representation.
W1. Isolating the Geometry-Stage Contribution
We acknowledge this oversight and thank the reviewer for pointing it out. The geometry-stage measurements referenced in Sec. 4.3 were omitted from Appendix H. We report them in Q1 below, together with final rendering results under matched appearance-stage training. These ablations remove the strain map T from the corrective network Φ or disable the wrinkle-band loss Lwb, while retaining geometry optimization. They assess the geometry stage separately from the Gaussian-decoder strain ablation in Table 4.
Q1. Quantitative Geometry and Rendering Ablations
For each variant, we retain geometry-stage optimization and either set the input strain map T of the corrective network Φ to zero or disable Lwb. All other geometry-stage losses and the 200k-iteration budget remain the same. We then retrain the appearance stage on each resulting corrected mesh Vw for 600k iterations, using the same hyperparameters and FLAME finetuning protocol.
On subject 104 under Novel-View Synthesis, we evaluate the MAE of the rasterized mesh depth against the same stereo-depth reference and mask used in Appendix H.
For the final appearance-stage outputs, we report rendering quality and Gaussian-normal NCS against DAViD over the full foreground, following Table 4:
Both ablations increase mesh depth MAE, while the differences in rendering quality remain small. This is consistent with the appearance stage absorbing the photometric residual, as discussed in Sec. 4.3; image similarity alone does not isolate the contribution of the recovered geometry.
Q2. Choice of Deformation Representation
The main benefit is that the network predicts local Jacobian correctives ΔJf, and the Poisson solve integrates the corrected Jacobian field into a single coherent surface Vw. Compared with directly adding predicted displacements, this jointly determines vertex positions by fitting the local deformation gradients under the tracked face-boundary constraints.
We compare this formulation with direct UV displacement prediction using the same strain map T, U-Net backbone, RGB supervision, geometry-stage losses, boundary constraints, and 200k-iteration budget.
We report mesh depth MAE and mesh-normal NCS on the held-out FREE sequence of subject 210 against COLMAP MVS depth and its derived normals. We use the same hair-free masking protocol as in the paper, with frames sampled at one-second intervals.
Jacobian correctives followed by canonical Poisson integration reduce mesh depth MAE by approximately 8.0% under matched RGB supervision, supporting this design choice in the evaluated setting.
We have uploaded a preliminary revised manuscript with additional experiments and clarifications highlighted in blue, and will continue to update it during the discussion period.