- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper addresses monocular articulated portrait mesh estimation. It identifies three gaps in current pipelines: (i) a representation gap — face-centric 3DMMs assume a "floating head" with no torso-rooted kinematic reference, while body-centric models lack FLAME-level facial expression; (ii) a supervision gap — neck/shoulder/torso variables are weakly supervised in-the-wild; (iii) a jaw–expression shortcut — regressors let expression blendshapes absorb mouth opening, leaving the jaw joint nearly static. The authors propose: (a) the Portrait Parametric Model (PPM), which fuses a FLAME-2023 head onto an SMPL-X torso in canonical space via eye-center alignment and neck-ring smoothing, rooting the kinematic chain at the spine; (b) Progressive Anatomical Alignment (PAA), a three-expert (Global/Mid/Local) query-based regressor on a frozen Sapiens-2 encoder, trained with a Graduated-Mask Router that randomly drops deeper-expert supervision so articulation must explain the image before expression is available; (c) multi-source supervision (Sapiens-2 keypoints, Pixel3DMM feature distillation, foreground masks, relative geometry). Experiments report the lowest average 68-point NME across four in-the-wild datasets, the highest jaw–skeletal consistency proxy, head/face vertex errors comparable to TEASER on NoW/Stirling, and downstream gains for DiffPoseTalk and RGBAvatar.
Critical strengths
- Well-diagnosed problem with a matched training formulation. The jaw–expression shortcut is precisely characterized, and the Graduated-Mask Router directly targets it: forcing ψ=0 when the L-expert is masked means mouth opening must be explained by jaw articulation. The ablation supports the mechanism (JSR 0.368 flat → 0.710 w/o router → 0.799 full), and Fig. 5's JCR-vs-jaw-pitch time series is convincing evidence that the gain is mechanistic, not incidental.
- A usable, reusable artifact. PPM gives the community a simple torso-rooted portrait representation built from standard assets (FLAME + SMPL-X), with demonstrated downstream utility (DiffPoseTalk training-loss reduction, RGBAvatar floating-head artifact removal). This kind of representation fix is often adopted.
Critical weaknesses
- Evaluation partially circular on the headline metric. JSR measures correlation between predicted jaw pitch and JCR, but JCR is built from the same nose–chin landmarks that L_nc supervises. A method could score well by overfitting that 2D cue without genuinely better skeletal consistency. An independent validation (e.g., measured jaw excursion from multi-view or EMOCA/SPECTRE-style perceptual judges, or re-animation quality judged by a third-party renderer) would materially strengthen RQ2.
- Component attributions are muddy. The ablation shows graduated pseudo-labels (not the router) carry most of the NME gain, and the Nersemble-V2 vertex advantage is confounded by pretraining on that dataset. As a result, the reader cannot cleanly tell how much of the headline improvement is the proposed routing vs. better labels and data.
Questions for authors
- For Table 3, Nersemble-V2: were the test identities/sequences disjoint from the stage-1 pretraining split? Please state the exact split.
- JSR reuses the L_nc landmark cue. Can you report jaw–skeletal consistency measured against a cue not used in training (e.g., 3D jaw angle from Nersemble multi-view, or mouth-opening from an off-the-shelf audio-visual sync model)?
- Table 1: each baseline "uses its own official crop" — GRAPE sees neck/shoulders, which could systematically change the 68-point normalization. How was NME normalized (inter-ocular? bounding-box?), and does the comparison remain fair under a unified normalization?