- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper introduces ExDiff, a diffusion prior over the expressive parts only (both hands and the face), using SMPL-X as a decoding and fitting scaffold. The method combines part-aware noise schedules with a part-balanced score loss, a zero-initialized cross-part fusion head trained on about 160k paired hand-face frames, and geometry-aware supervision through SMPL-X decoding. One model is evaluated on generation, recovery, and inverse kinematics across six datasets, and it beats DPoser-X and other recent baselines on most metrics, for example ARCTIC recovery 5.12 vs 6.53 mm and FreiHAND JPE 4.6 vs 5.0 mm. Ablations cover the noise schedules, the fusion head, and the training strategy.
Critical strengths
-
The paper targets a real gap that existing priors model the full body or treat hands and face separately, and EXDIFF is, to my knowledge, the first diffusion prior focused exclusively on both hands and the face. A single lightweight model supports generation, missing-part completion, and monocular recovery.
-
The part-balanced formulation with region-adaptive noise schedules is a sensible way to train hand and face parts in one diffusion space from heterogeneous hand-only, face-only, and paired data, and the zero-initialized residual head is a clean design for adding cross-part interaction without disturbing the pretrained part priors.
Critical weaknesses
-
The paper claims EXDIFF "consistently improves realism, coverage, and diversity on ARCTIC, FreiHAND, NOW, and BEAT2", but BEAT2 only appears in the Fig. 6 ablation with recovery errors, with no generation metrics and no baseline comparison, and in Table 1 NRDF is actually higher on APD (6.74 vs 6.52) and recall (0.94 vs 0.88). The authors should either add the missing BEAT2 generation results against baselines, or scope the claim to what the tables actually show.
-
The central hand-face dependency claim has only partial support: Table 8's joint-precision recovery is consistent with learned dependencies, but the dependencies named in Sec. 3.6 are never directly measured. The authors may need to add a matched-capacity control without cross-part attention and directly measure the named dependencies for solidity of the experiment supports.
Questions for authors
Please explain the raised weakness.
4 comments
Weakness 2
We thank the reviewer for this suggestion. We have added a matched-capacity control and directly evaluated the hand–hand and hand–face dependencies described in Section 3.6.
A. Matched-capacity control without cross-part attention. F-indep uses the same 2.1M-parameter fusion head and 160k paired training frames as ExDiff-F, but replaces cross-part attention with block-diagonal attention (Table R4, in our response to Reviewer 2). Compared with F-indep, ExDiff-F reduces recovery error from 5.52 to 5.12 mm on ARCTIC and from 7.50 to 6.52 mm on BEAT2, and improves joint generation precision from 0.80 to 0.83.
B. Direct measurement of cross-part dependencies. We evaluate the dependencies described in Section 3.6 by conditional completion on the ARCTIC and BEAT2 test sets, with 10 samples per frame: right-hand completion given the left hand, hand completion given the face, and jaw completion given the hands. We also measure the correlation between jaw opening and finger flexion. Table R8 gives the results together with the independent and shuffled-pair controls.
Both dependencies show up against the matched control. Mirrored fingers are strong: the right-hand error given the left hand drops from 32.2 to 27.1 mm on ARCTIC and from 37.8 to 31.2 mm on BEAT2, and F-shuf, which keeps true left–right pairs, obtains most of this gain. Jaw–finger co-articulation is weaker and appears mainly on BEAT2, where faces carry expression: the hand error given the face drops from 37.2 to 34.5 mm, the jaw error given the hands from 8.5 to 7.1 degrees, and the jaw–finger correlation is 0.09, against 0.11 in the real data. On ARCTIC, where the face is jaw motion only, the face–hand gain is small.
Table R8. Direct evaluation of cross-part dependencies. Conditional completion on ARCTIC and BEAT2 test, mean over 10 samples per frame. R|L and H|F: hand PA-MPJPE (mm) given the ground-truth partner part. F|H: jaw-angle error (degrees) given the ground-truth hands. ρ: correlation between jaw opening and mean finger flexion over 5,000 generated joint samples; the real value is from BEAT2 test. F-shuf randomizes face–hand pairing while keeping true left–right hand pairs.
| Model | R|L ARCTIC ↓ | R|L BEAT2 ↓ | H|F ARCTIC ↓ | H|F BEAT2 ↓ | F|H ARCTIC ↓ | F|H BEAT2 ↓ | ρ (jaw, fingers) |
|---|---|---|---|---|---|---|---|
| ExDiff-B | 34.4 | 39.4 | 33.8 | 39.3 | 3.3 | 8.7 | 0.00 |
| F-indep | 32.2 | 37.8 | 32.4 | 37.2 | 3.3 | 8.5 | 0.00 |
| F-shuf | 28.7 | 32.5 | 32.1 | 36.1 | 3.3 | 8.3 | 0.01 |
| ExDiff-F | 27.1 | 31.2 | 31.3 | 34.5 | 3.1 | 7.1 | 0.09 |
| Real BEAT2 test data | – | – | – | – | – | – | 0.11 |
C. Manuscript revisions. We have revised Section 3.6 to clarify the supported cross-part dependencies, added the completion protocol and Table R8 to Appendix A.1, and updated Section 4.2 to discuss the direct measurements and their limitations.
We would like to thank you again for your careful review and constructive suggestions. We hope our responses have addressed your concerns regarding the paper. Please let us know if you have any further questions or if any additional information is required.
Thanks for pointing out both issues. The BEAT2 baselines, the scoped abstract, and especially the matched-capacity control with the direct completion evaluation address the majority of my concerns, and I am keeping my weak acceptance.
Weakness 1
We agree on both points. We add BEAT2 baselines and scope the claim in the abstract.
A. BEAT2. Correct. BEAT2 was used only for recovery, and only among our own variants. Table R7 adds DPoser-X with existing checkpoints, both for recovery in the setting of Fig. 6 and for joint generation on BEAT2 test under the Table 1 protocol. The ranking matches ARCTIC. ExDiff-F lowers the recovery error from 9.71 to 6.52 mm (face from 7.31 to 4.52) and is best on every generation metric, with FID falling from 0.14 to 0.05, precision rising from 0.67 to 0.82 and recall from 0.75 to 0.87. DPoser-X†, trained on our data, is behind ExDiff-B on FID, recall and dNN and level with it on precision (0.72).
Table R7. BEAT2 test with baselines. The recovery columns report PA-MPVPE (mm) with SMPLer-X initialisation, as in Fig. 6. The generation columns follow the Table 1 protocol, 5k generated against 5k BEAT2 test samples with 3 nearest neighbours.
B. NRDF is higher on APD and recall. Correct, and our own text said "second-highest" while the abstract said "consistently". NRDF's high APD and recall come with a precision of 0.05 and an FID of 0.78, so its samples are spread widely and mostly lie off the real manifold. Separately, the revision removes the NRDF and VPoser rows from Table 1, because neither has a verified whole-body hand–face checkpoint (Reviewer 1, Weakness 2).
C. The scoped claim. Against DPoser-X, ExDiff-F is best on every generation metric except hand FID (0.02 against 0.01) and has the lowest recovery error on ARCTIC, FreiHAND, NoW and BEAT2. The abstract now reads: "Against DPoser-X, ExDiff improves generation realism and coverage on joint, hand and face benchmarks and lowers recovery error on ARCTIC, FreiHAND, NoW and BEAT2." The words "consistently" and "diversity" are removed, Sec. 4.1 states the one exception (hand FID), and Table R7 goes into the revision.