- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 2
Read review
Paper summary
The paper proposes RetargetingFace, a framework for transferring facial expressions between systems with incompatible native controls. It learns a shared expression representation from expressive-minus-neutral DINOv3 features using expression supervision, conditional rig confusion, and cross-rig alignment. Rig-specific predictors are subsequently trained through same-rig coefficient reconstruction and reused across source systems. Validation includes learned expression-space similarity, coefficient-cycle recovery, and blinded human ratings.
Critical strengths
- The authors evaluate additional native systems and expressions extracted from three sources and considered human evaluation
+The proposed Neutral-relative FE that is subtracting matched-neutral features aims to suppress static appearance information while retaining expression-induced changes, is a great idea and and sound. Experiments seems to confirm its relevance.
-
Authors consider human evaluation for this task which is crucial as often quantitative metrics used in the field do not reflect well human perception.
-
The controlled cross-rig dataset can help/enable systematic evaluation of expression transfer and I believe it can be a useful resource for 3d animation modelling/facial animation research.
Critical weaknesses
-
My main concern is with the paper’s writing. I had some difficulties reading it due to several reasons. The authors mention several acronyms/concepts without properly definig them since the abstract (like GNM). Overall I had difficulty following the manuscript, particularly the motivation and positioning of the contribution. The introduction explains the objective of reusing target predictors across source rigs, but it remains unclear what technical limitation of existing shared-representation approaches prevents this. What is the main methodological advance beyond combining a shared encoder with rig-specific predictors? This distinction needs to be developed more explicitly to assess the significance of the contribution. For instance , why we need a controlled cross-rig dataset is just discussed on section 3 (lines 116 to 120). However, even there the motivations/argument for this need is severely lacking.
-
Moreover, Related work section is largely presented as a sequence of brief descriptions rather than a connected argument establishing the unresolved problem and the proposed advance. Terms such as “shared expression interface” and “native-control interoperability” are repeated without making the methodological distinction substantially clearer. At the end I am still not sure if the paper has significant novel methodological contribution other than the controlled dataset.
-
Subsection "Expression coverage and native realization (lines 121 to 127) were quite confusing to me. Seems a mix of related/past work and proposal but the message is blurry. I would like the authors to clarify that in the rebuttal and manuscript. In addition, as the paper does not fully uses the 9 pages, it would be greatly appreciated if some sort of overview were presented at the beginning of section 3.
-The choice of DINOv3 as the visual backbone is insufficiently justified. Section 4.2 (lines 193–194) specifies its frozen CLS features, but the paper neither explains why these features are particularly suitable for capturing subtle facial actions nor compares them with other alternative backbones. So it is still unclear how much the results depend on DINOv3 and whether a simpler or face-specific encoder would suffice.
-
In my opinion, relative loss weighting of the training objectives is not sufficiently examined. The authors refer to Appendix A.5 for details, but this section combines many implementation choices, making the relevant information difficult to locate. It specifies equal weights for the terms in Equation (2), an alignment weight of 0.25, and a confusion weight increasing to 0.2, without explaining how these values were selected or assessing their sensitivity. Expression discrimination and rig invariance may impose competing constraints: for example, suppressing rig-dependent differences in how unilateral eye closure is realized could also remove useful expression details. A small sensitivity analysis would help establish whether the selected weights provide a robust balance between these objectives.
-
I also question whether the headline 18.7% improvement in human similarity is more like overselling the results (lines 24 - 28 and later lines 80 - 84) as it omits other comparisons /baselines like the Direct MLP). Appendix A.3 clarifies that this comparison excludes Direct MLP, the strongest evaluated baseline. Against it, the human scores are 2.33 versus 2.17 and 2.48 versus 2.32 (lines 299–305), approximately 7.1% average relative improvement. So the headline seems misleading.
-
The unseen-rig human results reported in Table 4 shows very small gain margins such as 2.13 versus 2.06 for FLAME→GNM, 2.11 versus 2.02 for MHR→GNM, and 2.14 versus 2.04 for MHR→FLAME.Are these gains consistent across source cases, or driven by a few examples? Since both methods are evaluated on the same 50 cases by the same 20 raters, paired confidence intervals could be computed from the existing ratings. In addition, I would like to see a paired analysis of the existing ratings for example, a paired t-test or a non-parametric case (Wilcoxon signed-rank test if differences is not normal), together with confidence intervals to assess whether these small differences provide reliable evidence of a perceptual advantage.
-
Lastly, to my understanding, the cross-rig supervision assumes that the same action and relative strength correspond to comparable expressions in both rigs. However, FLAME and ARKit coefficients are independently extracted and then scaled along their respective native paths. Does the same scaling produce comparable perceptual intensity? How frequently do the extractors disagree on the expression being conveyed? These points seems to impact the positive correspondences and as such should not be neglected.
Questions for authors
My main concerns are with the manuscript’s clarity and positioning, as well as the justification of several methodological choices and the strength of the empirical conclusions (see the critical weaknesses for details). At the current moment I lean on rejection mostly due to writing and issues in the experiments. Here are some more detailed points that I would like to see addressed based on the weaknesses raised in the previous section.
1)What is the main methodological advance beyond combining a shared expression encoder with rig-specific predictors? Can the authors clarify which technical limitations of the closest shared-representation approaches prevent the proposed functionality, and why a new controlled cross-rig dataset is necessary?
-
How were the relative loss weights selected? Can the authors provide a small sensitivity analysis showing whether the conclusions remain consistent under alternative weights, particularly when balancing expression discrimination, rig confusion, and cross-rig alignment?
-
Are the small human evaluation score gains (presented in Table 4) consistent across source cases? Can the authors provide paired confidence intervals and an appropriate paired statistical analysis (e.g., paired t-test or wilcoxon signed rank test) of the existing ratings whether these gains rare significant?
-
What evidence supports treating the same action and relative strength as perceptually comparable across rigs? Since FLAME and ARKit coefficients are independently extracted and scaled, can the authors quantify disagreements in action or intensity on a representative sample and explain how these affect the positive correspondences used for training?
5)Is the 8.7% improvement the headline justified, given that it excludes Direct MLP from it (strongest baseline apparently)?
3 comments
Q2: How were the loss weights selected, and how sensitive are the results to the balance between expression discrimination, rig confusion, and cross-rig alignment?
A2: We selected a confusion cap of 0.20 and an alignment weight of 0.25 to bring the two rigs’ mean codes closer for the same action and strength while preserving retrieval of individual matches. We varied one weight at a time with the same seed and update budget. In all runs, the four expression terms had unit weight and 72 tuples each per update. We introduced rig confusion gradually after expression training began.
Table R3.1. Weight sensitivity at the same seed and training budget. The first numeric column in each panel gives the default. F = FLAME, A = ARKit. Strength order and mAP are percentages.
(a) Confusion cap (alignment weight fixed at 0.25)
| Metric | 0.20 | 0.10 | 0.40 |
|---|---|---|---|
| Rig-mean discrepancy ↓ | 0.1335 | 0.1580 | 0.0933 |
| F→A mAP ↑ | 4.13 | 4.21 | 3.70 |
| A→F mAP ↑ | 3.24 | 3.30 | 2.94 |
| Strength order ↑ | 99.33 | 99.23 | 99.57 |
(b) Alignment weight (confusion cap fixed at 0.20)
| Metric | 0.25 | 0.50 | 1.00 |
|---|---|---|---|
| Rig-mean discrepancy ↓ | 0.1335 | 0.1483 | 0.1592 |
| F→A mAP ↑ | 4.13 | 4.27 | 4.32 |
| A→F mAP ↑ | 3.24 | 3.41 | 3.53 |
| Strength order ↑ | 99.33 | 98.97 | 99.07 |
Raising the cap from 0.10 to 0.20 reduced rig discrepancy with little change in retrieval. The paired 95% intervals for the mAP change were [−0.18, +0.01] percentage points for F→A and [−0.13, +0.02] for A→F. At 0.40 the means moved closer still, but mAP fell by 0.43 [0.31, 0.54] and 0.30 [0.21, 0.40] points, respectively, relative to 0.20.
Increasing the alignment weight improved retrieval but widened the gap between rig means. We kept the default weight of 0.25, which also gave the lowest Validation rig separability (0.669, versus 0.723 at 0.50 and 0.686 at 1.00). Sec. 4.3 lists the training weights. Appendix A.5 (Table 13) reports the sensitivity results and paired intervals obtained by resampling the same 478 Test actions with checkpoints and galleries fixed.
Q3: Are the human-rating gains in Table 4 consistent across source cases, and what do paired confidence intervals and appropriate statistical tests show?
A3: In the supplied Table 4 scores, RetargetingFace had a higher mean rating on 27–39 of the 50 cases in each direction. We counted wins at the case level by averaging the 20 paired rating differences for each source case. Those same 50 case means enter the paired analysis, so every case carries equal weight.
Table R3.2. Case-level comparison of the supplied Table 4 scores. A win means that RetargetingFace had a higher case mean than Direct MLP.
| Direction | Mean gain | Adjusted interval | Wins / ties / losses | Holm p |
|---|---|---|---|---|
| FLAME→GNM | +0.070 | [0.040, 0.105] | 27 / 23 / 0 | 2.98e-08 |
| FLAME→MHR | +0.340 | [0.239, 0.444] | 39 / 11 / 0 | 2.18e-11 |
| GNM→FLAME | +0.300 | [0.210, 0.393] | 37 / 13 / 0 | 7.28e-11 |
| MHR→FLAME | +0.100 | [0.059, 0.145] | 29 / 21 / 0 | 1.49e-08 |
| GNM→MHR | +0.120 | [0.070, 0.175] | 29 / 21 / 0 | 1.49e-08 |
| MHR→GNM | +0.090 | [0.053, 0.132] | 29 / 20 / 1 | 2.98e-08 |
We resampled cases using 200,000 paired bootstrap draws and applied a Bonferroni adjustment for approximately 95% simultaneous coverage across the Table 4 comparisons. All adjusted intervals lie above zero. Exact two-sided sign-flip tests using the same case-level differences gave a maximum Holm-adjusted p-value of 2.98×10⁻⁸.
The case-level analysis treats the observed rater panel as fixed. It assumes independent cases and, under the null, exchangeable method labels. To check sensitivity to the choice of raters, we also resampled cases and raters together. The adjusted intervals remained positive in every direction (Appendix A.7, paired analysis and statistical protocol).
Q4: What evidence shows that the same action and relative strength are perceptually comparable across rigs, how often do the independently extracted and scaled realizations disagree, and how do these disagreements affect the training positives?
A4: We form a cross-rig positive pair by matching action and relative strength, . We tested how closely that label describes the rendered faces by comparing 9,560 renders per rig (478 actions × two held-out variants × ten strengths), with geometry fixed within each rig. We scored changes from neutral on both sets of renders with the same frozen MediaPipe readout. MediaPipe also supplies the ARKit labels.
Table R3.3. Agreement between extracted FLAME and ARKit expressions on strength and action, using a frozen MediaPipe readout.
| Comparison | Estimate [95% action-bootstrap interval] |
|---|---|
| Strength vs. readout magnitude, FLAME (Spearman ρ) | 0.9730 [0.9665, 0.9787] |
| Strength vs. readout magnitude, ARKit (Spearman ρ) | 0.9873 [0.9855, 0.9890] |
| FLAME vs. ARKit strength ordering (Spearman ρ) | 0.9618 [0.9553, 0.9678] |
| Matched ARKit action closer than a different action to the FLAME query (%) | 82.71 [81.81, 83.60] |
The first three rows use expression magnitude across all 51 channels. As we increased the strength of each action, the two rigs gave similar orderings (mean Spearman ρ = 0.9618). We then held variant and strength fixed and compared each FLAME expression with its matching ARKit action and all 477 alternatives. In this exploratory test, the match had higher cosine similarity on the 43 non-gaze channels in 82.71% of pairwise comparisons, counting ties as half.
To see where a positive pair can disagree, we looked at which eye closed more. In 1,517 pairs, the left and right blink scores differed by at least 0.05 on both rigs. The readout indicated that the same eye closed more on both rigs in 1,396 pairs (92.02%). In the other 121 (7.98%), it indicated opposite sides. Those pairs still share an action and strength label, so they can give the encoder a noisy signal about which eye closes more. During training, expression triplets and strength ordering constrain the codes alongside these cross-rig positives (Sec. 3, p. 3 and Appendix A.1).
Q5: How should the headline improvement be interpreted when Direct MLP, the strongest baseline, is excluded from that percentage?
A5: Direct MLP is the strongest human-rated baseline on the main transfers. On the 1–3 scale, our mean gains over it are +0.16 points in each direction (2.33 vs. 2.17 for FLAME→ARKit and 2.48 vs. 2.32 for ARKit→FLAME). The relative gain depends on which baseline supplies the denominator.
Table R3.4. Average relative gain in mean expression-similarity ratings for each choice of baseline.
| Baseline used for relative gain | Main transfers | External expressions |
|---|---|---|
| Direct MLP | 7.1% | 8.4% |
| Strongest previously published method evaluated per setting | 18.7% | 9.9% |
We compute the relative gain within each setting, then average the two main directions or the six external dataset–direction pairs equally (Appendix A.3). The main published baselines are MediaPipe for FLAME→ARKit and SMIRK for ARKit→FLAME. The revised abstract and Introduction report the Direct MLP gain first and identify the published baselines separately.
W4: The choice of DINOv3 as the visual backbone needs further justification.
Response to W4: We retained DINOv3 because it met our reconstruction criterion on both rigs. For each rig, we set the limit at 5% above the original same-seed predictor’s mean Validation NRMSE before examining the Test results. To compare backbones under that criterion, we retrained the Projector and both predictors with the same seed and training budget for each backbone.
Table R3.5. Change in mean within-rig Validation error relative to the original same-seed predictor. Lower values indicate better reconstruction.
| Backbone | FLAME change in mean NRMSE | ARKit change in mean NRMSE | Rigs within the 5% limit |
|---|---|---|---|
| DINOv3 | +2.73% | −1.32% | 2 / 2 |
| DINOv2 | +6.93% | −0.75% | 1 / 2 |
Thank you for recognizing the value of the neutral-referenced features and the controlled dataset. Your questions helped us explain why the training design allows predictor reuse, how the experiments informed our choices, and what the reported gains mean.
Q1: What is the methodological advance beyond a shared encoder with rig-specific predictors, how does it differ from related work, and why is a controlled dataset needed?
A1: We learn expression correspondence across rigs, then train each predictor using only its own rig’s codes and controls. The ARKit predictor, for example, learns from (zA,cA) pairs. Because the encoder has aligned the expression codes, the same predictor can take a FLAME code at inference and produce ARKit controls as DA(zF). We can therefore change the source and reuse the trained mapping.
For this to work, a code from the source rig must carry the expression information the predictor learned to use on the target rig. We reference expressive features to matched neutrals and use expression supervision to preserve differences in action and strength. Conditional rig confusion and multi-positive alignment encourage the two rigs to encode those expressions compatibly. We then freeze the encoder and train each native predictor separately. In Sec. 5.4, we test reuse by adding the GNM and MHR predictors with the representation and existing predictors fixed.
The closest methods differ in what their shared representations drive. Kim et al. (2021) synthesize a target image from a shared image latent and predict blendshapes from that image. SEREP (Josi et al., 2025) separates expression from identity and decodes a mesh. UniTalker (Fan et al., 2024) shares an audio-driven model across heterogeneous output annotations. In our method, an independently trained target predictor maps the shared visual code directly to that rig’s existing controls.
The controlled dataset supplies the two kinds of supervision these stages need. The encoder needs expressions matched across rigs, and each predictor needs the corresponding native controls. Captured images alone do not directly give us both. We therefore render source expressions with the Generative aNthropometric Model (GNM), an existing parametric head model, and extract native controls using SMIRK and MediaPipe. We link the rigs by matching action and strength. Rendering fixed coefficients on different FLAME shapes isolates shape variation, and rendering the same expression at successive strengths gives us an ordered sequence. The extracted coefficients then supervise each rig’s predictor. We organize the data by expression composition, source variant, strength trajectory, and shape, with held-out splits for evaluation.
As the revised abstract puts it (p. 1), “Each target predictor trains once on its own rig's expression codes and native controls, then decodes expression codes from different source rigs.” The Introduction defines the rigs and cites their sources. Sec. 2 discusses the closest methods, and Sec. 3 gives a worked example of the supervision.