- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper studies how strong facial expressions affect identity preservation and its evaluation in video generation. It extends Stand-In to use expression-diverse reference images and proposes gallery-based identity scoring across expression-intensity ranges. Results show improved similarity scores, although it remains unclear how much these gains reflect better identity preservation versus reduced expression intensity.
Critical strengths
The paper highlights an important evaluation issue: genuine expression changes can lower face-recognition similarity even when identity is unchanged. It supports this observation with real photographs and MEAD data, and includes useful ablations on reference count and expression diversity.
Critical weaknesses
- Showing that neutral-reference similarity decreases with expression intensity does not establish that taking the maximum over an expression gallery better measures identity. The same gallery also supplies the model’s conditioning images, making it difficult to distinguish identity generalization from reference matching.
- Table 2 shows that the proposed method produces fewer extreme-expression frames than several baselines. It is therefore unclear whether its higher identity scores reflect better identity preservation under equally challenging expressions. Table 3 considers peak intensity, which does not establish comparable expression strength throughout the video. A comparison at matched expression intensity would help resolve this concern.
- The proposed model is trained on 400k curated clips, whereas the baseline comparisons use official checkpoints. The existing ablations do not fully isolate the multi-reference extension from the benefits of this training.
- The authors do not discuss or compare with recent paper [1], which addresses closely related questions about identity preservation, expressive behaviour and the effect of reference expressions on identity evaluation. Comparisons with other recent methods, such as Slot-ID and Mv²ID, are also missing. The authors should clarify the novelty relative to these approaches and provide comparisons under compatible reference inputs, or explain why a fair comparison is not feasible.
[1] Baptiste, P., Quignon, N., Bahram, Y., Zeeshan, M. O., Dantcheva, A., & Granger, E. (2026). BEACON: Behavior and Appearance Control for Subject-Specific Video Generation. arXiv preprint arXiv:2609.13264.
Questions for authors
- The authors must evaluate with held-out reference photographs and test whether gallery scoring better separates genuine and impostor identities or agrees more closely with human judgments.
- Experiment by reporting the subset sizes in Table 3 and compare identity on matched subject–prompt pairs after controlling for the intensity distribution, rather than peak intensity alone.
- Provide a single-reference Stand-In control trained on the same data and budget, or clarify whether such a control already exists. For the synthetic setting, giving AnyID the same synthesized references would help isolate the benefit of the proposed conditioning mechanism.
3 comments
Response Part 2 of 3
Comparison at Matched Expression Intensity (W2, Q2)
We agree that matching peak intensity alone does not control the intensity distribution throughout a video. Table 6 (Table 3 in the original submission) now reports the proportions of video and frame pairs included in the higher-intensity subsets, together with the additional frame-level comparisons.
To control for the intensity distribution more directly, we added Table 13 in Appendix B.2. Within each subject–prompt pair, we divide frames into expression-intensity bins of width 0.5. For each bin reached by both models, we compare their mean nearest-gallery similarity. We then average the differences, weighting each bin by the smaller of the two frame counts. Thus, each comparison holds subject, prompt, and expression-intensity bin fixed.
Table 13. Identity difference at matched expression intensity.
Ours (Real) minus each baseline:
| Baseline | All: ΔArc | All: ΔCur | WR Arc % | WR Cur % | Mild: ΔArc | Mild: ΔCur | Intense: ΔArc | Intense: ΔCur | Extreme: ΔArc | Extreme: ΔCur |
|---|---|---|---|---|---|---|---|---|---|---|
| Concat-ID | +19.64 | +22.02 | 99% | 100% | +15.62 | +17.62 | +20.36 | +22.88 | +27.47 | +29.37 |
| Phantom | +8.02 | +8.43 | 92% | 92% | +4.48 | +5.93 | +7.82 | +8.24 | +12.18 | +11.63 |
| ConsisID | +41.21 | +42.06 | 100% | 100% | +34.77 | +36.32 | +40.69 | +41.53 | +47.37 | +47.91 |
| Stand-In | +18.55 | +19.66 | 97% | 97% | +6.88 | +7.31 | +20.13 | +21.44 | +26.50 | +27.46 |
| AnyID | +13.39 | +14.28 | 98% | 99% | +11.04 | +11.78 | +13.67 | +14.56 | +13.61 | +14.69 |
| Gen-4.5 | +21.08 | +21.99 | 99% | 100% | +12.47 | +12.96 | +22.22 | +23.25 | +30.87 | +31.82 |
Ours (Synthetic) minus each baseline:
| Baseline | All: ΔArc | All: ΔCur | WR Arc % | WR Cur % | Mild: ΔArc | Mild: ΔCur | Intense: ΔArc | Intense: ΔCur | Extreme: ΔArc | Extreme: ΔCur |
|---|---|---|---|---|---|---|---|---|---|---|
| Concat-ID | +17.59 | +18.88 | 98% | 98% | +15.40 | +15.62 | +17.84 | +19.37 | +17.84 | +18.49 |
| Phantom | +4.89 | +4.60 | 79% | 77% | +1.71 | +2.49 | +4.10 | +4.04 | +8.27 | +6.99 |
| ConsisID | +37.58 | +38.21 | 100% | 100% | +32.39 | +32.23 | +37.12 | +37.71 | +40.08 | +40.97 |
| Stand-In | +16.41 | +16.73 | 95% | 94% | +8.02 | +7.08 | +16.35 | +16.99 | +19.77 | +19.51 |
| AnyID | +9.14 | +9.40 | 93% | 93% | +9.89 | +9.66 | +9.33 | +9.65 | +7.95 | +8.08 |
| Gen-4.5 | +19.90 | +20.34 | 99% | 100% | +13.33 | +12.27 | +19.77 | +20.42 | +24.55 | +25.15 |
Both reference settings retain positive identity-similarity differences against every evaluated baseline in all three intensity regimes under both recognizers. These results support an identity-fidelity advantage after controlling for expression intensity under this protocol.
Response Part 3 of 3
Single-Reference Stand-In and Matched-Reference AnyID Controls (W3, Q3)
We have added the requested single-reference training control. We finetune the original Stand-In model on our curated OpenHumanVid data using the same training recipe as our multi-reference model, except that each training clip uses only its sharpest reference image.
Table 5 (last three rows). Single-reference training control.
| Model | All: Arc ↑ | All: Cur ↑ | Mild: % | Mild: Arc ↑ | Mild: Cur ↑ | Intense: % | Intense: Arc ↑ | Intense: Cur ↑ | Extreme: % | Extreme: Arc ↑ | Extreme: Cur ↑ |
|---|---|---|---|---|---|---|---|---|---|---|---|
| Finetuned Stand-In | 64.65 | 65.06 | 29.4 | 83.47 | 84.79 | 39.3 | 64.75 | 64.89 | 31.3 | 46.85 | 46.74 |
| Ours (Synthetic) | 76.35 | 76.54 | 14.9 | 83.16 | 84.31 | 71.0 | 76.48 | 76.53 | 14.1 | 68.48 | 68.36 |
| Ours (Real) | 80.78 | 81.68 | 29.0 | 83.68 | 85.03 | 63.5 | 80.23 | 80.95 | 7.4 | 74.27 | 74.80 |
Finetuning improves Stand-In's overall identity similarity, but our multi-reference model retains higher scores overall and in the intense and extreme regimes under both real and synthetic reference settings. This indicates that the gains cannot be attributed solely to additional training on our curated data.
We also compare our model with AnyID using identical references in both settings: five real images (5), or one real neutral portrait and four LivePortrait-generated expressions (1R+4S).
Table 8 (first four rows). Matched five-reference comparison.
| Model | Refs. | All: Arc ↑ | All: Cur ↑ | Mild: % | Mild: Arc ↑ | Mild: Cur ↑ | Intense: % | Intense: Arc ↑ | Intense: Cur ↑ | Extreme: % | Extreme: Arc ↑ | Extreme: Cur ↑ |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| AnyID | 5 | 62.83 | 62.74 | 7.8 | 73.86 | 73.77 | 54.4 | 66.36 | 66.02 | 37.8 | 55.48 | 55.77 |
| AnyID | 1R+4S | 60.95 | 60.82 | 5.9 | 74.80 | 74.52 | 52.3 | 66.13 | 65.37 | 41.8 | 52.53 | 53.18 |
| Ours | 5 | 77.22 | 77.77 | 24.9 | 83.92 | 85.05 | 60.6 | 77.03 | 77.39 | 14.5 | 66.51 | 66.82 |
| Ours | 1R+4S | 74.42 | 74.49 | 15.4 | 82.87 | 83.27 | 67.8 | 74.80 | 74.83 | 16.8 | 65.11 | 65.06 |
Our model retains higher identity similarity in both settings, supporting its benefit when the reference inputs are held fixed.
Comparison with Related Work (W4)
We have expanded the related-work section to discuss Mv$^2$ID, Slot-ID, and BEACON and clarify how their reference inputs and objectives relate to ours:
-
Mv$^2$ID uses multiview portraits to improve robustness to pose variation. Viewpoint changes describe how the same facial geometry is observed from different directions; expressions change the facial geometry itself through combinations of local deformations with varying intensity. Their coverage is consequently less intuitive: several visibly different expressions may provide similar information about one facial region while leaving other deformations unrepresented. A set of reference angles therefore does not directly translate into a criterion for selecting complementary expressions. This motivates our standardized eight-image expression gallery, which makes the choice of reference deformations explicit and reproducible. We also draw on pretrained facial reenactment models to synthesize the expressive references from a single portrait, using the prior knowledge of facial deformation encoded in LivePortrait.
-
Slot-ID and BEACON use video references to capture expressive variation, but do not specify explicit criteria for the expression coverage of those videos or a principled procedure for constructing suitable reference videos when only one portrait is available. The reference video's expression content and quality can therefore introduce uncontrolled differences into the comparison. We therefore cannot make a reliable, fair comparison under our reference-input setting without introducing additional assumptions about reference-video selection or synthesis. One of our contributions is to make both the reference content and its construction explicit: we define a standardized eight-image expression gallery, provide a LivePortrait-based procedure for generating it from a single portrait, and demonstrate identity-fidelity improvements using these synthetic galleries. Table 8 further compares our model with AnyID using exactly the same five-image synthetic gallery, controlling for the reference images supplied to both models. Our method thus provides a reproducible protocol for both captured and synthetic references, without requiring video-reference encoding or an additional VLM.
Thank you for considering the additional experiments and clarifications.
Response Part 1 of 3
Thank you for recognizing the importance of expression-aware identity evaluation and for highlighting the need to separate the effects of scoring, expression intensity, and additional training. We have added experiments and clarified the discussion to address these concerns.
Validation of Nearest-Gallery Identity Scoring (W1, Q1)
We agree that taking the maximum over a gallery requires validation beyond an increase in genuine similarity. Table 2 adds a leave-one-out evaluation on real photographs, excluding each query's expression from every subject's gallery. Nearest-gallery scoring reduces the mild-to-extreme gap in genuine similarity while slightly widening the margins against both mean and hardest-impostor scores.
Table 2. Leave-one-out metric validation.
Table 3 adds a study with six human evaluators. Nearest-gallery rankings agree with 79.3% of individual judgements and 88.6% of pairwise majorities, compared with 62.7% and 68.2% for single-reference rankings. These results provide complementary evidence from generated frames.
Table 3. Agreement with human identity judgements.
We also copy the held-out-expression evaluation from the appendix below. The query photographs depict four expressions excluded from the eight-image gallery. Nearest-gallery scores vary less across these expressions than neutral-reference scores, showing that the metric's robustness extends to expressions outside the gallery.
Table 11. Identity similarity on held-out real expressions.