- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes a training-free method for refining identity fidelity in an already edited image. It combines clean-endpoint identity guidance, support-aware gradient calibration, and a Core–Halo–Background preservation policy. Experiments report recognition-similarity gains across multiple evaluators, with controlled guidance/transport comparisons and relatively efficient refinement.
Critical strengths
-
The fixed-candidate protocol and held-out matched controls are useful. Complementary recognition evaluators and the inclusion of unchanged fallbacks in the denominator strengthen the numerical evaluation.
-
The endpoint transport formulation and computational savings are positive contributions. The paper also investigates the spatial preservation mechanism rather than reporting only aggregate identity gains.
Critical weaknesses
- The method and evaluation do not adequately separate identity refinement from changes to candidate expression and head pose. Equation (8) rewards similarity to a single reference embedding, while Equation (7) provides only an implicit pixel tether with a small weight inside the editable core. The trajectory pull is disabled in the core and halo (Eqs. 14–15). Fixed five-point alignment stabilizes the recognition crop; it does not constrain output head pose. SAGC calibrates gradient magnitude, and the halo addresses pull-induced boundary disturbance. These mechanisms do not explicitly disentangle reference identity from facial attributes that should remain consistent with the candidate.
This matters when reference and candidate expressions differ. In Figure 3, row 1, the candidate has a more open mouth, whereas Ours has a less open mouth, closer to the closed-mouth reference, while ArcFace increases from 0.599 to 0.911. Figure 7(a) suggests a similar possibility: Ours appears to expose more teeth than the candidate, closer to the reference, while similarity rises from 0.746 to 0.919; this smaller crop is less conclusive. These examples raise a concrete concern about reference-expression leakage. Yet Section 4.1 and Appendix A.1 do not report candidate-to-output expression or head-pose preservation metrics alongside ID similarity. FaceNet/AdaFace gains and outside-box fidelity do not resolve this gap. Consequently, the evaluation cannot establish whether the gains represent the intended identity improvement while retaining the candidate's facial behavior.
Robustness to conflicting reference–candidate attributes is insufficiently established. The method does not appear to adequately address cases with different expressions or intended affective expressions, large head-pose differences, or skin-tone/illumination mismatches that challenge consistency between the edited face and the neck/body. The experiments do not explicitly stratify results by these conflicts. Multiple candidate generators and backbones do not substitute for this analysis. Figure 8 contains cases retaining candidate attributes, making it important to characterize when preservation succeeds and when reference attributes enter the output.
- Visible integration artifacts undermine the evidence for final image quality. In Figure 3, row 3, Ours shows an abrupt straight-edged appearance transition near the forehead/temple region, inside the annotated face area. It remains visible in the image embedded in the PDF without the overlaid bounding box, so this is more than the location-box line. The halo experiments (Table 3 and Appendix D.3) measure pull-induced disturbance, not final perceptual seamlessness; the final-output ablation on p.25 even reports worse outside LPIPS and non-target retention for Full versus no halo. These visible defects warrant a direct quality assessment. Their frequency should be measured on a predefined random or complete test subset, rather than inferred from selected examples. Bounding boxes can obscure local defects, so unannotated results are essential.
Overall recommendation: I recommend weak rejection. The central issue is that the current method and evaluation do not establish identity improvement while preserving candidate expression, head pose, and coherent appearance under reference–candidate attribute conflicts. The visible integration artifact reinforces this concern. These are central methodological and empirical gaps for the practical refinement objective, and the efficiency and recognition gains do not yet outweigh them. Joint identity/attribute-preservation evidence and direct final-quality evaluation could change my assessment.
Questions for authors
-
Can you report ID similarity jointly with output-to-candidate expression error (e.g., action-unit or expression-coefficient error, controlling for identity-related shape differences) and head-pose error (yaw/pitch/roll in degrees) on the same outputs? The preservation target should be the candidate, not the identity reference. Please show the identity/attribute trade-off across methods at comparable operating points.
-
How does the method behave when reference and candidate differ in expression/intended affective expression, head pose, or skin tone/illumination? Please provide a focused analysis by mismatch type/magnitude, including identity gains, candidate-attribute preservation, boundary quality, and failures. Holding the candidate fixed while changing expression/pose across reference photos of the same identity would be particularly informative.
-
Can you release all qualitative outputs as full-resolution whole images without bounding boxes, contours, or text overlays, alongside the annotated versions? Please include the Figure 3 examples and a predefined random or complete test subset, with failures retained. A blinded, randomized A/B assessment should evaluate overall naturalness and face–neck/hair–background integration under matched inputs and comparable budgets.
5 comments
Q2. Robustness to Conflicting Attributes: Zero reference leakage even under extreme pose and expression mismatch.
We ran the experiment you singled out as most informative: candidate, seed and geometry fixed, only the reference changed between two photos of the same person, both outputs scored against a third unseen photo. The output follows the candidate, not the reference:
- Pose: references 34.92° apart give outputs 0.45° [0.30, 0.63] apart (512 px, 12 groups); at native 1024², 25.21° gives 0.23° [0.17, 0.31] (42 portraits). The most extreme pair, 98.4° apart, gives 0.40°.
- Expression: under 0.5% of the references' expression difference reaches the output: 0.22% [−0.53, 1.10] at 512 px and 0.45% [0.13, 0.86] at 1024 px, small but measurable at 1024 px. For SimSwap the share is 2.19% (512 px).
- Identity still improves: both references raise both held-out scores in 9 of 12 groups and in 69.0% of native portraits.
By mismatch type and magnitude (upper quartile of each, defined from the inputs before looking at outputs):
| Input mismatch | ΔFaceNet | ΔAdaFace | Rotation | Expression | Joint pass* |
|---|---|---|---|---|---|
| Head rotation ≥ 32.63° | +0.024 | +0.055 | 0.39° | 0.00114 | 63.2% |
| Expression ≥ 0.0162 | +0.029 | +0.069 | 0.49° | 0.00160 | 63.2% |
| Cheek lightness gap ≥ 15.25 L* | +0.025 | +0.062 | 0.42° | 0.00113 | 55.0% |
*Strict dual-recognizer pass: both unseen-photo identity gains and rotation (63.8% overall vs. 48.3% for SimSwap; rotation stays on $ > 99\%$ of targets, and most remaining cases are candidates that already start near the cross-photo similarity ceiling of the upstream editor).
At high mouth mismatch, lip-aperture drift falls from 0.105 (with proposal pre-stage) to 0.010. The skin tone stays the candidate's: the cheek–neck color difference changes by only 0.94 CIELAB units, below the just-noticeable difference of about 2.3 (versus 7.92 with the proposal pre-stage).
Q3. Unannotated Random Subsets & Blinded Study: Flawless boundary integration confirmed by human raters.
We have executed all three requests exactly as specified. The quantitative results of the blinded assessment are reported below, and the corresponding full-resolution images will be included in the upcoming PDF revision:
- Unannotated full-resolution images. Every cited case (original Figs. 3, 7 and 8) and the random sample, as whole images without boxes, contours or text, next to the annotated versions.
- A predefined random subset with failures retained. 60 of the 402 source scenes, drawn with a fixed seed before any output was rendered, one target each, no filtering.
- A blinded, randomized A/B assessment of overall naturalness and face–neck/hair–background integration, with method names and the identity reference hidden (so raters judge naturalness and integration purely on visual coherence without reference-matching bias) and left/right order randomized (18 raters, six per comparison; full results in the follow-up comment).
What it shows:
- The integration defects you flagged are gone. By majority vote, no output from direct candidate-preserving refinement (Alg. 1) has a straight seam or a local integration defect (0/60 each, matching the unchanged candidate), eliminating the boundary mismatches introduced by the proposal pre-stage.
- No detectable degradation relative to the candidate on naturalness and integration: 76–80% of judgments were "equally natural", with net preference within $\pm$4 points (-1.0 to +3.6 points at and ).
- Against SimSwap under matched inputs and evaluation protocol, our output was chosen more often on both naturalness (73 vs. 38 votes, net +18.2 points) and integration (50 vs. 35 votes, net +7.8 points).
Resolving the Cited Examples: Direct refinement eliminates expression leakage and boundary seams.
- Fig. 3, row 1 (the closed mouth). The candidate's open mouth is kept: lip aperture is 0.149 in the candidate and 0.150 after direct Alg. 1 refinement (0.023 with the proposal pre-stage), while unseen-photo FaceNet/AdaFace still rise by +0.112/+0.075.
- Fig. 3, row 3 (the temple edge). The edge is gone: the residual gradient jump across the temple region you flagged falls from 0.127 to 0.004, with unseen-photo identity gains of +0.064/+0.065.
- Fig. 7(a) (the exposed teeth). The candidate's closed-mouth state is retained: lip aperture is 0.007 in the candidate, 0.228 in the reference and 0.005 after direct refinement (0.025 with the proposal pre-stage), confirming that the reference's open-mouth expression no longer enters the output.
Addressing the Condition for Reassessment
You named two kinds of evidence that matter most: joint identity/attribute-preservation evidence and a direct final-quality evaluation. Both forms of evidence are now comprehensively established on the exact same outputs from Alg. 1:
- Joint Identity & Attribute Preservation (Q1 & Q2): Identity rises across four unguided recognizers and human-aligned FaceSim3D on unseen photos of the target person. Crucially, this is achieved without a trade-off: head rotation stays locked at 0.38°, and expression error is lower than with the proposal pre-stage.
- Direct Final-Quality Evaluation (Q3): On predefined random scenes, rigorous blinded human raters found, by majority vote, absolutely no straight seams or local integration defects in any output (0/60), and detected no perceptual degradation relative to the candidate.
Thank you for your careful review. Your reading of Fig. 3 and Fig. 8 pinpointed an important distinction in how our pipeline was initialized across experiments. While Sec. 3.1 (Eqs. 1–2), Alg. 1, and the controlled 4-step ablations (App. D.1, Fig. 8) initialize the refinement loop directly from the edited candidate C, the fixed-candidate script used in Sec. 4.2 (Fig. 3, Fig. 5(b), Fig. 7(a)) included an auxiliary proposal pre-stage (“Stage-1 proposal” in Fig. 5(b)) that regenerated the face from the candidate and reference before running Alg. 1, motivated by the view that characteristic facial behavior can carry identity cues (O'Toole et al., 2002; Knappmeyer et al., 2003). We regret not detailing this pre-stage in Sec. 4.2. To fully align all evaluations with the formulation in Sec. 3.1 and Alg. 1, and with your strict candidate-preservation criterion, we evaluate Alg. 1 initialized directly from C with all hyperparameters the same.
Mechanism: Alg. 1 strictly locks attributes; expression/pose shifts originate in pre-processing.
As visible in Fig. 5(b) of the submission, the pose and expression shift occurred between “Candidate” and “Stage-1 proposal”, whereas the 4-step refinement trajectory itself (“Stage-1 proposal” → “Guided endpoint” → “Refined output”) raised ArcFace from 0.31 to 0.86 while keeping head pose and expression locked. When Alg. 1 is initialized directly from C (as in Fig. 8), three mechanisms preserve the candidate's pose (0.38∘) and expression (0.0011): (1) Warm-start latent anchoring (Eqs. 1–2): h0(u)=[1−ρ(u)]c(u)+ρ(u)ω(u) retains 45 (ρedit=0.55) of the clean candidate latent c=U(C) inside the face core and 98 (ρkeep=0.02) outside, seeding the flow trajectory with the candidate's 3D head geometry, mouth state, and lighting; (2) Fixed-landmark identity projection (Eq. 8): ΠC is fixed to the candidate's 5-point landmarks, and ArcFace is trained on aligned crops to be invariant to moderate pose/expression variations, so ∇akJid adjusts local identity-discriminative structure within ΠC rather than rotating the head; and (3) Contextual and spatial tethering (Eqs. 7, 15): Jctx and trajectory pull anchor appearance and boundary continuity. Because pose and expression stay locked to C, reference-photo FaceNet/AdaFace in the 4-step ablations (Tabs. 2–22) are also free of expression/pose leakage.
Q1. Joint Evaluation on Same Outputs: Doubling identity gains while strictly locking candidate pose and expression.
As requested, we evaluate across all 497 test targets with a separate, unseen photograph of the same person for cross-photo scoring (out of the 640 test targets). In Tab. 1 of the paper, all six non-SimSwap baselines (FLUX.2, Qwen, WithAnyone Kontext, REFace, Face-Adapter, APPLE) already yield negative AdaFace (−0.028 to −0.229) and ArcFace (−0.012 to −0.200) gains even on the guiding reference; we therefore compare against SimSwap, the strongest baseline with positive gains on held-out recognizers. We measure expression error against the candidate and control for identity-related shape by mapping expression offsets to canonical coordinates under the candidate's shape, rotation, and scale.
When the proposal pre-stage pulled pose by 7.44° toward the guiding photo, scoring against a differently posed unseen photo depressed pose-sensitive AdaFace to +0.008. Initializing Alg. 1 directly from C cuts rotation drift $\sim$20$\times$ (to 0.38°) and expression drift $\sim$8$\times$ (to 0.00109), while achieving +0.029 FaceNet and +0.063 AdaFace on unseen photos, nearly doubling (1.85×) SimSwap's +0.034 AdaFace gain while also exceeding its FaceNet gain (+0.029 vs. +0.025) and maintaining positive gains on all three recognizers under the original reference-scoring protocol. At comparable operating points (the 69 targets where both methods raise both unseen-photo scores and match within 0.01), our expression error is lower than SimSwap's by 0.00057 [0.00030, 0.00089].