- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
-
A single SD-1.5-based model for face, hair, motion, and head transfer, with adaptation to low-level tasks (eye, lip, glasses, hat).
-
Data: swapping-based corruption, where spatially-dynamic attributes are corrupted with an external generative model.
-
Model: BioMoE, consisting of a global expert, 8 routed experts selected by a structure-aware router, and task-specific LoRA experts.
-
Training: Task-Isolated Initialization per task, then averaging the FFNs and fitting the residuals via SVD before Task-Unified Fine-tuning.
-
Evaluated on 987 FFHQ pairs against task-specific, multi-task, and unified baselines, with per-component ablations.
Critical strengths
-
The swapping-based corruption strategy is the most convincing part of the paper. The problem statement is clear (a mask alone leaks the target silhouette), and Table 5 (hair 0.492→0.421, head 0.555→0.460) and Table 6 (0.424 vs 0.421 when the external generator is swapped) support it.
-
Handling four tasks from one checkpoint and adapting to low-level tasks (Table 3) has practical value. The gap in Table 4 over naive parameter sharing (0.478) and one-stage training (0.507) versus the full model (0.637) is substantial.
Critical weaknesses
-
The claim that the unified model outperforms task-specific models is not well supported. In Table 4 the difference between the task-specific row and the full model is small (e.g., 0.622 vs 0.637), and the effect of structure-aware routing is at the level of 0.634→0.637, with no variance reported. “Comparable performance from a single model” would be a sufficient claim; the paper states more than that.
-
The comparison conditions are mixed. Some competing methods perform better on individual metrics (CanonSwap on face-transfer pose distance, 2.92 vs 3.63; Face-Adapter on motion-transfer pose distance, 6.89 vs 7.09), REFace for head transfer was retrained by the authors (A.5), and the hair/head training data is constructed with StableHair, which is then used as a baseline (B.1.2).
Questions for authors
Q1) A.5 acknowledges that the original REFace paper supports both face and head transfer, but Table 1 classifies it under “training separately for different tasks.” Does this distinction concern the method’s capability or the publicly available implementation, and did you try the released REFace checkpoint on head transfer?
Q2) What happens with only the global expert and task-specific LoRA experts, without routed experts? This combination is not in Table 4. Could you also report variance over repeated runs for Table 4?
Q3) In Stage I, how are the components other than the FFN (attention, refNet, task-specific projection MLPs) trained and carried into Stage II? The text only describes averaging the FFNs. Please also check the reference to “Appendix 3.3” in Sec. 3.3.
We sincerely thank the reviewer for the careful reading and constructive suggestions. We have revised the manuscript accordingly and respond below.
W1. Comparison with Task-Specific Methods
CanonSwap’s lower pose distance should be considered together with its identity-transfer fidelity. As illustrated in Figures 4(b) and 14, CanonSwap exhibits a more conservative identity transfer, with its outputs retaining more characteristics of the target face. Retaining the original face can yield a low pose distance even when the intended identity transfer is incomplete. This is consistent with its lower ID similarity in Table 2 (0.551 versus our 0.637). The visual comparisons and user study further support our overall face-transfer quality: Figure 10(a) reports a preference rate of 64.6% for
Oursversus 19.6% for CanonSwap.Face-Adapter achieves a slightly lower motion-transfer pose distance (6.89 versus 7.09), while
Oursachieves higher identity similarity (0.602 versus 0.503) and a lower expression distance (2.09 versus 2.27). These results support a better balance of identity preservation and expression transfer while maintaining competitive pose accuracy.We have refined our claims to emphasize the overall benefits of task consolidation while acknowledging the trade-offs on individual metrics.
W2. Comparison Conditions
(a). REFace for head transfer
The original REFace method supports face and head transfer through separate retraining, resulting in two task-specific models. As no head-transfer checkpoint is publicly released, we retrain it by expanding the masking region to include hair and accessories, using its official hyperparameters.
(b). StableHair for data construction
StableHair is used to corrupt the training inputs, while the supervision target remains the original real image. Our model therefore learns to reconstruct real images rather than imitate StableHair’s outputs. Furthermore, our models trained using corruptions generated by StableHair and HairCLIPv2 achieve comparable performance, supporting low dependence on the choice of external corruption generator (Table 6).
Q2. BioMoE Ablation and Multi-seed Results
We repeat the model-design ablations in Table 4 and add two variants:
w/o Task-specific Expertsandw/o Routed Experts. Results are aggregated over 12 training–inference seed combinations (see Appendix B.3 for details).Each cell reports mean ± STD, followed by the gain of
Oursover that variant with an unpaired Welch 95% CI. Positive gains favorOurs.Overall, the full model achieves the best mean scores across all four tasks, with predominantly positive gain CIs supporting the effectiveness of each design choice.
Q3. Training and Initialization Across Stages
The refNet is a reference network that follows the same parameter-handling strategy as the main denoising network. In both networks, FFNs undergo the averaging and residual-based initialization described in Section 3.3. Attention projections and the projection MLPs following the CLIP encoder remain task-specific and retain their Stage-I weights in Stage II, introducing only a small parameter overhead because these projections are much smaller than FFNs (see Appendix A.1).
We have clarified these details and corrected “Appendix 3.3” to “Section 3.3” in the revised paper. Thanks for your valuable reviews!