- Rating
- 4(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 3
Read review
Paper summary
This paper introduces a restoration-aware VAE mechanism in diffusion-based face restoration task, named RAEFace. Compared to current latent representation, RAEFace leverages information on degraded LQ images, and reaches better performance on simulated and real-world benchmarks.
Critical strengths
-
The novelty sounds good. To better achieve the face resotration task, RAEFace introduce latent alignment from REPA to give better guidance for diffusion process.
-
The experiments are convincing. RARFace achieve better performance on CelebA-Test, LFW-Test, WebPhoto-Test and Wider-Test.
Critical weaknesses
-
A detaied comparison is needed for related latent space alignment works, like FaithDiff[1] and other related similar works.
-
I have some concerns on results shown in Figure 9 (smiling woman at top-right) and Figure 11. (1) How could RAEFace know that the woman shows her tooth when smiling, as this area is inpainted? (2) Why in Figure 11, results provided by CodeFormer has wrong color?
-
The training target is not presented well in Method section. The authors may provide a detailed algorithm, showing how the loss is calculated, and if there are additional training stage for the VAE encoder.
Questions for authors
Please see "Critical Weaknesses" part for my questions for authors.
4 comments
W2: Interpretation of inpainting and colorization results
I have some concerns on results shown in Figure 9 (smiling woman at top-right) and Figure 11. (1) How could RAEFace know that the woman shows her tooth when smiling, as this area is inpainted? (2) Why in Figure 11, results provided by CodeFormer has wrong color?
1) Inpainting: inferring missing details from informative facial attributes. RAEFace cannot determine with certainty whether teeth are present in the masked region. The teeth in Fig. 9 are a plausible completion conditioned on the visible facial context and learned priors, rather than guaranteed recovery of the hidden ground truth. Our masked-face linear probes provide supporting evidence that useful facial attributes remain accessible under occlusion:
| Train set | Test set | DINOv3-L | SigLIP2-L | MAE-L | SD-VAE |
|---|---|---|---|---|---|
| HQ | HQ | 85.22% | 86.87% | 78.90% | 49.97% |
| Rectangle-masked | Rectangle-masked | 81.03% | 80.86% | 74.59% | 49.61% |
| HQ | Rectangle-masked | 77.60% | 71.12% | 65.39% | 47.15% |
| Brush-masked | Brush-masked | 81.18% | 81.13% | 74.14% | 49.84% |
| HQ | Brush-masked | 76.53% | 71.95% | 66.07% | 47.10% |
HQ denotes unmasked input; Mask denotes inputs with the specific mask.
These results show that facial attributes remain more accessible in the semantic encoders (DINOv3, SigLIP and MAE) than in SD-VAE under both mask types. They support context-informed completion, but do not establish the correctness of a particular hidden detail.
2) Colorization: color ambiguity and preservation of visible details. A color difference from the reference alone does not necessarily indicate an error, because grayscale inputs admit multiple plausible colorizations. Our objective is to generate natural colors while preserving the facial structure and details visible in the input. The concern with CodeFormer in Fig. 11 of the original submission (Fig. 15 in the revised manuscript) is that it also modifies visible facial details, going beyond color assignment and conflicting with this preservation goal. Appendix F.1 now clarifies the color ambiguity and the importance of preserving visible facial structure and details.
W3: Training objectives, loss computation, and encoder training
The training target is not presented well in Method section. The authors may provide a detailed algorithm, showing how the loss is calculated, and if there are additional training stage for the VAE encoder.
We have included a detailed training algorithm in the revised manuscript, clarifying the training targets, loss computation, and which components are trained or frozen (Sec. 3.2 and Appendix A, Algorithms 1–2).
Thanks for the authors' feedback. I have no more concerns and will raise my score to 4.
Thank you very much for your kind words and for revising the score. We are greatly encouraged by your positive feedback and sincerely appreciate your time and guidance throughout the review process.
We thank Reviewer Hotu for the positive assessment and insightful feedback. We have added experiments and analyses, revised the manuscript, and provide point-by-point responses below. We welcome further questions and discussion.
W1: Comparison with FaithDiff
We thank the reviewer for highlighting FaithDiff and related work. We have added an end-to-end BFR comparison with FaithDiff using the same test inputs and evaluation metrics. RAEFace consistently outperforms FaithDiff on all four no-reference quality metrics—CLIPIQA, MUSIQ, MANIQA, and TOPIQ—across LFW, WebPhoto, Wider, and CelebA.
At the level of latent alignment, FaithDiff’s alignment module and our REPA objective serve different roles. FaithDiff uses a learned module to align and integrate LQ encoder features with the current noisy latent to construct denoising inputs. In RAEFace, REPA, adopted from RAEv2, instead supervises intermediate DDT features using paired HQ encoder representations to regularize their spatial token-to-token similarity structure.
LFW
WebPhoto
Wider
CelebA