- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper introduces PortraitSR, a task that super-resolves portrait images containing both a human subject and natural background. The authors argue that existing general image SR models produce low-fidelity faces, while face SR specialists require blending with a general model, which introduces boundary artifacts. To address this, they propose HeadsUp, a one-step diffusion model with three key components:
- Dual-branch UNet: a face branch operating on affine-aligned face regions and a portrait branch processing the whole image, fused via pyramidal back-projection and additive feature merging.
- Face-aware supervision: a face fidelity loss (MSE+LPIPS on the aligned face) and an identity loss based on cosine similarity of face recognition features, optionally weighted by a reference face image.
- PortraitSR-4K dataset: 30K 4K portrait images with triplets of (LQ portrait, HQ portrait, reference face) constructed via identity similarity thresholding.
Experiments on PortraitSR-4K-Te show improvements over OSEDiff, PiSA-SR, and two blending baselines across most metrics, with additional generalization experiments on general SR and face SR.
Critical strengths
- Well-motivated problem formulation. The mismatch between general SR and face SR specialists—and the resulting blending/boundary artifacts—is a genuinely practical limitation. The formulation of PortraitSR as an end-to-end task is reasonable and relevant.
- Valuable dataset contribution. PortraitSR-4K with paired HQ/LQ/reference triplets and identity-based reference construction is a potentially useful resource for the community, assuming the ethical and reproducibility concerns are addressed.
- Clean and practical architecture. Initializing from a pretrained one-step diffusion model with LoRA adaptation, and adding a parallel face branch with affine projection/back-projection, is simple, efficient, and well described.
- Reasonably comprehensive evaluation. Multiple fidelity metrics, no-reference metrics, identity scores, user studies (WRid, WRN), and generalization to face SR and general SR demonstrate breadth beyond the main task.
Critical weaknesses
- Incremental technical novelty. The core building blocks—one-step diffusion SR (OSEDiff), aligned face branch, identity loss with pretrained recognition features, reference-based face restoration, and LoRA fine-tuning—are all borrowed from prior work. The main contribution appears to be the integration itself. The authors should clearly articulate what is fundamentally new beyond combining existing components, particularly at the representation or optimization level.
- Unfair comparison setting regarding reference information. At inference, HeadsUp is provided with a reference face image of the same identity, while the general SR baselines (OSEDiff, PiSA-SR) and blending baselines do not have access to such information. The improvement in ID-score and user-study win rates may largely reflect this extra conditioning rather than architectural superiority. A reference-free variant of HeadsUp should be the primary comparison; or baselines should also be given reference information where applicable.
- Weak ablation and marginal quantitative gains. The ablation table (Table 2) is difficult to interpret and appears partially mislabeled. The full model's PSNR of 25.70 is within 0.05 dB of the no-face-loss baseline (25.65), and using only the identity loss yields the highest PSNR (25.85) yet is deemed worse due to blur. The authors report no variance across seeds or statistical significance. These differences are far too small to support claims of "significant" improvement.
- Reference contribution is stated to be small. The paper admits that adding the reference "slightly improves identity score but reduces image quality by a small margin." If the reference-based mechanism is a major claimed contribution, the evidence for it is weak. Sensitivity to reference quality (e.g., different identities, pose, lighting) is not analyzed.
- Dataset concerns. The dataset is scraped from public web collections, and the authors state that only labels and URLs will be released. This creates reproducibility risks (URL rot) and unresolved privacy/consent issues for face data. Additionally, no demographic diversity analysis or bias discussion is provided, despite the known biases in web-scraped face datasets. The test-set identity matching protocol (in-set reference construction) also needs clarification regarding potential identity leakage between training and test subsets.
- Insufficient comparison with strong recent baselines. The blending baseline OSEDiff+OSEDiff_Face is weak; a more competitive comparison would include strong face restoration models (e.g., CodeFormer combined with a better general SR model, VQFR, DR2, PG-Diff, InstantRestore). No diffusion-based general SR baselines such as DiffBIR or StableSR are compared, and the one-step diffusion competitor SinSR is cited but not evaluated.
- Missing efficiency analysis. The motivation includes practical deployment, yet the paper reports no model size, inference latency, FLOPs, or GPU memory compared with baselines. Given the dual-branch design and the reference/mask inputs, the computational overhead should be demonstrated to be modest.
- Limited failure-mode analysis. How does the method handle multiple faces, very small faces, extreme poses, strong occlusion, or cases where no reference is available? None of these scenarios are examined. Qualitative results appear cherry-picked.
- Poor manuscript quality in places. The conclusion contains duplicated and garbled sentences; Table 1 is misformatted; several notation issues and missing values appear throughout.
Questions for authors
- The main comparison gives HeadsUp access to a reference face of the same identity while baselines do not. Could you report results for HeadsUp in the reference-free setting (x_r = 0) on the full test set, and discuss how much of the reported improvement is attributable to the reference information?
- Why was the OSEDiff+OSEDiff_Face blending chosen as the only face-inclusive baseline? Would a stronger combination, e.g., Real-ESRGAN + CodeFormer with masked blending or more recent face SR models, change the conclusions?
- In Table 2, the setup labels for each row are unclear. Specifically, which row is the "no face loss" baseline, and which is the full model? The PSNR differences between configurations are within 0.2 dB—are these differences stable across random seeds? Please provide mean ± std over at least three runs.
- The identity loss applied alone produces the highest PSNR but is described as causing blur. Could you analyze why this happens and how the λ_fid / λ_id / λ_LPIPS balance was chosen? Provide sensitivity analysis over these weights.
- How is the reference mask M obtained at test time? Is the mask derived from the face detector or the affine projection? What happens when the face detection fails or the mask is inaccurate?
- How was the identity threshold γ = 0.65 chosen for pair construction? Is there any identity overlap between PortraitSR-4K-Tr and PortraitSR-4K-Te? Can a face in the test set have its reference pair drawn from an image whose identity also appears in training?
- What is the computational cost of HeadsUp in terms of parameters, inference time, and memory, compared with OSEDiff and PiSA-SR?
- The paper claims the dual-branch design avoids blending artifacts. Could you provide a quantitative boundary-artifact metric, or at least a visualization of feature differences at the face–background boundary?
- The reference face appears to provide only marginal gains. Have you considered making the reference mechanism a conditioning pathway that can be dropped more gracefully (e.g., via feature dropout or a learnable gate), rather than simple concatenation?
- How does HeadsUp perform on portraits with multiple faces? Does the single binary mask mechanism scale to multiple face regions, and if not, what is the planned extension?