acceptodds
Withdrawn from ICLR 2027

HeadsUp! One-Step High-Fidelity Portrait Image Super-Resolution

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Portrait pictures, which typically feature both human subjects and natural backgrounds, are one of the most prevalent forms of photography on social media. Existing image super-resolution (SR) techniques generally focus either on generic real-world images or strictly aligned facial images (i.e., face super-resolution). In practice, separate models are blended to handle portrait photos: the face specialist model handles the face region, and the general model processes the rest. However, these blending approaches inevitably introduce blending or boundary artifacts around the facial regions due to different model training recipes, while human perception is particularly sensitive to facial fidelity. To overcome these limitations, we study the portrait image super-resolution (PortraitSR) problem, and propose HeadsUp, a single-step diffusion model capable of restoring and upscaling portrait images in an end-to-end manner. Specifically, we build a dual-branch diffusion model that processes the face region separately from the whole image, following a pyramidal feature-fusion strategy. We also develop a face supervision mechanism to guide the model in focusing on the facial region. We then integrate a reference-based mechanism to help with identity restoration, reducing face ambiguity in low-quality face restoration. Additionally, we have built a high-quality 4K portrait image super-resolution dataset dubbed PortraitSR-4K, to support model training and benchmarking for portrait images. Extensive experiments show that HeadsUp achieves state-of-the-art performance on the PortraitSR task.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Portrait Image Super-resolution

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_UQdDOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper introduces PortraitSR, a task that super-resolves portrait images containing both a human subject and natural background. The authors argue that existing general image SR models produce low-fidelity faces, while face SR specialists require blending with a general model, which introduces boundary artifacts. To address this, they propose HeadsUp, a one-step diffusion model with three key components:

  1. Dual-branch UNet: a face branch operating on affine-aligned face regions and a portrait branch processing the whole image, fused via pyramidal back-projection and additive feature merging.
  2. Face-aware supervision: a face fidelity loss (MSE+LPIPS on the aligned face) and an identity loss based on cosine similarity of face recognition features, optionally weighted by a reference face image.
  3. PortraitSR-4K dataset: 30K 4K portrait images with triplets of (LQ portrait, HQ portrait, reference face) constructed via identity similarity thresholding.

Experiments on PortraitSR-4K-Te show improvements over OSEDiff, PiSA-SR, and two blending baselines across most metrics, with additional generalization experiments on general SR and face SR.

Critical strengths
  1. Well-motivated problem formulation.​ The mismatch between general SR and face SR specialists—and the resulting blending/boundary artifacts—is a genuinely practical limitation. The formulation of PortraitSR as an end-to-end task is reasonable and relevant.
  2. Valuable dataset contribution.​ PortraitSR-4K with paired HQ/LQ/reference triplets and identity-based reference construction is a potentially useful resource for the community, assuming the ethical and reproducibility concerns are addressed.
  3. Clean and practical architecture.​ Initializing from a pretrained one-step diffusion model with LoRA adaptation, and adding a parallel face branch with affine projection/back-projection, is simple, efficient, and well described.
  4. Reasonably comprehensive evaluation.​ Multiple fidelity metrics, no-reference metrics, identity scores, user studies (WRid, WRN), and generalization to face SR and general SR demonstrate breadth beyond the main task.
Critical weaknesses
  1. Incremental technical novelty.​ The core building blocks—one-step diffusion SR (OSEDiff), aligned face branch, identity loss with pretrained recognition features, reference-based face restoration, and LoRA fine-tuning—are all borrowed from prior work. The main contribution appears to be the integration itself. The authors should clearly articulate what is fundamentally new beyond combining existing components, particularly at the representation or optimization level.
  2. Unfair comparison setting regarding reference information.​ At inference, HeadsUp is provided with a reference face image of the same identity, while the general SR baselines (OSEDiff, PiSA-SR) and blending baselines do not have access to such information. The improvement in ID-score and user-study win rates may largely reflect this extra conditioning rather than architectural superiority. A reference-free variant of HeadsUp should be the primary comparison; or baselines should also be given reference information where applicable.
  3. Weak ablation and marginal quantitative gains.​ The ablation table (Table 2) is difficult to interpret and appears partially mislabeled. The full model's PSNR of 25.70 is within 0.05 dB of the no-face-loss baseline (25.65), and using only the identity loss yields the highest PSNR (25.85) yet is deemed worse due to blur. The authors report no variance across seeds or statistical significance. These differences are far too small to support claims of "significant" improvement.
  4. Reference contribution is stated to be small.​ The paper admits that adding the reference "slightly improves identity score but reduces image quality by a small margin." If the reference-based mechanism is a major claimed contribution, the evidence for it is weak. Sensitivity to reference quality (e.g., different identities, pose, lighting) is not analyzed.
  5. Dataset concerns.​ The dataset is scraped from public web collections, and the authors state that only labels and URLs will be released. This creates reproducibility risks (URL rot) and unresolved privacy/consent issues for face data. Additionally, no demographic diversity analysis or bias discussion is provided, despite the known biases in web-scraped face datasets. The test-set identity matching protocol (in-set reference construction) also needs clarification regarding potential identity leakage between training and test subsets.
  6. Insufficient comparison with strong recent baselines.​ The blending baseline OSEDiff+OSEDiff_Face is weak; a more competitive comparison would include strong face restoration models (e.g., CodeFormer combined with a better general SR model, VQFR, DR2, PG-Diff, InstantRestore). No diffusion-based general SR baselines such as DiffBIR or StableSR are compared, and the one-step diffusion competitor SinSR is cited but not evaluated.
  7. Missing efficiency analysis.​ The motivation includes practical deployment, yet the paper reports no model size, inference latency, FLOPs, or GPU memory compared with baselines. Given the dual-branch design and the reference/mask inputs, the computational overhead should be demonstrated to be modest.
  8. Limited failure-mode analysis.​ How does the method handle multiple faces, very small faces, extreme poses, strong occlusion, or cases where no reference is available? None of these scenarios are examined. Qualitative results appear cherry-picked.
  9. Poor manuscript quality in places.​ The conclusion contains duplicated and garbled sentences; Table 1 is misformatted; several notation issues and missing values appear throughout.
Questions for authors
  1. The main comparison gives HeadsUp access to a reference face of the same identity while baselines do not. Could you report results for HeadsUp in the reference-free setting (x_r = 0) on the full test set, and discuss how much of the reported improvement is attributable to the reference information?
  2. Why was the OSEDiff+OSEDiff_Face blending chosen as the only face-inclusive baseline? Would a stronger combination, e.g., Real-ESRGAN + CodeFormer with masked blending or more recent face SR models, change the conclusions?
  3. In Table 2, the setup labels for each row are unclear. Specifically, which row is the "no face loss" baseline, and which is the full model? The PSNR differences between configurations are within 0.2 dB—are these differences stable across random seeds? Please provide mean ± std over at least three runs.
  4. The identity loss applied alone produces the highest PSNR but is described as causing blur. Could you analyze why this happens and how the λ_fid / λ_id / λ_LPIPS balance was chosen? Provide sensitivity analysis over these weights.
  5. How is the reference mask M obtained at test time? Is the mask derived from the face detector or the affine projection? What happens when the face detection fails or the mask is inaccurate?
  6. How was the identity threshold γ = 0.65 chosen for pair construction? Is there any identity overlap between PortraitSR-4K-Tr and PortraitSR-4K-Te? Can a face in the test set have its reference pair drawn from an image whose identity also appears in training?
  7. What is the computational cost of HeadsUp in terms of parameters, inference time, and memory, compared with OSEDiff and PiSA-SR?
  8. The paper claims the dual-branch design avoids blending artifacts. Could you provide a quantitative boundary-artifact metric, or at least a visualization of feature differences at the face–background boundary?
  9. The reference face appears to provide only marginal gains. Have you considered making the reference mechanism a conditioning pathway that can be dropped more gracefully (e.g., via feature dropout or a learnable gate), rather than simple concatenation?
  10. How does HeadsUp perform on portraits with multiple faces? Does the single binary mask mechanism scale to multiple face regions, and if not, what is the planned extension?
Reviewer_6EN8OpenReview ↗
Rating
2
Confidence
4
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

HeadsUp is a one-step diffusion framework for portrait image super-resolution that jointly restores facial details and the surrounding image. It combines portrait and aligned-face branches through multi-scale feature fusion, reducing boundary artifacts associated with separately restoring and blending faces. Face-specific fidelity and identity losses, together with an optional reference face, improve identity preservation. The authors also introduce PortraitSR-4K, a dataset of 30,000 high-resolution portrait images. Experiments demonstrate improvements over general SR and blending-based baselines across most evaluated metrics and human preferences.

Critical strengths
  1. Useful dataset contribution: PortraitSR-4K provides 30,000 high-resolution portrait images for training and evaluation.
  2. Practical motivation: Addresses the difficulty of restoring faces and backgrounds consistently in portrait super-resolution.
Critical weaknesses
  1. Insufficient justification for the limited methodological novelty. A central motivation is to avoid boundary artifacts introduced by restoring faces separately and blending them back into the image. However, the paper does not establish whether these artifacts require the proposed dual-branch diffusion architecture or can be substantially mitigated by simpler compositing strategies, such as Gaussian masks or color matching. Controlled comparisons using the same face and background restoration outputs would help isolate the limitations of blending itself. Moreover, the architecture combines familiar components. The face-aware losses also have clear precedents: Focus and Retain [1] applies pixel and perceptual losses to facial regions extracted from generated full images, while Liquid Warping GAN [2] applies an identity loss to faces cropped from synthesized human images.
  2. Insufficient and unequal baseline comparisons. The main PortraitSR evaluation should include strong general restoration methods such as SUPIR [3] and DiffBIR [4] on the same full-image portrait test set. Although DiffBIR appears in the supplementary general SR experiments, those results do not establish its performance on the target task. The blending baselines should also incorporate recent face restoration methods, such as OSDFace [5], combined with a strong background restoration model.

[1] Ge et al. “Focus and Retain: Complement the Broken Pose in Human Image Synthesis.” WACV, 2021. Section 3.4, Equation (8).

[2] Liu et al. “Liquid Warping GAN: A Unified Framework for Human Motion Imitation, Appearance Transfer and Novel View Synthesis.” ICCV, 2019. Section 3.4.

[3] Yu et al. “Scaling Up to Excellence: Practicing Model Scaling for Photo-Realistic Image Restoration In the Wild.” CVPR, 2024.

[4] Lin et al. “DiffBIR: Towards Blind Image Restoration with Generative Diffusion Prior.” ECCV, 2024.

[5] Wang et al. “OSDFace: One-Step Diffusion Model for Face Restoration.” CVPR, 2025.

Questions for authors
  1. Clarification on test-time reference selection. Section 3 describes constructing reference pairs within the test split using a CVLFace similarity threshold of 0.65. For the results in Table 1, how is the reference selected when multiple candidates satisfy this threshold, and are the target image itself and near-duplicate images explicitly excluded?
  2. Handling multiple faces. The framework is described using a single face branch, affine transformation, and reference image. How does it handle portraits containing multiple faces? Are all faces restored through the face branch or only a selected face, and how are reference images associated with individual faces?
Reviewer_KSWYOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper focuses on the task of Portrait Image Super-Resolution (PortraitSR), which will restore or do super-resolution of the face area or the surrounding environments of a Portrait image. The existing methods usually use the general SR models which produces the faces with lower fidelity and usually is less natural. The existing hybrid blending methods, such as using a general SR model with a face alignment model, will introduce some boundary artifacts which makes the face looking unnatural. The author proposes the HeadsUp model which is an end-to-end and one-step latent diffusion model.

The contributions:

  1. The author proposes a one-step UNet with (1) a Portrait Branch and (2) a Face Branch.
  2. The proposed method uses a face-aware supervision & reference guidence.
  3. The author publishes a PortraitSR-4K Dataset containing 30,000 high-resolution portrait images
Critical strengths

(1) The architecture and framework are technically sound and the problem formulation is clear and practical. This paper formulates a well-known limitation in the real-world image restoration. The difficulty is the boundary seams caused by the procedure of "crop", "restore", and "paste" when combining face specialists with general SR models.

(2) The PortraitSR-4K Benchmark dataset. The creating of this dataset containing 30K high (4K) resolution portrait images provides a valuable benchmark for training and evaluating joint face-and-background restoration models.

Critical weaknesses

(1) Inconclusive Ablation Study: (i) In Table 2, adding the reference image (Row 5 vs. Row 4 in Table 2) yields only a marginal improvement (e.g., PSNR is identical at 25.70, FID goes from 101.55 to 101.29, and ID-Score moves from 0.3595 to 0.3721), which weakly justifies the added complexity of reference-pair curation, an 8/9-channel conv-in modification, and mask concatenation. (ii) There is no ablation study isolating the impact of the Dual-Branch UNet architecture itself. It is unclear how much performance gain comes from the dual-branch latent feature projection (Ω\Omega / Ω−1\Omega^{-1}) versus simply fine-tuning a single-branch OSEDiff model on PortraitSR-4K using the proposed pixel-level face-aware loss LF\mathcal{L}_{\mathcal{F}}.

(2) Potential Identity Leakage in Reference Pair Construction & Real-World Applicability: In Section 3, reference pairs in PortraitSR-4K are constructed by running a face recognition model (CVLFace) across all cropped faces in the dataset and pairing images with cosine similarity φ(x,y)>0.65\varphi(x, y) > 0.65. Because web-scraped datasets (LAION, PD12M) contain unrelated individuals, pairing arbitrary images solely by embedding threshold risks pairing different people who happen to look similar (or duplicate uploads of the exact same photo). Moreover, during inference on in-the-wild images, users rarely have an aligned HQ reference face of the subject on hand, yet Table 1 does not specify whether "Ours" is evaluated with or without reference images.

Questions for authors

(1) Architecture vs. Loss Ablation: What is the performance of a single-branch baseline (i.e., standard OSEDiff architecture without the Face Branch ϵF\epsilon_F) trained on the exact same dataset mixture with your proposed Face Fidelity and Face Identity losses (LF\mathcal{L}_{\mathcal{F}})? This is critical to verify whether the dual-branch feature projection is necessary or if region-weighted supervision alone suffices.

(2) Handling Multiple Faces or Detection Failures: How does the affine projection Ω\Omega and the dual-branch UNet behave when a portrait contains multiple faces (e.g., group photos), heavily occluded/profile faces where landmark detection fails, or zero faces (such as in the DIV2K/RealSR generalization experiments in Table 3)?

(3) Latent Space Affine Projection Resolution: In Section 4.4, Ω\Omega is applied directly to the LQ latent zL∈Rc×(h/8)×(w/8)z_L \in \mathbb{R}^{c \times (h/8) \times (w/8)}. If an input image is 256×256256 \times 256, zLz_L is 32×3232 \times 32, meaning a small face occupying 20% of the image is only ∼6×6\sim 6 \times 6 pixels in latent space. What is the spatial resolution of the aligned latent Ω(zL)\Omega(z_L) fed into the Face Branch ϵF\epsilon_F, and does interpolating such a coarse latent grid cause severe aliasing before ϵF\epsilon_F processes it?

(4) Potential Typo or formatting issue: (i) in Table 2, it seems the table only has 4 rows, why the caption mentions fifth row? In figure 7, do you mean "\mathcal{L}{P}^{fid}" instead of \mathcal{L}{P}^{id}?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.