acceptodds
Withdrawn from ICLR 2027

FaceKeeper: Multi-Expert LoRA for High-Fidelity Portrait Retouching

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Portrait retouching must balance blemish removal, texture preservation, and identity consistency. Existing learning or diffusion methods often struggle to balance these three objectives: high-frequency details are lost, skin is overly smoothed, or multiple conflicting objectives are compressed into a single LoRA, leading to trade-offs and degradation. This paper proposes FaceKeeper, a multi-expert LoRA framework that internalizes these objectives into a unified training process and gradually resolves conflicts across three dimensions: In terms of supervision, we propose a semantically guided spatial-high-frequency coupled detail oversampling strategy that focuses on salient local regions; at the parameter level, we adopt a three-stage adaptation process:freezing, dense training, and sparse refinement,where the shared projection A is frozen, and the intent-specific factor B is merged into a lightweight adapter via mask-aware integration after dense training and sparse refinement; at the semantic level, we bridge the gap between pixel restoration and texture perception through multi-expert reward pre-alignment. Based on FLUX.2-klein-base-9B, our method outperforms dedicated retouching models and general-purpose editors on the self-built FaceKeeper-Bench and CelebA-HQ benchmarks for reference-free metrics, while remaining competitive for reference-based metrics. User studies further validate its excellent balance between blemish removal and the preservation of texture and identity.

Primary area:
generative models
Keywords:
Portrait Retouching, Multi-Expert LoRA, Diffusion Models, Identity Preservation, Texture Preservation, Reward Learning

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_VweNOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

This paper aims to address the trade-offs between blemish removal, texture preservation, and identity consistency in facial retouching. It combines saliency-aware patch sampling, a multi-expert LoRA and reward-weighted flow-matching. Experiments on FaceKeeper-Bench and a CelebA-HQ show visually strong results.

Critical strengths
  1. In section 3.2, It is very interesting that merging experts is not element-wise but row-wise by using SLERP. The explanation is also interesting that the direciton of a scalar can only be positive or negative while the direction of a vector is much more geometrically informative.
  2. The visual results are good.
Critical weaknesses
  1. Writing is a bit awful. Relatively simple operations are often described using unusual or insufficiently justified terminology. For example, in line 216, it says "we no longer process the entire graph" but it actually means "we no longer process the entire image". Why use "graph" at all when the whole paper is talking about image? In line 214, it says "random or fixed windows" but it actually means "random or fixed crops during training.". The term "Gaussian semantic kernel" actually measures Gaussian similarity in a learned positional embedding space. Why it is semantic at all? It is OK to use fancy words, but it is not OK to allow the terminology to obscure rather than clarify the underlying method.
  2. Related to above, although the description is lengthy, several basic details are missing. For example, in section 3.1, how large is the patch? Is the training only on patch? How to balance patch loss and whole image loss? Where is the saliency computed, source or GT? If there is no enough space, then don't waste the space on the fancy words and complex description.
  3. The ablation test is insufficient. For the example in section 3.1 again, the proposed method essentially combines a local high-gradient saliency score with a mechanism that encourages spatially diverse coverage. Why not just use NMS on high-gradient patches and take top-K or sample K from top-N? In section 3.2, why use global threshold rather than per-expert threshold? What's the insight of it? If the number of experts is large, won't it cut some experts entirely? Is there any trade-off? I know it is mentioned in limitation, but there is no analysis at all. In section 3.2, matrix A is described as "A carries general-purpose directions independent of intent”, but A is actually randomly-initialized and frozon during training. Why it carries general-purpose directions independent of intent? Is it just another fancy sentence or it really means much more than "A provides a shared low-dimensional basis"?
  4. Some claims are not supported by method theory or experiments. For example, in section 3.2, it says "Following sparsification, the intersection of the support sets of different experts in parameter space is minimized". Even if each expert is sparse, there is no way to discourage them from overlap at random entries. For example, if all experts are [1, 0, 0, 0], their sparsity is 75% but their overlapping is still 100%.
  5. Some important formulas are missing. In section 3.4, there are 6 different losses, but only one of them is defined. They should be at least defined in Appendix.
  6. The description of data is not sufficient. For example, in FaceKeeper-Bench, where does the GT come from (since it reports PSNR and LPIPS)? What makes me more confused is CelebA-HQ, what is the reference? If the blemish is synthesized, there should be construction pipeline in the paper.
  7. There are many baselines in the table, but I am not sure if they are fine-tuned on the same data? I guess not since it contains GPT Image and Nano Banana. Then it is unclear whether the performance comes from model design or data. Especially when we look at the ablation table, it seems that LoRA itself gives the majority of the improvement.
  8. User study, 5 evaluators labelling 100 images, seems too small-scale.
Questions for authors

See weakness 2-7.

Reviewer_kAgDOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
2
Read review
Paper summary

FaceKeeper adapts a portrait editor using detail-focused patch sampling, sparse multi-expert LoRA merging, and reward-guided alignment. It aims to remove blemishes while preserving texture and identity, and reports comparisons on FaceKeeper-Bench and CelebA-HQ plus a small user study.

Critical strengths

The task is useful, and the evaluation includes dedicated retouching methods as well as general image editors. Merging the experts into a single adapter is also practically attractive.

Critical weaknesses
  1. The added complexity is not convincingly justified. Frozen-A LoRA, magnitude pruning, and adapter merging are familiar components, but Table 2 does not isolate the proposed expert-merging strategy against simpler alternatives. The complete model is worse than plain +LoRA in LPIPS, NIMA, NRQM, and TOPIQ; adding reward also worsens FID from 4.0408 to 4.8054. This weakens the evidence for the full design.

  2. The central claim of preserving original facial detail is not sufficiently established. Generic image-quality scores cannot distinguish faithful pores from smoothing or synthesized texture, and no dedicated identity-similarity metric is reported. ABPN has better LPIPS and FID in Tables 1 and 3, while the five-evaluator study provides no uncertainty estimates. Figure 2 also contains partly obscured text below the left source image, adding to presentation concerns. These results do not convincingly demonstrate a substantial advance over simpler retouching.

Questions for authors
  1. What are the results for a single LoRA and simple expert averaging under matched data and training budgets?
  2. Can you provide direct identity/texture-preservation evidence and uncertainty for the user study, and explain the regressions in Table 2? Please also correct the obscured text in Figure 2.
Reviewer_YbNVOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

To address the triple challenge of flaw removal, texture preservation, and identity consistency in portrait image editing, this paper proposes the FaceKeeper solution, which builds a multi-expert LoRA framework based on FLUX.2-klein-base-9B. The method tackles these challenges at three levels: spatial-frequency coupled sampling to focus on salient regions, a Freeze-Dense-Sparse multi-expert LoRA with SLERP-Tree merging to reduce conflicts, and a multi-expert CLIP reward alignment mechanism. FaceKeeper achieves best on NIMA, MANIQA, NRQM, TOPIQ on FaceKeeper-Bench. Qualitative comparisons against seven baselines show effective blemish removal while preserving pores, luster, and identity markers, whereas baselines leave artifacts, oversmooth, or hallucinate pseudo-textures.

Critical strengths

The freeze-dense-sparse multi-expert LoRA architecture addresses gradient-level interference in multi-intent adaptation, a real but under-explored problem. Its parameter-support decoupling via amplitude masks and SLERP-Tree merging is principled.

Critical weaknesses
  1. Regarding Table 2, the "+LoRA" variant outperforms both "+patch" and "+reward (ours)" on four metrics: LPIPS, NIMA, NRQM, and TOPIQ. The complete method achieves the best result only on MANIQA. This weakens the claimed necessity and novelty of the full three-module design.
  2. In the quantitative comparison on the CelebA-HQ dataset, the proposed method underperforms FLUX.2-klein-base-9B on MANIQA and TOPIQ, which undermines the generalization claim.
  3. The paper claims that freezing A avoids relearning each channel from scratch, but A is randomly initialized (Appendix D: "randomly initialized LoRA down-projection A"). Whether a randomly frozen A can provide "general representations" lacks experimental support. A more reasonable comparison would be random frozen A versus pretrained A.
  4. Common presentation and formatting errors can compromise readability and credibility.
Questions for authors

In Table 2, the "+LoRA" variant achieves the best scores on the most metrics (LPIPS 0.0512, NIMA 5.2581, NRQM 6.9622, TOPIQ 0.6110), outperforming both "+patch" and the full "+reward (ours)" on these measures. Could the authors clarify why the complete method does not surpass the simpler "+LoRA" baseline on these metrics, and whether the full pipeline's advantage is primarily confined to MANIQA?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.