- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
This paper proposes PortraitGen, a framework for improving photorealistic portrait generation through Real-Image-Anchored GRPO. The method introduces real-image trajectories reconstructed via BELM inversion into GRPO groups, allowing them to participate in both relative advantage estimation and policy optimization. The authors further develop two complementary reward models targeting general visual quality and portrait-specific artifacts, and introduce PortraitBench for evaluation.
Critical strengths
- Meaningful and well-motivated approach: Incorporating real-image trajectories into GRPO provides a practical way to introduce external photographic references into generative post-training. The combination of real-image anchoring and portrait-specific reward modeling is intuitive and addresses relevant limitations of existing methods.
- Strong empirical results: PortraitGen demonstrates substantial improvements in portrait realism and artifact reduction over competitive baselines. The evaluation includes external realism metrics, qualitative comparisons, and a human preference study, providing reasonably convincing evidence of the method's effectiveness.
Critical weaknesses
-
In my understanding, the core method can be viewed as a combination of advantage-weighted SFT-like learning on real-image trajectories and anchor-augmented GRPO. Specifically, real-image trajectories provide direct supervision through their advantage-weighted likelihood objectives, while also serving as relative anchors that influence the advantage estimation of other generated samples. However, the paper does not sufficiently disentangle the contributions of these two mechanisms. Table 3 evaluates real-image anchoring as a whole, but does not clarify whether the improvements mainly arise from the additional real-image supervision or from the proposed group-relative anchoring mechanism. A comparison with standard GRPO combined with supervised learning on real-image trajectories would help clarify the distinct value of the proposed formulation.
-
The reliability of real-image anchoring depends on reward accuracy. Real-image exemplars are not assigned fixed positive rewards; their advantages are determined by the learned reward models. If a genuine photograph is incorrectly evaluated as exhibiting synthetic artifacts, its trajectory may receive a negative advantage, potentially discouraging the model from reproducing that real-image example. Since the exemplar also affects group-level reward normalization, such errors may influence the optimization of generated trajectories as well. While the paper reports strong reward-model accuracy, it does not analyze the distribution of exemplar advantages or the sensitivity of the proposed mechanism to erroneous reward assignments.
Questions for authors
Please refer to the Critical Weaknesses. I also noticed a minor issue regarding the references. Some reward models and evaluation components, such as OmniAID, Forensic-Chat, and Long-CLIP, are mentioned without direct citations to their original papers. I had to trace these references through other cited works, such as RealGen, which was somewhat inconvenient. I suggest adding direct citations to the original papers to make it easier for readers to locate and understand these components.
4 comments
Response to Reviewer Ptyr (2/2)
[W2] Distribution of real-image advantages and sensitivity to erroneous reward assignments
Thank you for this thoughtful comment. We have added an analysis of the exemplar advantage distribution in Section 5.6 (Fig. 8), and the statistics are summarized in the table below. For each training stage, we report the mean advantage of the real exemplar and the percentage of groups in which it receives a negative advantage. As a reference, we also report the percentage of generated samples with negative advantages, which indicates how often an ordinary sample falls below the group average.
-
In the early stage, when generated samples are still clearly distinguishable from real photographs, the exemplar receives a negative advantage in only 11.2% of the groups, whereas 55.7% of the generated samples fall below the group average. Its mean advantage also remains clearly positive (+1.43), indicating that the reward models seldom judge a genuine photograph as worse than the group average. In the remaining 11.2% of groups, a negative exemplar advantage implies that some generated samples in the same group are rated higher than the real image. This can happen because even FLUX.1-dev occasionally rolls out highly realistic portraits. These rollouts then receive positive advantages and are reinforced, so the optimization still moves toward realism. When the model can already produce comparably realistic samples by itself, it does not need to fit a specific real photograph, and learning from its own high-quality rollouts provides an on-policy signal that is easier to absorb. Since such groups are a small minority and the exemplar still receives a positive advantage in the other 88.8% of groups, they do not alter the overall trend, and the model is consistently pulled toward realism, as evidenced by the clear realism gains of PortraitGen.
-
As training proceeds, the mean exemplar advantage decreases toward zero, and its negative rate rises until it matches that of the generated samples after about 80 steps. This does not indicate an increasing number of misjudged real images. Instead, the generated samples progressively approach real-image quality, so the exemplar becomes an ordinary member of the group. Consequently, the influence of the exemplar is strongest when generated samples are far from realism and naturally fades as they catch up, which also prevents it from dominating the later optimization.
| Training steps | Mean exemplar adv. | Exemplar adv. < 0 (% of groups) | Generated adv. < 0 (% of generated samples) |
|---|---|---|---|
| 1–20 | +1.43 | 11.2% | 55.7% |
| 21–40 | +0.92 | 23.8% | 52.4% |
| 41–60 | +0.35 | 39.1% | 50.5% |
| 61–80 | +0.21 | 43.1% | 50.0% |
| 81–100 | +0.03 | 52.8% | 48.9% |
| 101–128 | +0.07 | 50.9% | 49.4% |
[Q1] Adding references
Thank you for the careful reading and helpful suggestion. We have added a citation to RealGen at the first mention of Forensic-Chat, in the caption of Fig. 1.
Responses to Authors
The authors' responses have addressed my concerns; therefore, I maintain my rating, which leans toward acceptance.
Dear Reviewer Ptyr,
Thank you very much for your prompt response and for confirming that our responses have addressed your concerns. We are glad that the new ablation on group-relative anchoring and real-image supervision, together with the analysis of the exemplar advantage dynamics, helped clarify how PortraitGen works.
Your suggestions have substantially strengthened our paper, and we sincerely appreciate your time and insights.
Best regards,
The Authors
Response to Reviewer Ptyr (1/2)
We sincerely thank the reviewer for the insightful and constructive feedback, and we have further strengthened the work along all of the dimensions you highlighted.
Thank you for this insightful comment. Following your suggestion, we disentangle the two mechanisms with two variants under the same rewards and settings. "GRPO + group-relative anchoring" uses the real image only for advantage normalization, and "GRPO + real-image supervision" adds a fixed-weight supervised loss on the real-image trajectory to standard GRPO. We have added this ablation and the corresponding analysis in Appendix D.2 (Tab. 9). The results are also shown in the table below.
GRPO + group-relative anchoring. The anchoring-only variant already improves over FLUX.1-dev, especially in Content and Forensic-Chat. Since the real image raises the reference level of the group, less realistic generated samples receive lower advantages, which pushes the policy toward realism.
GRPO + real-image supervision. Adding a supervised loss on the real image further improves realism over the anchoring-only variant. With anchoring alone, the policy is still updated only on its own samples, so it can at most reinforce the few realistic traits that already appear in them. The supervised loss instead lets the model learn directly from the real-image trajectory, providing realistic details that the model rarely produces by itself. This variant also obtains the highest OmniAID score. OmniAID detects feature-level artifacts, primarily frequency anomalies and abnormal noise patterns, and a fixed-weight loss that fits the real-image trajectory throughout training directly pulls these low-level statistics toward those of real photographs. However, low-level statistics are only one aspect of realism. Since this loss is reward-agnostic and pulls the model toward the real image with the same strength regardless of how close the generated samples already are, it cannot focus on portrait-specific defects such as oily skin and distorted limbs, and thus still lags behind PortraitGen on Content and Forensic-Chat.
Our formulation outperforms GRPO with real-image supervision. PortraitGen achieves the best overall performance, especially in realism. Instead of a fixed-weight SFT loss, the real-image trajectory is weighted by its group-relative advantage, so its supervision is strong when generated samples are far from realism and weakens as they approach it.