acceptodds
Under review as a conference paper at ICLR 2027

PortraitGen: Anchoring GRPO with Real-Image Trajectories for Photorealistic Portrait Generation

Abstract

Reinforcement Learning like Group Relative Policy Optimization (GRPO) has significantly advanced text-to-image post-training. However, existing methods often favor superficial aesthetics, such as over-saturated colors, leaving critical flaws like AI artifacts and biological implausibilities unresolved. We attribute these limitations to two primary factors: (1) GRPO relies solely on self-sampled groups without real-image trajectory references, constraining optimization to the model's existing generative distribution; (2) existing reward signals are insufficiently sensitive to fine-grained portrait artifacts, such as anatomical inconsistencies and unnatural skin appearance. To address this, we propose PortraitGen, a novel framework tailored for photorealistic portrait generation. First, we introduce a real exemplar into each GRPO group and employ BELM inversion to construct its latent trajectory, enabling the exemplar to participate in both relative comparison and policy optimization. Second, to explicitly steer the model toward photorealism, we introduce a complementary dual-reward mechanism: OmniReward for general quality and AI-Portrait for human-centric fidelity. Furthermore, we curate PortraitBench, a comprehensive portrait-centric benchmark. Extensive experiments demonstrate that PortraitGen significantly outperforms existing baselines, effectively suppressing AI artifacts and improving portrait realism.

Primary area:
generative models
Keywords:
Post-training, Text-to-Image Generation, Reward Models

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_PtyrOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper proposes PortraitGen, a framework for improving photorealistic portrait generation through Real-Image-Anchored GRPO. The method introduces real-image trajectories reconstructed via BELM inversion into GRPO groups, allowing them to participate in both relative advantage estimation and policy optimization. The authors further develop two complementary reward models targeting general visual quality and portrait-specific artifacts, and introduce PortraitBench for evaluation.

Critical strengths
  1. Meaningful and well-motivated approach: Incorporating real-image trajectories into GRPO provides a practical way to introduce external photographic references into generative post-training. The combination of real-image anchoring and portrait-specific reward modeling is intuitive and addresses relevant limitations of existing methods.
  2. Strong empirical results: PortraitGen demonstrates substantial improvements in portrait realism and artifact reduction over competitive baselines. The evaluation includes external realism metrics, qualitative comparisons, and a human preference study, providing reasonably convincing evidence of the method's effectiveness.
Critical weaknesses
  1. In my understanding, the core method can be viewed as a combination of advantage-weighted SFT-like learning on real-image trajectories and anchor-augmented GRPO. Specifically, real-image trajectories provide direct supervision through their advantage-weighted likelihood objectives, while also serving as relative anchors that influence the advantage estimation of other generated samples. However, the paper does not sufficiently disentangle the contributions of these two mechanisms. Table 3 evaluates real-image anchoring as a whole, but does not clarify whether the improvements mainly arise from the additional real-image supervision or from the proposed group-relative anchoring mechanism. A comparison with standard GRPO combined with supervised learning on real-image trajectories would help clarify the distinct value of the proposed formulation.

  2. The reliability of real-image anchoring depends on reward accuracy. Real-image exemplars are not assigned fixed positive rewards; their advantages are determined by the learned reward models. If a genuine photograph is incorrectly evaluated as exhibiting synthetic artifacts, its trajectory may receive a negative advantage, potentially discouraging the model from reproducing that real-image example. Since the exemplar also affects group-level reward normalization, such errors may influence the optimization of generated trajectories as well. While the paper reports strong reward-model accuracy, it does not analyze the distribution of exemplar advantages or the sensitivity of the proposed mechanism to erroneous reward assignments.

Questions for authors

Please refer to the Critical Weaknesses. I also noticed a minor issue regarding the references. Some reward models and evaluation components, such as OmniAID, Forensic-Chat, and Long-CLIP, are mentioned without direct citations to their original papers. I had to trace these references through other cited works, such as RealGen, which was somewhat inconvenient. I suggest adding direct citations to the original papers to make it easier for readers to locate and understand these components.

4 comments
AuthorsOpenReview ↗

Response to Reviewer Ptyr (1/2)

We sincerely thank the reviewer for the insightful and constructive feedback, and we have further strengthened the work along all of the dimensions you highlighted.

[W1] Comparison with standard GRPO combined with supervised learning on real-image trajectories

Thank you for this insightful comment. Following your suggestion, we disentangle the two mechanisms with two variants under the same rewards and settings. "GRPO + group-relative anchoring" uses the real image only for advantage normalization, and "GRPO + real-image supervision" adds a fixed-weight supervised loss on the real-image trajectory to standard GRPO. We have added this ablation and the corresponding analysis in Appendix D.2 (Tab. 9). The results are also shown in the table below.

  • GRPO + group-relative anchoring. The anchoring-only variant already improves over FLUX.1-dev, especially in Content and Forensic-Chat. Since the real image raises the reference level of the group, less realistic generated samples receive lower advantages, which pushes the policy toward realism.

  • GRPO + real-image supervision. Adding a supervised loss on the real image further improves realism over the anchoring-only variant. With anchoring alone, the policy is still updated only on its own samples, so it can at most reinforce the few realistic traits that already appear in them. The supervised loss instead lets the model learn directly from the real-image trajectory, providing realistic details that the model rarely produces by itself. This variant also obtains the highest OmniAID score. OmniAID detects feature-level artifacts, primarily frequency anomalies and abnormal noise patterns, and a fixed-weight loss that fits the real-image trajectory throughout training directly pulls these low-level statistics toward those of real photographs. However, low-level statistics are only one aspect of realism. Since this loss is reward-agnostic and pulls the model toward the real image with the same strength regardless of how close the generated samples already are, it cannot focus on portrait-specific defects such as oily skin and distorted limbs, and thus still lags behind PortraitGen on Content and Forensic-Chat.

  • Our formulation outperforms GRPO with real-image supervision. PortraitGen achieves the best overall performance, especially in realism. Instead of a fixed-weight SFT loss, the real-image trajectory is weighted by its group-relative advantage, so its supervision is strong when generated samples are far from realism and weakens as they approach it.

MethodOmniReward-ContentOmniReward-ClarityOmniReward-ColorOmniReward-CompositionForensic-ChatOmniAID
FLUX.1-dev0.630.960.951.2528.4220.23
Vanilla GRPO (DanceGRPO)0.650.950.931.2532.5723.46
GRPO + group-relative anchoring0.780.960.971.2744.5022.67
GRPO + real-image supervision0.930.950.961.2752.8928.83
PortraitGen (Ours)0.970.960.951.3056.6224.58
AuthorsOpenReview ↗

Response to Reviewer Ptyr (2/2)

[W2] Distribution of real-image advantages and sensitivity to erroneous reward assignments

Thank you for this thoughtful comment. We have added an analysis of the exemplar advantage distribution in Section 5.6 (Fig. 8), and the statistics are summarized in the table below. For each training stage, we report the mean advantage of the real exemplar and the percentage of groups in which it receives a negative advantage. As a reference, we also report the percentage of generated samples with negative advantages, which indicates how often an ordinary sample falls below the group average.

  • In the early stage, when generated samples are still clearly distinguishable from real photographs, the exemplar receives a negative advantage in only 11.2% of the groups, whereas 55.7% of the generated samples fall below the group average. Its mean advantage also remains clearly positive (+1.43), indicating that the reward models seldom judge a genuine photograph as worse than the group average. In the remaining 11.2% of groups, a negative exemplar advantage implies that some generated samples in the same group are rated higher than the real image. This can happen because even FLUX.1-dev occasionally rolls out highly realistic portraits. These rollouts then receive positive advantages and are reinforced, so the optimization still moves toward realism. When the model can already produce comparably realistic samples by itself, it does not need to fit a specific real photograph, and learning from its own high-quality rollouts provides an on-policy signal that is easier to absorb. Since such groups are a small minority and the exemplar still receives a positive advantage in the other 88.8% of groups, they do not alter the overall trend, and the model is consistently pulled toward realism, as evidenced by the clear realism gains of PortraitGen.

  • As training proceeds, the mean exemplar advantage decreases toward zero, and its negative rate rises until it matches that of the generated samples after about 80 steps. This does not indicate an increasing number of misjudged real images. Instead, the generated samples progressively approach real-image quality, so the exemplar becomes an ordinary member of the group. Consequently, the influence of the exemplar is strongest when generated samples are far from realism and naturally fades as they catch up, which also prevents it from dominating the later optimization.

Training stepsMean exemplar adv.Exemplar adv. < 0 (% of groups)Generated adv. < 0 (% of generated samples)
1–20+1.4311.2%55.7%
21–40+0.9223.8%52.4%
41–60+0.3539.1%50.5%
61–80+0.2143.1%50.0%
81–100+0.0352.8%48.9%
101–128+0.0750.9%49.4%

[Q1] Adding references

Thank you for the careful reading and helpful suggestion. We have added a citation to RealGen at the first mention of Forensic-Chat, in the caption of Fig. 1.

Reviewer_PtyrOpenReview ↗

Responses to Authors

The authors' responses have addressed my concerns; therefore, I maintain my rating, which leans toward acceptance.

Dear Reviewer Ptyr,

Thank you very much for your prompt response and for confirming that our responses have addressed your concerns. We are glad that the new ablation on group-relative anchoring and real-image supervision, together with the analysis of the exemplar advantage dynamics, helped clarify how PortraitGen works.

Your suggestions have substantially strengthened our paper, and we sincerely appreciate your time and insights.

Best regards,

The Authors

Reviewer_kNCHOpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

My understanding on the paper's contributions are as follows:

  • The authors propose a method to include real trajectories in GRPO using BELM inversion. Specifically, one real sample rollout can be inserted into a group of G (G-1 synthetic and 1 real sample).
  • The paper also proposes portrait-image specific reward models such as content, clarity, color and composition, artifacts and skin oiliness. These reward models (OmniReward, AI-Portrait Reward) are trained reward functions.
  • The paper additionally proposes a dataset of 1000 samples for portrait generation. These cover diverse scenarios and photorealism.
  • The results on the proposed dataset show overall improvement in the axes measured by the paper.
Critical strengths

The strengths of the paper are listed below.

  • Clarity: I found the paper very clear in its framing of the problem, demonstration of results as well as explaining the solution. This is highly appreciated. Figures and tables are self explanatory, a motivation is defined earlier and the paper does an overall good job at presenting it's findings.

  • Real Trajectories: The idea to include real trajectories in the RL loop is strong and has a clear motivation. Additionally, the authors demonstrate that this works in the settings they analyze it. This does have widespread impact on the community.

  • Performance: What is impressive is that performance on OmniAID and Forensic chat improve but these weren't used as reward models. Hence, there is likelihood that the output results are not reward-hacked.

Critical weaknesses

[W1] Reward Ablations: The major claim in the papers motivation for reward training is that off the shelf rewards (HPS3, Pickscore, etc.) produce poor images on the axes the authors are measuring (ai-generation, oiliness, color, etc). While this is a valid motivation to train and propose their own models, the paper does not have any experiment showing how the proposed rewards outperform a simple setup of HPS3+Real trajectory BELM. The comparisons to DanceGRPO are not apples to apples for the reward bit per-se, as DanceGRPO hasn't been trained on real trajectories.

[W2] Benchmark diversity: While the paper proposes a benchmark, all their results are on this benchmark. This is a slight problem as the validity of the benchmark isn't proven (the benchmark isn't public). The authors should consider reporting results on other benchmarks such as Geneval2<1>(normal scenes), MultiHuman-TestBench<2> or MultiID-Bench<3>(specific for single/multi-person generation). The current benchmark does not have a wide representation of multi-person/object scenes. The metrics can remain the same but type of data can vary.

[W3] Task diversity: RL improves performance on the axes you finetune, and hence the results are great on the rewards and Forensic metric. However, there is no mention of if the performance on other tasks (such as composition, etc. measured from Geneval2) is retained. It will be interesting to see if the performance of base model is retained on these tasks. Additionally, Pickscore isn't a great metric to report overall prompt alignment/Quality. Consider reporting on newer metrics (e.g. HPS3).

[W4] Missing Related Work and Comparisons: Another recent method<4> has RL finetuned specifically on human scenes. This includes quality and diversity finetuning.

<1> https://arxiv.org/abs/2512.16853

<2> https://arxiv.org/abs/2506.20879

<3> https://arxiv.org/abs/2510.14975

<4> https://arxiv.org/abs/2510.01399

Questions for authors

[Q1] Have the authors considered HPSv3<5> instead of HPSv2, as the claim was that HPSv2 makes faces appear oily?

[Q2] For the unified rewards which have been proposed, have the authors performed a correlation analysis with SOTA T2I metrics? If there is decorrelation, this can motivate the use of these rewards as they track a different axis than the existing metrics.

[Q3] What type of reward hacking was observed during RL finetuning with the proposed rewards? For a reader, this is important to know before they start training with the proposed rewards.

<5> https://arxiv.org/abs/2508.03789

7 comments
AuthorsOpenReview ↗

Response to Reviewer kNCH (1/3)

Thank you for your diligent efforts in reviewing our paper and for recognizing the contributions of our work. We provide our detailed responses below.

[W1] Reward Ablations

Thank you for this insightful suggestion. We would like to kindly note that the 4th row of Table 3 in the main paper (Reward Models: ×, Real-Image Anchor: ✓) corresponds to an off-the-shelf reward + real-trajectory BELM setting, where our rewards are replaced with HPSv2.1 while the real-image anchoring via BELM is kept unchanged. To make the ablation more complete, we further follow the reviewer's suggestion and train HPSv3 + real-trajectory BELM under the same setting. We have added this experiment and the corresponding analysis in Appendix D.3 (Tab. 10). The results are shown in the table below, which indicate that HPSv3 + real-trajectory BELM clearly improves over FLUX.1-dev. However, it remains behind our method in realism, especially on portrait-specific AI artifacts such as distorted limbs. We believe this is because, although HPSv3 covers a broader aesthetic distribution than HPSv2.1 and produces slightly less oily faces, it is primarily designed as a holistic aesthetic preference model rather than a fine-grained discriminator for local portrait anatomy (e.g., extra fingers, twisted joints) or synthetic artifacts. The guidance of the real-image anchor is better exploited when it is combined with our OmniReward and AI-Portrait Reward.

MethodOmniReward-ContentOmniReward-ClarityOmniReward-ColorOmniReward-CompositionForensic-ChatOmniAID
FLUX.1-dev0.630.960.951.2528.4220.23
HPSv3 + Real-trajectory BELM0.940.870.911.2554.9920.58
PortraitGen (Ours)0.970.960.951.3056.6224.58

[W2] Benchmark diversity

Thank you for the valuable comments. We further evaluate representative methods on two additional benchmarks, GenEval2 for general scenes and MultiID-Bench for single- and multi-person generation, using the main metrics adopted in the paper. We have added these results and the corresponding analysis in Appendix D.6 (Tab. 12). The results are also shown in the table below, which indicate that PortraitGen achieves the best overall performance on realism-related metrics on both benchmarks, while remaining comparable to FLUX.1-dev on the other general quality metrics. This demonstrates that our method effectively improves the realism of generated content beyond our own benchmark. In addition, our PortraitBench also covers multi-person scenes, and the corresponding qualitative comparisons are shown in Fig. 4 of the main paper and Fig. 17 in Appendix F, where PortraitGen renders natural skin textures and plausible anatomy for every subject in scenes with up to five people.

BenchmarkMethodOmniReward-ContentOmniReward-ClarityOmniReward-ColorOmniReward-CompositionForensic-ChatOmniAID
GenEval2FLUX.1-dev0.750.991.001.0117.2914.80
DanceGRPO0.740.980.991.0115.6314.23
PortraitGen (Ours)0.86 (+0.11)0.990.991.03 (+0.02)29.71 (+12.42)14.66
MultiID-BenchFLUX.1-dev0.820.810.971.2038.9018.14
DanceGRPO0.790.720.961.1837.8218.22
PortraitGen (Ours)1.15 (+0.33)0.800.931.23 (+0.03)56.07 (+17.17)18.65 (+0.51)

[W3] Task diversity

Thank you for your valuable suggestion. To examine whether the general capabilities of the base model are retained after realism-oriented fine-tuning, we evaluate PortraitGen on GenEval2 with its Soft-TIFA metrics, and additionally report HPSv3 as a human preference metric. We have added these results and the corresponding analysis in Appendix D.7 (Tab. 13). As summarized in the table below, PortraitGen surpasses FLUX.1-dev on every GenEval2 metric, from the overall Soft-TIFA GM/AM scores to the object, attribute, count, position, and verb sub-scores, and also yields a slight gain in HPSv3. Rather than trading general ability for realism, our fine-tuning preserves and even slightly improves the aesthetic quality as well as capabilities such as compositional generation and counting.

BenchmarkMethodSoft-TIFA GMSoft-TIFA AMObjectAttributeCountPositionVerbHPSv3
GenEval2FLUX.1-dev20.8467.6788.4968.3751.2741.0027.897.544
DanceGRPO21.3267.6188.4669.2751.0740.7224.867.357
PortraitGen (Ours)23.5168.5289.3468.5352.8841.1128.467.563
AuthorsOpenReview ↗

Response to Reviewer kNCH (2/3)

[W4] Missing Related Work and Comparisons

Thank you for pointing us to this relevant work. We have added a discussion of DisCo in Section 2.2. Although both methods apply GRPO to flow-matching models in human-centric scenarios, they address different problems and differ substantially in algorithm design.

  • At the task level, DisCo targets the identity crisis in multi-human generation, where models produce duplicated faces, merged identities, or incorrect person counts. Its objective is therefore identity diversity and count accuracy, both within an image and across samples. PortraitGen instead focuses on photorealism, aiming to remove the synthetic appearance of generated portraits, such as AI artifacts and unnatural or oily skin. In other words, DisCo is concerned with whether each person is distinct, whereas PortraitGen is concerned with whether each person looks real.

  • At the algorithm level, the two methods differ in both the optimization data and the reward design. DisCo is trained purely on self-sampled groups without any real data, so the policy is optimized within its own generative distribution. PortraitGen introduces a real exemplar into each GRPO group and constructs its latent trajectory via BELM inversion, allowing the real image to participate in both the relative comparison and the policy update, which explicitly pulls the generated distribution toward real images. Regarding rewards, DisCo combines identity-oriented signals, including facial-similarity penalties, a group-wise counterfactual reward against cross-sample identity repetition, and a person-count reward, with human preference scores that preserve visual quality. PortraitGen relies on two learned VLM-based reward models, OmniReward for multidimensional visual quality and AI-Portrait Reward for portrait-specific artifacts and skin oiliness.

[Q1] Using HPSv3 instead of HPSv2

Thank you for raising this point. Following your suggestion, we retrain DanceGRPO with HPSv3 in place of HPSv2 under an identical setting. We have added this experiment and the corresponding analysis in Appendix D.3 (Tab. 10). The results are also shown in the table below. Adopting HPSv3 yields a noticeable improvement in color appeal (OmniReward-Color 0.93 → 0.97) along with modest gains in Content and Composition over FLUX.1-dev. However, its improvement over the HPSv2 counterpart remains relatively limited, particularly in terms of realism, where Forensic-Chat stays at a comparable level. We attribute this to the fact that realism in our setting extends well beyond the oily appearance of skin. It also encompasses portrait-specific AI artifacts such as distorted or incomplete limbs, which a holistic aesthetic preference model like HPSv3 is not designed to penalize. In contrast, as reported in our response to W1, once the real-image trajectory is introduced into training (HPSv3 + Real-trajectory BELM), HPSv3 brings a substantial improvement in realism, clearly exceeding its HPSv2.1 counterpart in Table 3 of the main paper. This indicates that a stronger reward is indeed beneficial, but its advantage mainly emerges when it is combined with our real-image guided training. Building on this, PortraitGen adopts OmniReward and AI-Portrait Reward, which are specifically designed for portrait realism and AI artifacts, and achieves the best overall performance.

MethodOmniReward-ContentOmniReward-ClarityOmniReward-ColorOmniReward-CompositionForensic-ChatOmniAID
FLUX.1-dev0.630.960.951.2528.4220.23
DanceGRPO (HPSv2)0.650.950.931.2532.5723.46
DanceGRPO (HPSv3)0.670.920.971.2731.1418.99
PortraitGen (Ours)0.970.960.951.3056.6224.58
AuthorsOpenReview ↗

Response to Reviewer kNCH (3/3)

[Q2] Correlation analysis between the proposed rewards and SOTA T2I metrics

Thank you for this constructive suggestion. We conduct an image-level correlation analysis on PortraitBench, using 9,000 images generated by the nine models listed in Tab. 2 of the main paper. For each image, we compute the Spearman correlation between the four dimensions of OmniReward and widely used T2I metrics, including the preference and alignment metrics PickScore, HPSv3, and UnifiedReward, as well as the realism-oriented detector Forensic-Chat and OmniAID. We have added this analysis in Section 5.6 (Fig. 9). As shown in the table below, OmniReward is largely decorrelated from existing preference metrics, with |ρ| ≤ 0.10 against PickScore, HPSv3, UnifiedReward-Alignment, and UnifiedReward-Style across all four dimensions. Meanwhile, the Content dimension shows its strongest correlation with Forensic-Chat, indicating that it captures portrait realism, an aspect to which current preference metrics are largely insensitive.

PickScoreHPSv3UR-AlignmentUR-CoherenceUR-StyleForensic-ChatOmniAID
OmniReward-Content−0.090.030.060.22−0.050.38−0.09
OmniReward-Clarity0.04−0.010.020.040.04−0.100.06
OmniReward-Color0.100.040.01−0.050.07−0.170.01
OmniReward-Composition0.080.100.080.230.030.05−0.06

[Q3] Reward hacking observed with the proposed rewards

Thank you for raising this practical point. We observed one main form of reward hacking, which emerges under prolonged training, and the multi-dimensional design of our rewards allows it to be detected early. We have added this analysis and the corresponding visualization in Appendix E (Fig. 13).

Our rewards penalize the synthetic appearance of generated portraits, including AI artifacts such as distorted limbs, which are captured by both AI-Portrait Reward and OmniReward-Content, as well as the overly smooth and oily skin. Under aggressive optimization, the policy may learn that adding grain-like textures or softening fine details makes an image look less synthetic. With our full method, such a drift toward blurred and grainy details appears only when training is extended well beyond the checkpoint reported in the paper.

Since OmniReward scores Clarity as a separate dimension, texture degradation is directly penalized, so realism gained at the cost of sharpness is not rewarded. As shown in Fig. 13, compared with earlier checkpoints, the late-stage samples generated from the same prompts become noticeably blurred and obtain equal or even higher Content and Forensic-Chat scores, while their Clarity scores clearly decline. Clarity is also the first dimension to decline once hacking begins, well before Content and Composition drop, which makes it a reliable early indicator.

Reviewer_fLwfOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

PortraitGen improves portrait realism by incorporating real photographs into GRPO, using image inversion to reconstruct reference trajectories that guide training alongside generated samples. It combines two reward models: OmniReward assesses overall visual quality, while AI-Portrait Reward targets synthetic artifacts such as unnatural skin and anatomical inconsistencies. Experiments on the newly introduced PortraitBench benchmark and a human preference study report improved realism over competing methods, with gains demonstrated on both FLUX and Stable Diffusion backbones.

Critical strengths

Using real photographs as training references helps reduce unnatural skin textures and anatomical errors, producing more realistic portraits.

Critical weaknesses
  1. I appreciate the effort to generate more realistic photos by incorporating generation trajectories derived from real photographs into training, but I question whether GRPO is necessary. Portrait images are very easy to obtain, and fine-tuning through SFT or LoRA should also be possible. Why is GRPO needed?

  2. Too few example images are shown. More portrait examples would make the results more convincing. I also do not believe GRPO solves every problem in portrait image generation, so an analysis of failure cases is needed.

  3. The FLUX.1-dev examples look overly cartoonish. I think this could be improved through prompting, but the authors appear to have deliberately omitted such controls from their prompts.

Questions for authors

Please refer to the weaknesses.

6 comments
AuthorsOpenReview ↗

Response to Reviewer fLwf (1/2)

We sincerely thank the reviewer for the careful reading and constructive feedback. We greatly appreciate the recognition of our real-image guided training strategy. We provide our detailed responses below.

[W1] Why is GRPO necessary instead of SFT / LoRA fine-tuning on real portraits?

We appreciate your concern regarding the necessity of GRPO compared with direct fine-tuning on real portraits. To examine this, we fine-tune FLUX.1-dev on the same real photographs used in our GRPO training, with both full-parameter fine-tuning (SFT-Full) and LoRA fine-tuning (SFT-LoRA). We have added this comparison and the corresponding analysis in Appendix D.1 (Tab. 8). The quantitative results are also shown in the table below. We summarize our findings as follows.

  • SFT-LoRA offers a better overall trade-off than SFT-Full. Full-parameter fine-tuning leads to clear degradation in Clarity (0.89), Color (0.90), and Composition (1.12), and the qualitative results also reveal image collapse. This is likely caused by overfitting, since updating all parameters on a limited set of photographs erodes the generative prior of the base model, while the low-rank constraint of LoRA acts as an implicit regularizer.

  • Both SFT variants remain behind PortraitGen (Ours). PortraitGen achieves the best realism (Forensic-Chat 56.62) while keeping clarity and color intact. We attribute this to two reasons. First, the real-image trajectory is not fitted with a fixed weight as in SFT. It is weighted by its group-relative advantage, so its supervision is strong when the model's own samples are far from realism and weakens as they approach it. Second, our reward models explicitly measure realism and anatomical defects such as distorted limbs, so GRPO can control what the model learns, whereas SFT treats every attribute of the training images as an equal fitting target, regardless of whether it is related to realism.

MethodOmniReward-ContentOmniReward-ClarityOmniReward-ColorOmniReward-CompositionForensic-ChatOmniAID
FLUX.1-dev0.630.960.951.2528.4220.23
SFT-Full0.880.890.901.1253.8322.12
SFT-LoRA0.900.950.941.1851.1924.41
PortraitGen (Ours)0.970.960.951.3056.6224.58

[W2] More portrait examples and failure case analysis.

Thank you for the constructive suggestion. Following your advice, we have added more portrait examples as well as a failure case analysis to the revised paper.

  • More portrait examples: We provide more qualitative comparisons in Fig. 4 of the main paper and in Appendix F (Figs. 14–17), covering single-person, two-person, and multi-person portraits. Compared with the baselines, PortraitGen consistently reduces oily skin textures and structural artifacts such as distorted hands.

  • Failure cases: We agree that GRPO does not solve every problem in portrait generation, and we have added a failure case analysis in Appendix F (Fig. 17). Since our post-training does not introduce a dedicated reward for text-image alignment, PortraitGen largely inherits the instruction-following ability of FLUX.1-dev and fails where the base model fails. For example, both models generate four men when three are requested, and render the word "Love" on the paper crowns rather than on the cheeks. Nevertheless, PortraitGen still improves the realism of each person in these scenes. Combining our method with count-aware or alignment-oriented rewards is a promising direction for future work.

AuthorsOpenReview ↗

Response to Reviewer fLwf (2/2)

[W3] Could the cartoonish look of FLUX.1-dev examples be fixed by prompting?

Thank you for this valuable suggestion. We address it from two perspectives.

  • As shown in Fig. 4 of the main paper and Figs. 14–16 in the Appendix, every prompt we use already specifies the photographic style (e.g., photography style, realistic photograph) on top of the scene description. Nevertheless, FLUX.1-dev still frequently produces overly cartoonish portraits, which suggests that this issue cannot be resolved by prompting alone. We have also discussed this observation in Section 5.2 (Lines 346–348), where baselines such as FLUX.1-dev, DanceGRPO, and RealGen often fall back to oil-painting or anime styles despite the explicit photorealistic instructions in the prompts.

  • Prompting only steers sampling within the model's existing output distribution. A better prompt helps the model reach the best results it can already produce, but it cannot make the model generate what lies beyond its capability. Post-training instead reshapes the distribution itself and raises the upper bound of the model. The two routes are therefore not in conflict with each other, and this work focuses on improving portrait realism through GRPO post-training. To verify the limitation of prompting empirically, we follow the reviewer's suggestion and use Qwen3.8-27B as a prompt enhancer (FLUX.1-dev + PE) to rewrite each PortraitBench prompt. The enhancer adds more visual details, includes realism-related descriptions, and explicitly asks to avoid common portrait AI artifacts. We have added this comparison and the corresponding analysis in Appendix D.4 (Tab. 11). As shown in the table below, prompt enhancement slightly improves Clarity and Color but brings no gain in realism, and the realism metrics even decline slightly. In contrast, PortraitGen substantially improves realism across all realism-related metrics.

MethodOmniReward-ContentOmniReward-ClarityOmniReward-ColorOmniReward-CompositionForensic-ChatOmniAID
FLUX.1-dev0.630.960.951.2528.4220.23
FLUX.1-dev + PE0.620.980.961.2826.2620.23
PortraitGen (Ours)0.970.960.951.3056.6224.58

Dear Reviewer fLwf,

Thank you again for your valuable comments on our paper. We hope our responses have addressed your concerns. In the rebuttal, we added a comparison with full-parameter and LoRA fine-tuning on the same real portraits, more portrait examples together with a failure case analysis, and a comparison with prompt enhancement, all of which have been included in the revised paper.

We would greatly appreciate any feedback you may have on our responses, and we would be happy to discuss any further questions.

Best regards,

The Authors

Comments on the paper

1 comment
AuthorsOpenReview ↗

Global response and summary of changes in the revision

We sincerely thank all reviewers for their thorough reviews and constructive feedback. We are encouraged that the reviewers recognize the well-motivated idea of incorporating real-image trajectories into GRPO (all reviewers) and its potential impact on the community (Reviewer kNCH), the substantial improvements in portrait realism with fewer unnatural skin textures and anatomical errors (all reviewers), the comprehensive and convincing evaluation (Reviewer kNCH, Ptyr), as well as the clear presentation of the paper (Reviewer kNCH, Ptyr).

Below, we summarize the changes made in the revised paper.

  • We added a comparison with supervised fine-tuning on the same real portraits, including full-parameter and LoRA fine-tuning, in Appendix D.1 (Tab. 8). (Reviewer fLwf)
  • We added an ablation that disentangles the group-relative anchoring and the real-image supervision roles of the exemplar in Appendix D.2 (Tab. 9). (Reviewer Ptyr)
  • We added experiments with HPSv3 as the reward, both in DanceGRPO and combined with the real-image anchor, in Appendix D.3 (Tab. 10). (Reviewer kNCH)
  • We added a comparison with prompt enhancement in Appendix D.4 (Tab. 11). (Reviewer fLwf)
  • We added evaluations on GenEval2 and MultiID-Bench in Appendix D.6 (Tab. 12), and evaluated the preservation of general capabilities with Soft-TIFA metrics and HPSv3 in Appendix D.7 (Tab. 13). (Reviewer kNCH)
  • We added an analysis of the exemplar advantage dynamics during training in Section 5.6 (Fig. 8). (Reviewer Ptyr)
  • We added an image-level correlation analysis between OmniReward and existing T2I metrics in Section 5.6 (Fig. 9). (Reviewer kNCH)
  • We added an analysis of reward hacking under prolonged training in Appendix E (Fig. 13). (Reviewer kNCH)
  • We added a two-person example to Fig. 4, as well as multi-person comparisons and a failure case analysis in Appendix F (Fig. 17). (Reviewer fLwf, kNCH)
  • We added a discussion of DisCo in Section 2.2. (Reviewer kNCH)
  • We added a citation to RealGen at the first mention of Forensic-Chat, in the caption of Fig. 1. (Reviewer Ptyr)

Please see our detailed responses below each review and all the changes in the revised paper are highlighted in blue.

We sincerely thank all reviewers again for their valuable suggestions, which have greatly helped strengthen our paper.

If you have any further questions, we would be happy to discuss them!

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.