acceptodds
Under review as a conference paper at ICLR 2027

Anatomically Plausible Human Image Generation via Synthetic Localized Preferences

Abstract

Large-scale text-to-image foundation models have substantially improved the visual realism of generated human images. However, their outputs may still exhibit anatomical artifacts in structurally complex regions. Preference-based alignment offers an architecture-preserving approach to improving anatomical plausibility. In practice, conventional preference pairs typically compare independently generated images whose differences extend beyond anatomy, making it unclear whether anatomical quality drives the preference. To provide direct anatomical supervision, we present Alignment via Synthetic Anatomical Preferences (ASAP), a data-centric framework built on synthetic localized preferences. ASAP constructs synthetic localized preferences by using verified anatomically plausible images as preferred anchors and creating dispreferred images via localized degradation, while largely preserving prompt semantics and non-target visual content. This pipeline yields the Human Anatomical Preference (HAP) dataset, comprising over 10K curated pairs with region masks and anatomical error metadata. To exploit their spatial attribution while preserving overall image quality, we adapt Direct Preference Optimization (DPO) with a region-aware preference gap that emphasizes modified regions and a margin-bounded regression objective that drives the gap toward a finite target. We further introduce HAF-Bench, a benchmark for systematic evaluation of anatomical fidelity across diverse scenarios. Extensive experiments across diffusion and flow-matching backbones demonstrate that ASAP consistently reduces anatomical errors while maintaining overall image quality.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Human Image Generation, Anatomical Plausibility, Synthetic Localized Preferences, Direct Preference Optimization, Region-Aware Alignment

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_LFpWOpenReview ↗
Rating
3
Confidence
4
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

This paper proposes ASAP, which achieves this by locally degrading images of the human body structure in a reasonable manner, constructing preference data with clear anatomical errors, and combining the preference optimization objective of region weighting and limited intervals to enhance the rationality of the generated image's human body structure. The authors also constructed the HAP dataset and the HAF-Bench evaluation benchmark. Experiments show that this method can effectively reduce the anatomical error rate of the human body on FLUX and SDXL, while maintaining good overall image quality.

Critical strengths
  1. The problem is of practical significance. By leveraging local synthesis preferences to construct targeted supervision signals for local anatomical errors in human body image generation, the approach is logically sound.
  2. The method design is relatively comprehensive, integrating data construction, region-aware optimization, and evaluation benchmarks, and has certain practical value.
  3. The experiments are sufficiently comprehensive, covering different generation models, baseline comparisons, ablation experiments, and manual evaluations, and the results as a whole support the effectiveness of the method.
Critical weaknesses
  1. The method is relatively limited in terms of innovation. Both the regional weighting and the finite interval optimization are based on existing ideas, and their main contributions lie in the combination and application for solving human anatomy problems.
  2. There may be a distribution difference between the synthetic anatomical errors and the errors naturally generated by the real generation model. The analysis of this issue in the paper is still insufficient.
  3. HAF-Bench mainly relies on visual language models for automatic evaluation. Although it provides manual verification, further explanations are needed regarding the reliability and generalization ability of the evaluation results.
Questions for authors
  1. Will the types and degrees of synthetic degradation affect the model's generalization ability for real anatomical errors?
  2. Can we further analyze the improvement extent on different anatomical regions or error types?
  3. It is suggested to add a direct comparison with existing region-aware preference optimization methods to more clearly demonstrate the contribution of the optimization objective itself.
2 comments

We sincerely appreciate the reviewer’s constructive feedback and insightful suggestions. Below, we provide point-by-point responses.


W1 & Q3: The method is relatively limited in terms of innovation. Both the regional weighting and the finite interval optimization are based on existing ideas, and their main contributions lie in the combination and application for solving human anatomy problems.

It is suggested to add a direct comparison with existing region-aware preference optimization methods to more clearly demonstrate the contribution of the optimization objective itself.

Our Contribution

Our main contribution is synthetic localized preferences, and the main benefit of our objective is better anatomy quality while preserving the global image quality. We add a direct comparison with LocalDPO to better assess the contribution of our optimization objective.

The main contribution of this work is synthetic localized preferences, a data construction strategy that enables DPO-style alignment to improve anatomical correctness in human image generation. Our objective is designed to make effective use of these localized preference pairs. Specifically, we introduce a margin-bounded DPO variant to better balance anatomical correctness and overall image quality.

We acknowledge that both finite target margins and region-aware weighting build on existing ideas. The contribution of our objective is to adapt and combine these ideas for localized anatomical preferences in diffusion and flow models. The results in Table 4 show further improvements over using the IPO formulation alone. Existing region-aware alignment methods differ in their training objectives and input requirements, which affects how directly they can be applied to our task:

  • PatchDPO (CVPR 2025) uses patch-weighted reconstruction of individual images with a reference image, rather than a pairwise DPO objective. Its patch matching requires a reference image of the subject, which is not available in our general text-to-image setting.
  • Focus-N-Fix (CVPR 2025) uses online optimization with differentiable rewards through DRaFT, rather than offline preference optimization. Applying it to our task would require a suitable differentiable reward for anatomical correctness, including details such as finger counts and joint connections.
  • LocalDPO (CVPR 2026) combines a local masked DPO loss, a global standard DPO loss, and a supervised fine-tuning loss on positive images through a weighted sum. This makes it a relevant baseline for assessing the balance between local anatomical improvements and global image quality.

We therefore added a direct comparison with LocalDPO :

MethodAER ↓\downarrowHPSv3 ↑\uparrow
FLUX0.41710.307
FLUX-LocalDPO0.30710.266
FLUX-ASAP (Ours)0.28910.360

In this comparison, FLUX-ASAP achieves both a lower AER and a higher HPSv3 score than FLUX-LocalDPO. These results support the effectiveness of our margin-bounded objective in balancing anatomical correctness and image quality with a simpler formulation.


W2 & Q1 & Q2: There may be a distribution difference between the synthetic anatomical errors and the errors naturally generated by the real generation model. The analysis of this issue in the paper is still insufficient.

Will the types and degrees of synthetic degradation affect the model's generalization ability for real anatomical errors?

Can we further analyze the improvement extent on different anatomical regions or error types?

Synthetic vs. Naturally Generated Errors

We provide qualitative comparisons of synthetic and naturally generated errors and conduct separate training experiments with hand-only and face-only degradation to examine how the targeted anatomical region affects performance.

We agree that the potential distribution gap between synthetic and naturally generated anatomical errors is important. This gap affects the quality of synthetic localized preferences and the training signal they provide during preference optimization. We considered this issue during our experiments and provide qualitative examples of both types of errors in Appendix Figure 7.

To examine the effect of the anatomical types targeted by synthetic degradation, we trained separate models using preference pairs with degradation applied only to hands or only to faces:

Data TypeAER ↓\downarrowHPSv3 ↑\uparrow
Hand0.30510.435
Face0.36710.358

The model trained with hand-only degradation achieves a lower overall AER than the model trained with face-only degradation, while both maintain HPSv3 scores close to that of the base FLUX model. This suggests that the anatomical region targeted during data construction affects the resulting performance.


W3: HAF-Bench mainly relies on visual language models for automatic evaluation. Although it provides manual verification, further explanations are needed regarding the reliability and generalization ability of the evaluation results.

Evaluation Reliability

We now report standard deviations and task-specific human–VLM agreement with Cohen’s κ to better characterize the variability of our results and the judge’s reliability on FLUX outputs.

All evaluations use 500 prompts and 5 seeds. To better characterize the variability of the results, we now report the mean and standard deviation:

MethodsAER ↓\downarrowHPSv3 ↑\uparrow
FLUX0.417 ±\pm 0.01110.307 ±\pm 0.224
FLUX-ASAP-DPO0.293 ±\pm 0.0039.680 ±\pm 0.163
FLUX-ASAP (Ours)0.289 ±\pm 0.00510.360 ±\pm 0.186

Compared with the FLUX base model, FLUX-ASAP achieves a lower anatomical error rate while maintaining a similar HPSv3 score. Compared with FLUX-ASAP-DPO, it achieves a similar AER and a higher HPSv3 score, indicating better image quality as measured by HPSv3.

We separate human–VLM agreement into absolute and pairwise judgments and report Cohen's κ\kappa to account for agreement expected by chance. The previously reported 82% agreement combined these two tasks on FLUX-generated images. The task-specific results are as follows:

TaskAgreementCohen's κ\kappa
Absolute86.2%0.657
Relative77.8%0.643

These results show agreement beyond chance for both tasks and support the use of the VLM as an evaluator on the assessed outputs.

Reviewer_CXtEOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper presents ASAP, a preference-based method for reducing anatomical errors in generated human images. The authors construct training pairs by introducing local anatomical defects into verified images, aiming to isolate anatomy from other visual differences. They use a region-aware, margin-bounded objective to learn from these pairs and report anatomical improvements with largely preserved image quality on diffusion- and flow-based models.

Critical strengths
  1. HAP provides over 10K curated preference pairs with localized anatomical edits and region masks, offering useful supervision for anatomical alignment. Its gains with vanilla DPO and the human and Human-Art evaluations support the practical value of this data construction.

  2. The region-aware, margin-bounded objective improves image quality over vanilla DPO while retaining comparable anatomical accuracy (Table 4). It works with both diffusion- and flow-based generators without changing their architecture or inference procedure.

Critical weaknesses
  1. My main reservation concerns the interpretation of the data comparison in Table 3. Although prompts, pair counts, and optimization are matched, Appendix D.3 does not specify the ranking reward or establish whether the ranking baselines receive anatomical verification comparable to HAP. It is therefore difficult to separate the contribution of locality from that of positive-image curation and anatomy-specific labeling. The results support the complete construction pipeline, but the attribution of its advantage specifically to reduced non-target differences in Section 6.3 is less well established.

  2. The precision of comparisons among adapted methods is not well characterized. The paper reports five-seed averages without uncertainty estimates, while human evaluation compares ASAP with the base models rather than competing adaptations. The reported 82% human–VLM agreement also pools absolute and pairwise judgments (Appendix C.2), leaving agreement on each task unclear. This limits confidence in the finer method comparisons, although the human results do support the overall anatomical gains.

Questions for authors
  1. Which reward is used for the ranking baselines in Table 3? Were their preferred images and preference order checked for anatomical correctness? A comparison matching these aspects of curation would help clarify the specific benefit of locality.

  2. Can the authors report uncertainty for paired method differences across prompts and evaluation seeds, and provide human–VLM agreement separately by task and backbone? Please also clarify how human ratings are aggregated. If direct human comparisons with ASAP-DPO or FairHuman† are available, do they support the automatic results?

3 comments

We sincerely appreciate the reviewer’s constructive feedback and insightful suggestions. Below, we provide point-by-point responses.


W1 & Q1: My main reservation concerns the interpretation of the data comparison in Table 3. Although prompts, pair counts, and optimization are matched, Appendix D.3 does not specify the ranking reward or establish whether the ranking baselines receive anatomical verification comparable to HAP. It is therefore difficult to separate the contribution of locality from that of positive-image curation and anatomy-specific labeling. The results support the complete construction pipeline, but the attribution of its advantage specifically to reduced non-target differences in Section 6.3 is less well established.

Which reward is used for the ranking baselines in Table 3? Were their preferred images and preference order checked for anatomical correctness? A comparison matching these aspects of curation would help clarify the specific benefit of locality.

Contribution of Locality

We clarify the baseline reward and anatomical checks, and add a comparison using the same high-quality positive image to assess the additional contribution of localized degradation.

The ranking baselines in Table 3 use PickScore as the reward model, following the data construction strategy used in existing methods such as HG-DPO. Their preferred images and preference rankings were not explicitly checked for anatomical correctness. We agree that the original comparison does not fully separate the effects of positive-image curation and anatomy-specific supervision.

To better control for anatomy-specific supervision, we added a comparison using the same high-quality positive images as HAP. For this variant, we used the HG-DPO-Easy pipeline to obtain negative images selected by the reward model, without applying localized degradation. With the positive images fixed, localized degradation reduces AER from 0.358 to 0.289 while maintaining a similar HPSv3 score. This suggests that localized anatomical degradation provides a useful training signal beyond using high-quality positive images alone.

MethodsAER ↓\downarrowHPSv3 ↑\uparrow
FLUX-ASAP (Ours)0.28910.360
FLUX-ASAP (w/o localized degradation)0.35810.337

W2 & Q2: The precision of comparisons among adapted methods is not well characterized. The paper reports five-seed averages without uncertainty estimates, while human evaluation compares ASAP with the base models rather than competing adaptations. The reported 82% human–VLM agreement also pools absolute and pairwise judgments (Appendix C.2), leaving agreement on each task unclear. This limits confidence in the finer method comparisons, although the human results do support the overall anatomical gains.

Can the authors report uncertainty for paired method differences across prompts and evaluation seeds, and provide human–VLM agreement separately by task and backbone? Please also clarify how human ratings are aggregated. If direct human comparisons with ASAP-DPO or FairHuman† are available, do they support the automatic results?

Evaluation Uncertainty and Judge Agreement

We now report standard deviations and task-specific human–VLM agreement, and add a direct VLM-based comparison with standard DPO to better assess evaluation reliability and differences between methods.

All evaluations use 500 prompts and 5 seeds. Following the reviewer's suggestion, we now report the mean and standard deviation:

MethodsAER ↓\downarrowHPSv3 ↑\uparrow
FLUX0.417 ±\pm 0.01110.307 ±\pm 0.224
FLUX-ASAP-DPO0.293 ±\pm 0.0039.680 ±\pm 0.163
FLUX-ASAP (Ours)0.289 ±\pm 0.00510.360 ±\pm 0.186

Compared with the FLUX base model, FLUX-ASAP achieves a lower anatomical error rate while maintaining a similar HPSv3 score. Compared with FLUX-ASAP-DPO, it achieves a similar AER and a higher HPSv3 score, indicating better image quality as measured by HPSv3.

Each image or image pair is rated by one human annotator. We pool these ratings to compute human-VLM agreement from binary yes/no labels and pairwise win/loss/tie rates from three-category labels. We separate human–VLM agreement into absolute and pairwise judgments and report Cohen’s κ\kappa to account for agreement expected by chance. The previously reported 82% agreement combined these two tasks on FLUX-generated images. The task-specific results are as follows:

TaskAgreementCohen's κ\kappa
Absolute86.2%0.657
Relative77.8%0.643

These results show agreement beyond chance for both tasks and support the use of the VLM as an evaluator on the assessed outputs.

We also conducted a direct pairwise comparison between ASAP and standard DPO using the VLM judge. ASAP achieved a 53.2% win rate, 34.3% tie rate, and 12.5% loss rate. These results favor ASAP and are consistent with the automatic evaluation results.

Dear Reviewer CXtE,

Thank you again for your detailed review. Since the discussion period ends soon, we want to briefly summarize our responses:

  • Contribution of locality (W1&Q1): The Table 3 ranking baselines use PickScore, following HG-DPO, and their pairs were not checked for anatomical correctness. To separate locality from positive-image curation, we added a controlled comparison. It uses the same HAP positive images, with negatives from the HG-DPO-Easy pipeline. With positives fixed, localized degradation lowers AER from 0.358 to 0.289, and HPSv3 stays similar.

  • Evaluation uncertainty and agreement (W2&Q2): We now report mean ± std over 500 prompts × 5 seeds. We also split human–VLM agreement by task: Absolute is 86.2% (κ\kappa = 0.657) and Relative is 77.8% (κ\kappa = 0.643). We explain how human ratings are aggregated. Finally, we added a direct VLM-judged comparison against standard DPO, where ASAP has a 53.2% win rate, 34.3% tie rate, and 12.5% loss rate.

We hope these additions address your concerns. If anything is still unclear, we would be glad to clarify before the deadline. If your concerns are resolved, we would appreciate it if you would consider updating your assessment.

Best regards,
The Authors

Reviewer_3SWeOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

The paper aims to reduce anatomical artifacts (e.g., malformed hands and faces) in text-to-image human generation through preference alignment. It constructs HAP, a dataset of about 10.6K preference pairs. In each pair, a human-verified synthetic image serves as the preferred sample, and a dispreferred counterpart is created by introducing a localized anatomical error through inpainting or skeleton perturbation. It then fine-tunes generators with a DPO variant that up-weights the degraded region and regresses the preference gap toward a finite margin. The paper also introduces HAF-Bench, a 500-prompt benchmark scored by a VLM judge. On FLUX.1-dev and SDXL, the authors report fewer anatomical errors than the base models and the baselines while preserving global image quality, supported by human evaluation.

[POST REBUTTAL] The authors did a very thorough rebuttal to address my concerns. As such, I have updated my overall recommendation from 2 to 3, and soundness from 2 to 3.

Critical strengths
  1. Controlled, well-validated preference data. Constructing each negative from its own preferred anchor largely isolates anatomy as the source of preference, which the paper verifies :
  • Table 1 shows that pairs are nearly identical outside the degradation masks (SSIM 0.98, LPIPS 0.01) and differ substantially inside them.
  • The construction-strategy comparison (Table 3) holds the prompt set, number of pairs, and optimization fixed. Synthetic localized pairs give the lowest error rate on both backbones.
  • Vanilla DPO on HAP alone reduces FLUX AER from 0.417 to 0.293 (Table 2). The data contribution is arguably the most clearly supported part of the paper.
  1. Consistent evidence across several evaluations. The automated results are corroborated in several independent ways:
  • human-assessed error rates fall from 45.2% to 26.8% on FLUX and from 97.5% to 85.7% on SDXL;
  • pairwise human votes favor ASAP on both backbones;
  • on independently collected Human-Art prompts, FLUX AER falls from 0.223 to 0.176. The DPO β\beta and early-stopping analysis (Fig. 8) also shows that the quality loss of vanilla DPO is not trivially removed by tuning its coefficient or stopping early.
Critical weaknesses

1. The proposed objective contributes mainly to quality preservation, and its benefit depends on hyperparameters apparently selected on the evaluation benchmark.

  • Anatomical gains come largely from the data: ASAP and vanilla DPO on the same pairs reach nearly identical AER: 0.289 vs. 0.293 on FLUX and 0.764 vs. 0.775 on SDXL (Table 2). If AER is computed over 2,500 images (500 prompts × 5 seeds), the FLUX difference is within sampling error (~ 0.009). The objective's demonstrated contribution is recovering global quality (FLUX HPSv3 from 9.680 to 10.360).
  • The components are established: The squared regression toward a finite target margin is the IPO formulation (Azar et al., 2024), which the paper cites. The mask-weighted preference gap is closely related to the patch- and region-aware alignment methods the paper also cites (e.g., PatchDPO, Focus-n-Fix). The novelty of the objective therefore lies in combining them for this setting.
  • The trade-off is narrow and appears tuned on HAF-Bench. Of the five margins tested in Fig. 5, only τ\tau = 0.01 attains both low AER and high HPSv3:
    • at τ\tau = 0.0001, AER equals the base model's (0.417);
    • at τ\tau = 0.001, AER is 0.395;
    • at τ\tau = 0.1 and 1.0, HPSv3 falls to 9.93 and 9.52. This choice appears to have been made on HAF-Bench itself. SDXL uses a different value (τ\tau = 0.002; Appendix D.1) with neither a sensitivity analysis nor a stated selection procedure. Without a separate validation prompt set, the reported quality–anatomy balance may be optimistic.

2. The evaluation relies mainly on a single VLM judge applied to the authors' own benchmark, without uncertainty estimates, and some comparisons are confounded.

  • Judge reliability is not established: AER and RSI come from Gemini-3-Pro. The 82% human agreement figure pools absolute and pairwise labels and is not chance-corrected. High base rates (e.g., 97.5% human-assessed error on SDXL) make raw agreement uninformative. On SDXL, human and automated base-model error rates also differ substantially (97.5% vs. 87.3%).
  • No variance is reported: No standard deviations or confidence intervals are given despite five sampling seeds, although several of the reported differences are small.
  • The SDXL results are confounded by distillation: The preferred anchors are generated by FLUX-ControlNet. On SDXL, supervised fine-tuning on the preferred images alone raises HPSv3 from 6.371 to 7.233 and lowers AER from 0.873 to 0.820. Part of the SDXL improvement may therefore reflect distillation from a stronger generator rather than the preference signal.
  • Baseline comparisons are partly confounded:
    • HG-DPO is reproduced only through its easy-stage construction on the authors' data (Appendix D.2).
    • The Standard Pairwise Ranking strategy in Table 3 orders pairs by an unspecified reward score, so that comparison conflates the label source with pair locality.
  • A possible shortcut is not ruled out. Inpainted negatives may carry inpainting-specific texture or boundary cues. The model could partly learn to avoid such traces rather than anatomical errors. A control with anatomically benign inpainting would address this.
Questions for authors
  1. Hyperparameter selection: On what data were τ\tau (for both backbones), α\alpha, the DPO β\beta, and the number of training steps selected? If HAF-Bench was used, can the authors report results on a separate held-out prompt set?
  2. Uncertainty: How many images contribute to each metric? Please report standard deviations or confidence intervals, and a paired test for the AER difference between ASAP and vanilla DPO on HAP.
  3. Judge reliability: Please report human–judge agreement separately for the absolute and pairwise protocols and for each backbone, with a chance-corrected statistic such as Cohen's κ\kappa. Are the 11 human raters the same annotators who curated HAP, and were they blinded to the method that generated each image?
  4. Inpainting shortcut: Does training with negatives produced by anatomically benign inpainting of the same regions reduce AER? Can results and the locality statistics of Table 1 be reported separately for image-space and skeleton-space negatives? How is the mask M\mathbf{M} defined for skeleton-space pairs, where the whole image is regenerated?
  5. Baseline construction: Which reward model ranks the pairs in Standard Pairwise Ranking and HG-DPO-Easy? How would the full multi-stage HG-DPO compare?
7 comments

We sincerely appreciate the reviewer’s constructive feedback and insightful suggestions. Below, we provide point-by-point responses.


W1-1 & W1-2: Anatomical gains come largely from the data: ASAP and vanilla DPO on the same pairs reach nearly identical AER: 0.289 vs. 0.293 on FLUX and 0.764 vs. 0.775 on SDXL (Table 2). If AER is computed over 2,500 images (500 prompts × 5 seeds), the FLUX difference is within sampling error (~ 0.009). The objective's demonstrated contribution is recovering global quality (FLUX HPSv3 from 9.680 to 10.360).

The components are established: The squared regression toward a finite target margin is the IPO formulation (Azar et al., 2024), which the paper cites. The mask-weighted preference gap is closely related to the patch- and region-aware alignment methods the paper also cites (e.g., PatchDPO, Focus-n-Fix). The novelty of the objective therefore lies in combining them for this setting.

Contribution of the Objective

Our main contribution is synthetic localized preferences, and the main benefit of our objective is better anatomy quality while preserving the global image quality.

We acknowledge that both finite target margins and region-aware weighting build on existing ideas. The contribution of our objective is to adapt and combine these ideas for localized anatomical preferences in diffusion and flow models. The results in Table 4 show further improvements over using the IPO formulation alone. Existing region-aware alignment methods differ in their training objectives and input requirements, which affects how directly they can be applied to our task:

  • PatchDPO (CVPR 2025) uses patch-weighted reconstruction of individual images with a reference image, rather than a pairwise DPO objective. Its patch matching requires a reference image of the subject, which is not available in our general text-to-image setting.
  • Focus-N-Fix (CVPR 2025) uses online optimization with differentiable rewards through DRaFT, rather than offline preference optimization. Applying it to our task would require a suitable differentiable reward for anatomical correctness, including details such as finger counts and joint connections.
  • LocalDPO (CVPR 2026) combines a local masked DPO loss, a global standard DPO loss, and a supervised fine-tuning loss on positive images through a weighted sum. This makes it a relevant baseline for assessing the balance between local anatomical improvements and global image quality.

We therefore added a direct comparison with LocalDPO under the same setting:

MethodAER ↓\downarrowHPSv3 ↑\uparrow
FLUX0.41710.307
FLUX-LocalDPO0.30710.266
FLUX-ASAP (Ours)0.28910.360

In this comparison, FLUX-ASAP achieves both a lower AER and a higher HPSv3 score than FLUX-LocalDPO. These results support the effectiveness of our objective in balancing anatomical correctness and image quality with a simpler formulation.


W1-3 & Q1: The trade-off is narrow and appears tuned on HAF-Bench. Of the five margins tested in Fig. 5, only τ\tau = 0.01 attains both low AER and high HPSv3: at τ\tau = 0.0001, AER equals the base model's (0.417); at τ\tau = 0.001, AER is 0.395; at τ\tau = 0.1 and 1.0, HPSv3 falls to 9.93 and 9.52. This choice appears to have been made on HAF-Bench itself. SDXL uses a different value (τ\tau = 0.002; Appendix D.1) with neither a sensitivity analysis nor a stated selection procedure. Without a separate validation prompt set, the reported quality–anatomy balance may be optimistic.

Hyperparameter selection: On what data were τ\tau (for both backbones), α\alpha, the DPO β\beta, and the number of training steps selected? If HAF-Bench was used, can the authors report results on a separate held-out prompt set?

Hyperparameter Selection

We set β\beta and training steps following standard DPO practices, selected τ\tau and α\alpha on a small held-out validation set, and the additional evaluation on Human-Art provides evidence that this balance generalizes beyond HAF-Bench.

Our selection criterion was to balance anatomical correctness and image quality. Figure 5 and Appendix Figure 8 illustrate this trade-off: increasing the margin can improve AER, but larger margins can also reduce global image quality. To assess how well the selected hyperparameters generalize, we conducted an additional comparison on the Human-Art natural-scene prompt set:

MethodAER ↓\downarrowHPSv3 ↑\uparrow
FLUX0.22310.140
FLUX-ASAP-DPO0.1839.873
FLUX-ASAP (Ours)0.17610.215

On this additional prompt set, FLUX-ASAP achieves a lower AER than both the base model and FLUX-ASAP-DPO, while also achieving a higher HPSv3 score. These results provide further evidence that the observed quality–anatomy balance extends beyond HAF-Bench.

SDXL and FLUX use different margin values because they have different latent spaces and training objectives. In particular, SDXL's noise prediction error and FLUX's flow velocity prediction error differ in scale. We therefore adjusted the margin for each model and added the following sensitivity analysis for SDXL:

SDXL Margin τ\tauAER ↓\downarrowHPSv3 ↑\uparrow
0.020.7496.828
0.0020.7647.450
0.00020.8147.251

Among these tested values, τ=0.002\tau = 0.002 achieves the highest HPSv3 score, while τ=0.02\tau = 0.02 achieves a lower AER at the cost of image quality. This supports our choice of τ=0.002\tau = 0.002 as a balance between the two metrics.


W2-1 & Q3: Judge reliability is not established: AER and RSI come from Gemini-3-Pro. The 82% human agreement figure pools absolute and pairwise labels and is not chance-corrected. High base rates (e.g., 97.5% human-assessed error on SDXL) make raw agreement uninformative. On SDXL, human and automated base-model error rates also differ substantially (97.5% vs. 87.3%).

Judge reliability: Please report human–judge agreement separately for the absolute and pairwise protocols and for each backbone, with a chance-corrected statistic such as Cohen's κ\kappa. Are the 11 human raters the same annotators who curated HAP, and were they blinded to the method that generated each image?

Judge Reliability

The task-specific agreement rates and Cohen’s κ scores support the judge’s reliability on the evaluated outputs.

The previously reported 82% agreement was measured on images generated by FLUX and combined absolute and pairwise judgments. We now provide the results separately for each task, together with Cohen's κ\kappa, which accounts for agreement expected by chance:

TaskAgreementCohen's κ\kappa
Absolute86.2%0.657
Relative77.8%0.643

These results show agreement beyond chance for both tasks and support the use of the VLM as an evaluator on the assessed outputs.

For SDXL, we observed images with broadly plausible outlines but subtle structural errors. Such cases may contribute to the discrepancy between human and VLM judgments.

The human raters were independent from the annotators who curated HAP, and they were strictly blinded to the method that generated each image, with image presentation order randomized.


W2-2 & Q2: No variance is reported: No standard deviations or confidence intervals are given despite five sampling seeds, although several of the reported differences are small.

Uncertainty: How many images contribute to each metric? Please report standard deviations or confidence intervals, and a paired test for the AER difference between ASAP and vanilla DPO on HAP.

Evaluation Uncertainty

We now report mean and standard deviation over the evaluation results, then conduct pairwise comparison with standard DPO.

All evaluations use 500 prompts and 5 seeds. To better characterize the variability of the results, we now report the mean and standard deviation:

MethodsAER ↓\downarrowHPSv3 ↑\uparrow
FLUX0.417 ±\pm 0.01110.307 ±\pm 0.224
FLUX-ASAP-DPO0.293 ±\pm 0.0039.680 ±\pm 0.163
FLUX-ASAP (Ours)0.289 ±\pm 0.00510.360 ±\pm 0.186

Compared with the FLUX base model, FLUX-ASAP achieves a lower anatomical error rate while maintaining a similar HPSv3 score. Compared with FLUX-ASAP-DPO, it achieves a similar AER and a higher HPSv3 score, indicating better image quality as measured by HPSv3.

We also conducted a direct pairwise comparison between ASAP and standard DPO using the VLM judge. ASAP achieved a 53.2% win rate, 34.3% tie rate, and 12.5% loss rate. These results favor ASAP and are consistent with the automatic evaluation results.


From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.