acceptodds
Withdrawn from ICLR 2027

RePair: Mining Multi-Directional Regional Preferences from Imperfect Pairs

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Post-training plays a crucial role in advancing generative models, yet acquiring reliable reward signals is costly and sparse. Conventional approaches evaluate an entire image with a global reward, implicitly assuming that artifacts are evenly distributed. However, modern text-to-image models usually produce localized failures rather than global degradation. To supervise local errors, recent preference construction heavily rely on manually designed contrasts, which inherently assume that all regional preferences point toward a target image, imposing a restriction we term the unidirectional preference constraint. Instead, we explore a different source of supervision by leveraging the intrinsic stochasticity of text-to-image models, where local successes and failures are often mixed, formalizing this rich signal as multi-directional regional preferences. Building on this insight, we introduce RePair, which transforms cross-generation variance into reliable regional supervision. First, correlated sampling creates candidate images where global structures remain aligned while local differences are preserved. Second, visual grounding directly mines verified regional preferences without requiring artificial editing. Finally, we optimize each regional relation independently using a signed, region-normalized objective, allowing a single image pair to simultaneously provide multiple preference directions. Extensive experiments on FLUX.2 and Ideogram 4 demonstrate that RePair, trained on only 5K automatically mined pairs, achieves substantial improvements across compositional benchmarks.

Primary area:
generative models
Keywords:
Text-to-Image Generation, Preference Optimization, Regional Preference Learning, Imperfect Generations

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_4kw2OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

With DPO, image pairs composed of a winner (preferred, to which the model must be stirred) and losing (less preferred) image are formed, then the model weights are adapted towards the whole winner image. Nevertheless, this is not optimal as the losing image might contain some parts which are better than the winner. The paper addresses this shortcoming. The framework starts by generating images from the same shared noise and thus aiming to obtain locally aligned images. Then, by leveraging GPT-5.6-Sol VLM, it assess each image on multiple failure criteria (e.g. object presence, counting, color etc.) on which image pairs can later be compared. A final verification step is done to make sure the same region is found in the two images. In the end, regional optimization is achieved employing a loss that takes into account the soft-masked region.

Critical strengths

The paper starts from an obvious problem when employing DPO (an image might not be fully better than another one) and proposes an automatic framework to resolve this. Each component is well-defined, with a clear motivation. The paper addresses the main objective with experiments on multiple benchmarks and two models.

Critical weaknesses
  1. The main weakness of the paper is the baselines in compares to. There are other region-aware DPO methods (a few examples found in Related Work are [1], [2], [3]) which are more suitable than a SFT baseline. Furthermore, when compared to method [2], the novelty is slightly incremental. This would also lead to a strong ablation study that isolates the proposed objective -> how is the performance when training with the default DPO loss on the mined samples (using a simple heuristic for selecting the winning sample i.e. larger number of preferred attributes) - this is potentially covered by Vanilla DPO baseline but there is no information how it was obtained
  2. Some results in Table 1 are very close to Vanila DPO -> in order to attest the method's performance, some statistical tests / multiple runs are required ~ Minor ~
  3. Table 4, Alighnment column: Full RePair is bold, but Uni-Directional Only has a higher score.
  4. Text could be simplified, few parts that were hard to follow, also due to wording e.g. "Since prevailing post-training paradigms Wallace et al. (2024); Liu et al. (2025) propagate a global preference to all semantic components."
  5. References are very inconsistent: some of them mention the journal/article, some of them not.
  6. Not enough information for reproducibility on VLMs calling

[1] - Xing, X., Saha, A., He, J., Hao, S., Vicol, P., Ryu, M., Li, G., Singla, S., Young, S., Li, Y. and Yang, F., 2025, June. Focus-n-fix: Region-aware fine-tuning for text-to-image generation. In 2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (pp. 18486-18496). IEEE. [2] - Oh, Y., Kim, Y., Kim, H., Chi, D. and Kim, S., 2026. OSPO: Object-Centric Self-Improving Preference Optimization for Text-to-Image Generation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (pp. 7620-7631). [3] - Zhuohan Liu, Wujian Peng, Yitong Chen, and Zuxuan Wu. Compositional text-to-image generation via region-aware bimodal direct preference optimization. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 36604–36614, June 2026.

Questions for authors
  • How were the SFT and Vanilla DPO baselines obtained?
  • How does your method compared to other region-aware DPO methods?
  • Do the results statistically stand?
2 comments
AuthorsOpenReview ↗

Response to Reviewer 4kw2 (1/2)

Response to Question1. How were the SFT and Vanilla DPO baselines obtained?

We thank the reviewer for pointing out the insufficient baseline details. For SFT, we select the best generation from each rollout and use GPT-Image2 to correct its remaining errors, obtaining refined training samples optimized with the standard flow-matching loss.For Vanilla DPO, we use the exact same image pairs mined by RePair and assign global preference labels using the suggested heuristic. The image with more correctly rendered regions is preferred. If tied, we compare the sum of VLM-estimated error severities across incorrect regions. These pairs are then optimized using the standard Flow-DPO objective. This provides the controlled comparison suggested by the reviewer, isolating the benefit of region-specific preference supervision over a single global winner–loser label on identical image pairs. Both baselines follow the backbone and training configurations described in Appendix C.1. We will clarify these details in the revised manuscript.

Response to Question2. Compared to other region-aware DPO methods

We thank the reviewer for suggesting stronger region-aware baselines. As discussed in Related Work, existing methods differ from RePair in their supervision sources and preference construction strategies. However, existing methods are either tailored to specific architectures, making faithful adaptation to our flow-matching backbone challenging (e.g., OSPO targets autoregressive multimodal models), or lack publicly available implementations (e.g., Focus-N-Fix). We therefore select BiDPO, which provides open-source code and training data, as a representative region-aware baseline. We reproduce BiDPO using the same FLUX.2 backbone and evaluation protocol as RePair. The results are shown below:

MethodGenEval ↑DPG-Bench ↑OneIG-Bench (EN) ↑
FLUX.2-klein-base-9B (Base)0.8385.410.547
BiDPO (region-aware DPO)0.8485.720.545
RePair (Ours)0.8786.290.576

RePair consistently outperforms BiDPO across all three benchmarks, demonstrating the advantage of our region-specific multi-directional preference optimization. We will include these comparisons in the revised manuscript.

Response to Question3 & Weakness2. Some results in Table 1 are very close to Vanilla DPO. Do the results statistically stand?

We thank the reviewer for this suggestion. We include Ideogram 4 to evaluate RePair on a recent, high-performing model. Its strong baseline performance (e.g., 0.92 in prompt alignment and 0.97 in text rendering) leaves limited room for improvement. Nevertheless, RePair improves all four evaluation metrics, whereas Vanilla DPO degrades three of them relative to the base model. To further check the reproducibility of these results, we repeated the Ideogram 4 evaluation once and observed consistent results. We also clarify that the four metrics in Table 1 are evaluated on four separate benchmarks, rather than subsets of a single benchmark. Moreover, experiments on the weaker SDXL[1] backbone demonstrate substantially larger gains (+0.10 GenEval and +5.23 DPG-Bench over Vanilla DPO**). These additional results support the robustness and cross-backbone effectiveness of RePair.

MethodGenEval ↑DPG-Bench ↑
SDXL (Base)0.5373.38
Vanilla DPO0.5575.13
BiDPO (region-aware DPO)0.6278.84
RePair (Ours)0.6580.36

Reference [1] Dustin Podell et al. SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesis. In The Twelfth International Conference on Learning Representations (ICLR), 2024.

AuthorsOpenReview ↗

Response to Reviewer 4kw2 (2/2)

Response to Weakness1. Compared to OSPO, the novelty is slightly incremental

We thank the reviewer for highlighting this concern. Although both methods exploit object-level or regional information, their sources and construction of preference supervision are fundamentally different. OSPO constructs object-centric preference pairs through prompt perturbation, evaluates object-level quality using VQA, and applies object-weighted SimPO. In contrast, RePair directly mines regional preferences from naturally generated correlated rollouts, without artificially constructing prompt-level contrasts. Specifically, correlated generation produces comparable candidates, while VLM-based regional analysis and cross-image comparison identify complementary local successes and failures. RePair then uses a mixed set of region-specific preference signals, preserving their individual directions during optimization rather than reducing them to a single global ranking. The combination of correlated generation and multi-directional regional optimization is central to RePair's contribution. It provides a way to learn from the natural diversity of model-generated samples, making post-training better aligned with how generative models actually produce successes and failures.

Response to Weakness3-Weakness6

We thank the reviewer for pointing out these issues. We will address them in the revised manuscript as follows:

W3. Table 4. We will correct the bold formatting in the Alignment column to highlight the highest score.

W4.Clarity. We will revise the manuscript throughout to simplify lengthy sentences, replace ambiguous expressions, and improve overall readability. In particular, the highlighted sentence will be revised as follows:"Most existing post-training methods assign a single image-level preference to the entire generated image (Wallace et al., 2024; Liu et al., 2025)."

W5. References. The inconsistency arises from BibTeX entry types for arXiv preprints. We will correct these entries to ensure consistent display of publication information in the revised manuscript.

W6.Reproducibility. We will clarify the VLM calling protocol and provide the complete system prompts and user-message templates in the supplementary material.The system prompts and user-message templates are summarized below:

  • Annotation — system prompt: Act as a strict image-quality auditor. Identify visible failures, group findings by semantic region, and return their bounding boxes, error types, descriptions, severity, and confidence in JSON format. Report only errors supported by visible evidence.
  • Annotation — user message: Inspect the supplied image against the generation prompt and semantic target references, and localize the detected failures.
  • Pairwise verification — system prompt: Act as an independent regional-preference verifier. Judge each proposed region against the prompt requirements, returning A-over-B, B-over-A, or tie, together with quality scores, confidence, and supporting evidence. Use tie for small or uncertain differences and keep the supplied bounding boxes fixed.
  • Pairwise verification — user message: Compare images A and B within the proposed regions using the prompt, target requirements, and corresponding crops. Determine regional preferences independently, without access to provisional preference directions or previous failure descriptions.
Reviewer_ZssdOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes RePair for text-to-image preference learning. It argues that two imperfect generations may exhibit complementary local strengths, which cannot be fully represented by a single image-level preference. The method combines correlated sampling, VLM-based regional preference mining and verification, and a signed region-wise preference objective. The authors evaluate it on FLUX.2-klein-base-9B and Ideogram 4.

Critical strengths
  • This paper is well written and easy to follow.
  • The motivation is reasonable. Modern text-to-image models often make localized errors, so a global winner may discard useful information about regions where the losing image performs better.
Critical weaknesses
  1. The methodological novelty is limited. RePair combines correlated sampling, VLM-based filtering, and a region-wise preference loss in a sensible way, yet I do not see an insight that goes beyond putting these pieces together. The paper would be stronger if it explained when this combination is necessary and why simpler regional approaches would not suffice.
  2. The empirical gains are modest, particularly for the proposed core idea. In the main tables, most Ideogram 4 gains are 0.01–0.03, while the FLUX.2 overall gains are 0.04 on GenEval, 0.88 on DPG-Bench, and 0.029 on OneIG-Bench EN. The ablations are also relatively weak: Full RePair exceeds “Uni-directional Only” by 0.009 and the variant without consistency verification by 0.007 on a 100-prompt subset. Removing correlated generation has a larger effect, making it unclear how much of the improvement is attributable specifically to multi-directional regional learning.
  3. The baselines are insufficiently specified and incomplete. The paper does not clearly describe the training data, preference-label construction, and training settings for SFT and vanilla DPO, making it difficult to assess whether those comparisons are fair. More importantly, although the paper discusses previously published region-level preference methods, it does not compare RePair against them under matched backbones, data, and training budgets. This is a significant omission for a paper whose main claim concerns regional preference learning.
Questions for authors
  • What is the key generalizable insight beyond combining correlated generation, VLM-based filtering, and a region-wise preference loss? Can the authors characterize conditions under which bidirectional regional supervision should outperform existing regional approaches?
  • Can RePair be compared with representative published region-level preference methods using the same base model and a matched training and evaluation protocol?
2 comments
AuthorsOpenReview ↗

Response to Reviewer Zssd (1/2)

Response to Question1&Weakness1. The key generalizable insight beyond combining correlated generation, VLM-based filtering, and a region-wise preference loss

We thank the reviewer for this question. Our key insight is a supervision bottleneck in compositional generation: globally superior samples become increasingly difficult to obtain as semantic constraints grow, while locally correct components remain available across imperfect rollouts. Compressing these complementary signals into a single winner–loser ranking can discard useful supervision and introduce incorrect regional optimization directions. RePair addresses this through region-specific preference learning, allowing imperfect generations to supervise each other's weaker regions without requiring globally superior samples. Correlated generation enables reliable preference discovery, while multi-directional optimization preserves region-specific preference directions, including conflicting directions when present. Their combination follows this learning principle rather than merely assembling existing components. This advantage is particularly relevant when complex prompts impose multiple interacting semantic constraints, as the resulting generations often exhibit heterogeneous errors across different image regions. Appendix B (Proposition 1) explains the resulting gradient conflicts under global preference optimization. Table 5 further shows that utilizing more regional preferences per pair improves overall performance (0.530 → 0.542), supporting the value of richer supervision from imperfect samples.

Response to Question2. Comparison for representative published region-level preference methods

We thank the reviewer for this suggestion. As detailed in the table, we reproduce BiDPO[1] using the same FLUX.2-klein-base-9B backbone, matched adaptation capacity, and identical evaluation protocols.BiDPO relies on constructed image-caption preference pairs with regional annotations, whereas RePair directly mines signed regional preferences from natural model rollouts. Enforcing identical training data would require additional annotation or data construction, introducing new assumptions. Notably, RePair uses only approximately 5K naturally mined image pairs, compared with roughly 10 times as many constructed training examples for BiDPO, yet achieves better performance across all three benchmarks. Thus, the BiDPO experiment provides a comparison of complete methods under matched model and evaluation settings, while our existing Vanilla DPO baseline provides a controlled comparison using the same RePair-mined image pairs.

MethodGenEval ↑DPG-Bench ↑OneIG-Bench (EN) ↑
FLUX.2-klein-base-9B (Base)0.8385.410.547
BiDPO (region-aware DPO)0.8485.720.545
RePair (Ours)0.8786.290.576

Response to Weakness2. The empirical gains are modest, particularly for the proposed core idea

We thank the reviewer for this observation. Correlated generation and multi-directional optimization are not independent components, but two successive stages of RePair. Correlated generation provides comparable candidates from which complementary regional preferences can be reliably identified. Without it, differences across generated images become less attributable to specific regions, making regional preference mining less reliable. Multi-directional optimization then preserves and learns from the discovered regional preferences, rather than collapsing them into a single global preference direction. Thus, correlated generation provides the necessary supervision for multi-directional optimization, and the two work together to enable RePair's learning paradigm. Regarding the modest overall gains, the strong initial performance of FLUX.2 may limit the room for improvement. As detailed in the table, additional experiments on the weaker SDXL backbone show substantially larger gains over Vanilla DPO and BiDPO[1], further supporting this explanation.

MethodGenEval ↑DPG-Bench ↑
SDXL (Base)0.5373.38
Vanilla DPO0.5575.13
BiDPO (region-aware DPO)0.6278.84
RePair (Ours)0.6580.36

Reference

[1] Zhuohan Liu et al. Compositional text-to-image generation via region-aware bimodal direct preference optimization. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 36604–36614, 2026.

AuthorsOpenReview ↗

Response to Reviewer Zssd (2/2)

Response to Weakness3.The baselines are insufficiently specified

We thank the reviewer for this concern. For SFT, we select the best generation from each rollout and use GPT-Image2 to correct its remaining errors. The refined samples are then used for training with the standard flow-matching loss. For Vanilla DPO, we use the exact same image pairs mined by RePair and assign global preference labels using a deterministic heuristic. The image with more correctly rendered regions is preferred, with ties broken by the sum of VLM-estimated error severities across incorrect regions. These pairs are then optimized using standard Flow-DPO. Importantly, Vanilla DPO provides a controlled comparison that isolates the benefit of region-specific preference supervision over global preference ranking on identical image pairs. Both baselines follow the backbone and training configurations described in Appendix C.1. We will clarify these details in the revised manuscript.

Reviewer_4awnOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This work is about post-training text-to-image generative model for better compositional generation. They focus that modern generative models can generate globally plausible images, but then often fails for localized feature, that was originally described in the text. Given a pair of image, they use pre-trained VLM (GPT 5.6 Sol) to annotate which generated image is better aligned with the given text, and give relative ranking to the pairs. Using the annotated ranking, they use flow-DPO to fine-tune the generative model. They also propose sampling method for noise for higher correlation. Experiments show some comparisons with SFT and global (vanilla) DPO with their proposed method.

Critical strengths

The problem statement is clear and seems sound. For modern generative models, I agree that post-training is one nice direction for refining the ability to generate samples that are more aligned to the localized features, described in the prompt. First obtaining the ranking within a pair of image, and fine-tuning the generative model with DPO style loss seems plausible.

Critical weaknesses

Some of the experimental section lacks to demonstrate the significance of the proposed method. For example, RePair seems to be not significantly gaining improve over vanilla DPO, even while vanilla DPO only looks into global alignment and does not see the alignment of the localized feature. Authors should include standard deviation across different runs or seeds and convince the improvement is statistically valid. Also, the comparisons should be made with other region-level preference learning methods. Current baselines for comparison is too weak for correct evaluation of the proposed work's significance.

Additionally, the proposed method relies on pre-trained VLMs and existing flow-DPO techniques, making it difficult to identify a sufficiently novel or compelling contribution.

Questions for authors

How does the method compare with GRPO-based methods and other region-level preference learning methods? How does this also compare with the inference-time steering methods that does not require post-training? Can the proposed method be generalized to preference learning involving more than two images, such as triplets or larger groups?

2 comments
AuthorsOpenReview ↗

Response to Reviewer 4awn (1/2)

Response to Question1. Baselines for comparison

We thank the reviewer for suggesting stronger baselines. We extend our evaluation by including BiDPO [1] for region-aware DPO, BoxFlow-GRPO [2] for reward-based RL post-training, and RewardFlow [3] for training-free inference-time steering. All methods are evaluated using the same FLUX.2-klein-base-9B backbone and evaluation protocols across GenEval, DPG-Bench, and OneIG-Bench EN.

MethodGenEval ↑DPG-Bench ↑OneIG-Bench (EN) ↑
FLUX.2-klein-base-9B (Base)0.8385.410.547
BiDPO (region-aware DPO)0.8485.720.545
BoxFlow-GRPO (region-aware GRPO)0.8285.400.551
RewardFlow (inference-time)0.8286.010.550
RePair (Ours)0.8786.290.576

As demonstrated in the table, RePair consistently achieves superior performance across all three evaluation benchmarks, highlighting the effectiveness of our proposed multi-directional regional preference learning.

Response to Question2. Preference learning involving more than two images

The pairwise formulation is adopted to align with standard DPO, rather than being an inherent limitation of RePair. Our pipeline already generates multiple correlated candidates per prompt, naturally supporting an extension to larger groups. For a group of images {I1,…,IN}\{I_1,\ldots,I_N\}, different regions can exhibit distinct preference rankings, e.g., R1:IA≻IB≻ICR_1:I_A\succ I_B\succ I_C, while R2:IC≻IA≻IBR_2:I_C\succ I_A\succ I_B. These rankings can be decomposed into region-specific pairwise relations and optimized using our existing signed objective:

Lgroup=−1∣P∣∑(i,j,k)∈Plog⁡σ(βyij(k)mij(k))L_{group} = -\frac{1}{|P|}\sum_{(i,j,k)\in P}\log\sigma(\beta y_{ij}^{(k)}m_{ij}^{(k)})

where P\mathcal P denotes the set of verified regional comparisons, yij(k)y_{ij}^{(k)} the preference direction, and mij(k)m_{ij}^{(k)} the corresponding regional margin. This enables preference learning from larger groups without imposing a single global ranking, while preserving region-specific preference directions.

Response to Weakness1. RePair seems to be not significantly gaining improve over vanilla DPO

We thank the reviewer for this observation. The relatively modest gains on FLUX.2 may partly reflect its strong initial performance, leaving less room for improvement. To further validate the effectiveness of RePair, we extend our experiments to SDXL[4], a comparatively weaker backbone.

MethodGenEval ↑DPG-Bench ↑
SDXL (Base)0.5373.38
Vanilla DPO0.5575.13
BiDPO (region-aware DPO)0.6278.84
RePair (Ours)0.6580.36

As shown in the table, RePair substantially outperforms Vanilla DPO by +0.10 on GenEval and +5.23 on DPG-Bench On SDXL, while also surpassing BiDPO[1] on both benchmarks. The improvements are more pronounced on SDXL, where the base model has greater room for improvement. We will include these additional experiments in the revised manuscript.

Reference

[1] Zhuohan Liu et al. Compositional text-to-image generation via region-aware bimodal direct preference optimization. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 36604–36614, 2026.

[2] Xiaoying Xing et al. Focus-n-fix: Region-aware fine-tuning for text-to-image generation.arXiv preprint arXiv:2501.06481, 2025.

[3] Huaisong Zhang et al. Where, what, why, and importance:Structured defect grounding for text-to-image feedback. arXiv preprint arXiv:2606.06113, 2026.

[4] Dustin Podell et al. SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesis. In The Twelfth International Conference on Learning Representations (ICLR), 2024.

AuthorsOpenReview ↗

Response to Reviewer 4awn (2/2)

Response to Weakness2. Multi-seeds evaluation

We thank the reviewer for this suggestion. To evaluate the robustness of our results, we conducted experiments with three independent training seeds for both Vanilla DPO and RePair on FLUX.2-klein-base-9B, using identical training configurations.

MethodSeedGenEval ↑DPG-Bench ↑OneIG-Bench EN ↑
Vanilla DPO (Mean ± Std)42, 123, 20260.833 ± 0.00785.47 ± 0.210.549 ± 0.007
RePair (Mean ± Std)42, 123, 20260.873 ± 0.00686.36 ± 0.150.577 ± 0.008

As shown in the table, RePair consistently outperforms Vanilla DPO across all three benchmarks, with improvements of +0.040, +0.89, and +0.028, respectively. The relatively small standard deviations across independent runs further demonstrate that these gains are robust to training randomness rather than being artifacts of a particular run. We will include these multi-seed results in the revised manuscript.

Response to Weakness3. The proposed method relies on pre-trained VLMs and existing flow-DPO techniques

We thank the reviewer for this concern. The novelty of RePair lies in exploiting a previously underutilized source of supervision: complementary local successes across naturally generated imperfect samples. Due to the stochastic nature of generation, different rollouts often exhibit different local successes and failures, rather than being uniformly good or bad. RePair turns this inherent diversity into fine-grained preference supervision by extracting multiple region-specific preferences from the same image pair. Importantly, correlated generation and multi-directional optimization jointly enable this learning paradigm. The former facilitates the discovery of complementary regional preferences, while the latter preserves their individual directions during optimization. Rather than simply applying existing Flow-DPO techniques to VLM-identified regions, RePair offers a post-training approach that better reflects the characteristics of generative models and exploits the natural diversity of their rollouts. We believe this perspective can also benefit future post-training methods by making better use of the different strengths and errors found across a model's own generations.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.