acceptodds
Under review as a conference paper at ICLR 2027

Learning from Model Failures Distilled as Executable Visual Programs

Abstract

Text-to-image (T2I) models can often satisfy a generation requirement on some inputs while failing on closely related ones, suggesting that useful capabilities may already be present but are expressed inconsistently. Preference-based alignment provides a natural mechanism for reinforcing desired generations and discouraging undesirable ones. However, its effectiveness depends not only on how preferences are optimized, but also on the informativeness of the preferred–rejected contrast. We introduce Failure-Guided Preference Supervision (F-GPS), a framework that constructs preference supervision from failures observed in the target T2I model. Rather than treating failures only as diagnostic signals, F-GPS distills recurring failure patterns into reusable programs: it atomizes observed coarse failure instances into fine-grained visual factors, aggregates these factors across samples into a candidate inventory, filters the inventory into reusable atomic visual cues, and operationalizes the retained cues as parameterized failure programs. A randomly sampled composition of these failure programs is then applied to an editable preferred example to construct a rejected example with targeted deviations, yielding an informative preference contrast. We study F-GPS on image composition and visual text rendering, two settings exhibiting distinct forms and granularities of generation failure. Across multiple T2I backbones and preference-optimization objectives, F-GPS improves downstream alignment, with gains of up to 5.6 pp for image composition and 10.4 pp for visual text rendering over SFT. These results suggest that failure patterns observed in the target model can provide targeted supervision for preference-based T2I alignment.

Primary area:
generative models
Keywords:
Diffusion Models, Preference alignment

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_iki6OpenReview ↗
Rating
2
Confidence
4
Informativeness
1
Presentation
1
Soundness
2
Read review
Paper summary

This paper proposes Failure-Guided Preference Supervision (F-GPS). It analyzes the generation failures of the target text-to-image model and distills recurring error patterns into executable visual programs. These programs are then used to construct Rejected Images with targeted biases, providing more effective preference supervision for Diffusion Preference Optimization. The method achieves fairly consistent performance improvements on image composition generation and visual text rendering tasks.

Critical strengths
  1. The problem is clearly defined.

  2. The experimental details are clearly described.

Critical weaknesses
  1. The novelty of the paper is very limited. In essence, it is a reorganization of the data used in preference optimization learning. The only new part is organizing recurring and reusable failure cases, but all the methods involved are existing techniques.

  2. The paper uses GPT-4o to analyze and interpret failure cases. However, is such interpretation really reliable? Could other models also work, and would more advanced models be better? Does this interpretation consider all possible visual elements?

Questions for authors
  1. The paper contains statements such as "The contribution therefore lies not in rendering or compositing itself, but in distilling observed model failures into reusable programs for constructing preference supervision". I do not know what such statements are trying to emphasize.

  2. The paper essentially reorganizes failure cases and forms reusable programs. Can the corresponding successful cases also be processed in this way? Why must it be failure cases?

  3. The readability of the paper is poor. There are many borrowed concepts that seem plausible but are not clearly defined. Such expressions make it unfriendly for readers to understand the core intent of the authors. In addition, the figures and the organization of the presentation rely too much on LLM tools, but the effect of the presentation is not good either.

  4. For the evaluation using VLM-as-a-Judge, the paper relies on a single model, Qwen3-VL-8B-Instruct. Such evaluation is not reliable enough, especially because human trustworthiness assessment is missing.

  5. Based on the experimental results and the corresponding visualizations, why do the basic preference optimization methods sometimes perform worse than the base model?

  6. Are the failure programs transferable? Can the failure programs extracted from failure cases on SD v1.5 be applied to other models?

11 comments

Reviewer iki6

We sincerely thank the reviewer for their careful evaluation of our work, for acknowledging the clearly defined problem and detailed experimental setup, and for providing constructive feedback. We have strengthened the manuscript with additional experiments, analyses, and clarifications addressing the reviewer's concerns.

1. Methodological novelty and contribution of failure-guided preference supervision

We apologize that the intended methodological contribution was not sufficiently clear in the original presentation. We would like to emphasize that F-GPS aims to provide the target model with preference supervision that explicitly reflects its own observed failures, rather than simply constructing additional generic negative examples. Our motivation is to enable the model to learn not only what a preferred image should look like, but also which specific visual deviations, corresponding to its observed weaknesses, should be avoided.

To achieve this, F-GPS (i) identifies sample-specific failures in the target model's generations, (ii) atomizes and consolidates recurring failure patterns into reusable atomic visual cues, (iii) operationalizes these cues as parameterized executable programs, and (iv) applies compositions of these programs to editable preferred examples. This enables the construction of controlled preferred–rejected contrasts that deliberately expose the model to the types of mistakes identified in its own generations.

Compared with approaches such as D-Fusion, OSPO, and RealAlign, our distinguishing contribution is therefore not the rendering or compositing mechanism itself, but the distillation of target-model failures into a reusable executable failure-program library for constructing informative preference supervision.

To substantiate this contribution, we have added component ablations investigating both the source of failure cues and the mechanism used to construct rejected examples. We compare predefined visual perturbations, LLM-suggested failures, F-GPS-identified failures with model-based editing, and the complete F-GPS pipeline. All configurations use the same preferred images, conditioning prompts, and SDXL Diffusion-DPO optimization settings.

Table 12: Component ablation for image composition (Appendix I.5).

MethodT2I-CompBench++ BLIPT2I-CompBench++ GITTIFA BLIPTIFA GIT
Base78.0875.3088.6087.20
Predefined Visual Perturbations + Executable Programs71.7174.0474.9173.03
LLM-Suggested Failures + Model-Based Editing43.8430.8640.7833.22
F-GPS-Identified Failure Cues + Model-Based Editing60.4952.5461.7353.14
F-GPS-Identified Failure Cues + Executable Programs81.5579.3092.3090.80

Table 13(a): Component ablation on CVTG (visual text rendering).

MethodEMNEMPSSWPWRWF1
Base24.227.077.950.150.948.6
Predefined Visual Perturbations + Executable Programs7.67.874.257.032.838.1
LLM-Suggested Failures + Model-Based Editing16.016.070.149.528.433.0
F-GPS-Identified Failure Cues + Model-Based Editing6.06.067.743.421.626.3
F-GPS-Identified Failure Cues + Executable Programs32.032.881.358.259.056.8

Table 13(b): Component ablation on MARIO-Eval (visual text rendering).

MethodEMNEMPSSWPWRWF1
Base0.80.839.83.13.73.1
Predefined Visual Perturbations + Executable Programs20.823.875.642.443.841.0
LLM-Suggested Failures + Model-Based Editing14.416.664.128.926.526.4
F-GPS-Identified Failure Cues + Model-Based Editing15.417.264.031.328.328.2
F-GPS-Identified Failure Cues + Executable Programs24.926.176.848.146.745.8

Tables 12–13 use SDXL-Base-1.0 with Diffusion-DPO. Model-Based Editing denotes instruction-guided editing with FLUX.2-klein-4B.

The complete F-GPS pipeline achieves the best results across the reported composition and text-rendering metrics. In particular, predefined executable perturbations substantially underperform F-GPS, suggesting that the source of visual deviations matters beyond the use of executable transformations alone. Similarly, using F-GPS-identified failure cues with model-based editing performs worse than the complete pipeline, highlighting the importance of how the identified failures are operationalized.

To better understand these differences, we additionally evaluate 500 matched preferred–rejected pairs per method and task using GPT-6.1 Sol with medium reasoning effort. The evaluation considers four criteria: intended alteration realization, new prompt violations, unrequested changes, and whether the rejected image is visually better than the preferred image.

Table 14(a): Negative-example validation for image composition.

Evaluation criterionNegative constructionYes (%)No (%)Unclear (%)
All intended changes visibly present?LLM-Suggested Failures + Model-Based Editing0.498.41.2
F-GPS-Identified Failure Cues + Model-Based Editing3.690.26.2
F-GPS-Identified Failure Cues + Executable Programs80.815.83.4
New clear prompt violation?LLM-Suggested Failures + Model-Based Editing2.463.833.8
F-GPS-Identified Failure Cues + Model-Based Editing27.452.420.2
F-GPS-Identified Failure Cues + Executable Programs58.220.421.4
Unrequested changes?LLM-Suggested Failures + Model-Based Editing82.018.00.0
F-GPS-Identified Failure Cues + Model-Based Editing78.220.41.4
F-GPS-Identified Failure Cues + Executable Programs10.289.20.6
Rejected image visually better?LLM-Suggested Failures + Model-Based Editing7.092.60.4
F-GPS-Identified Failure Cues + Model-Based Editing0.299.60.2
F-GPS-Identified Failure Cues + Executable Programs0.299.60.2

Table 14(b): Negative-example validation for visual text rendering.

Evaluation criterionNegative constructionYes (%)No (%)Unclear (%)
All intended changes visibly present?LLM-Suggested Failures + Model-Based Editing2.695.61.8
F-GPS-Identified Failure Cues + Model-Based Editing6.290.03.8
F-GPS-Identified Failure Cues + Executable Programs84.47.48.2
New clear prompt violation?LLM-Suggested Failures + Model-Based Editing26.434.039.6
F-GPS-Identified Failure Cues + Model-Based Editing36.632.431.0
F-GPS-Identified Failure Cues + Executable Programs51.623.025.4
Unrequested changes?LLM-Suggested Failures + Model-Based Editing54.845.20.0
F-GPS-Identified Failure Cues + Model-Based Editing71.628.20.2
F-GPS-Identified Failure Cues + Executable Programs0.0100.00.0
Rejected image visually better?LLM-Suggested Failures + Model-Based Editing17.482.60.0
F-GPS-Identified Failure Cues + Model-Based Editing10.489.60.0
F-GPS-Identified Failure Cues + Executable Programs0.0100.00.0

Table 14 reports Yes/No/Unclear judgments; lower Yes is desirable for unrequested changes and visually better negatives. The complete evaluation protocol is described in Appendix J.2.

These additional analyses show that executable F-GPS programs realize the intended alterations in 80.8%/84.4% of composition/text-rendering pairs, compared with 3.6%/6.2% for model-based editing using the same F-GPS-identified failure cues. They also introduce clearer prompt violations with substantially fewer unrequested changes.

Taken together, the downstream performance ablations and negative-example validation provide complementary evidence for the importance of both model-specific failure information and controlled executable negative construction. This supports our central motivation: rather than presenting the model with arbitrary rejected examples, F-GPS constructs preference contrasts that explicitly reflect the types of failures observed in its own generations.

We have also clarified the corresponding methodological statements in the revised manuscript (Appendix I.5, Tables 12–14).

Reviewer_Be99OpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper presents a method to construct preferences for DPO-style preference tuning in diffusion models, specifically for text-image generation. The goal here is to be able to construct informative preferences which can guide the model well, especially for visual composition and text rendering.

The paper proposes a strategy where GPT-4o is used to identify failures and, from that, come up with a set of operations which can be used to create informative negatives. Experiments on various preference optimization methods indicate that there are notable gaps, especially in text rendering and compositional image generation capabilities of the SDXL and SD 3.5 medium models.

Critical strengths

The overall method is extremely sound and intuitive (i.e finding better samples for preference optimization), yet has somewhat been neglected in the literature.

The experimental results are also quite compelling (even if on slightly older models), the performance improvements are certainly quite significant.

Critical weaknesses

To me, the biggest weakness I see in the paper is that there's not sufficient justification for why the failures actually matter. For instance, Page 23 in the appendix shows the specific corruptions used, and they seem quite general and don't need specific failure identification to just randomly apply those. While the baselines in the experimental results test different alternatives, there's no test where the negative follows the exact pipeline except without the failure identification (as far as I can understand). To me, proving this claim would be critical in justifying the acceptance of the paper.

Slightly more secondary (and not as critical) is that SDXL and SD3.5-Medium are not really used today. While I can understand that newer models have been post-trained to a large extent and don't really need these techniques (especially post release), it would certainly be nicer to see results on newer models.

Questions for authors
  1. Effect of Failure Guidance: As mentioned in the weakness, would it be possible to see ablations where F-GPS is run, except without the specific failure identification? This would prove the efficacy of the method conclusively in my view.

  2. How is this relevant for RL (i.e Flow-GRPO/Diffusion-NFT style optimization methods) approaches: In recent times, directly optimizing a reward model has shown greater efficacy than the approaches trying to directly optimize with pairwise preferences? Would curating better preference samples then be useful for training Reward Models or something else? Or is direct preference optimization still relevant even with these alternative post-training approaches?

I'm currently rating the paper at Weak Rejection (closer to borderline in my view) and would be happy to flip it to weak accept if these issues are addressed.

EDIT: Post-rebuttal, on account of the key comparison on F-GPS vs generic visual corruptions, I am increasing my score.

4 comments

Reviewer Be99

We sincerely thank the reviewer for their encouraging assessment of our work, for recognizing the significance of the experimental improvements, and for providing valuable suggestions to strengthen the paper. We have conducted additional component ablations and analyses to address the reviewer's concerns.

1. Contribution of target-model failure identification compared with generic visual corruptions

We thank the reviewer for highlighting the need to establish the contribution of target-model-specific failure identification. To investigate this, we have introduced component ablations that vary both the source of failure cues and the mechanism used to construct rejected examples.

Specifically, we compare four configurations: (i) predefined visual perturbations implemented as executable programs without target-model failure analysis, (ii) LLM-suggested failures applied through instruction-guided image editing, (iii) F-GPS-identified failure cues applied through instruction-guided editing, and (iv) the complete F-GPS pipeline using target-model-specific failure cues and executable visual programs.

All configurations are evaluated on SDXL-Base-1.0 using Diffusion-DPO, with the same preferred examples, conditioning prompts, optimization settings, and evaluation protocols.

Table 12 (revised manuscript): Ablation of perturbation source and negative construction — image composition.

Perturbation sourceNegative constructionT2I-CompBench++ BLIPT2I-CompBench++ GITTIFA BLIPTIFA GIT
Base—78.0875.3088.6087.20
Predefined Visual PerturbationsExecutable Visual Programs71.7174.0474.9173.03
LLM-Suggested FailuresInstruction-Guided Editing43.8430.8640.7833.22
F-GPS-Identified Failure CuesInstruction-Guided Editing60.4952.5461.7353.14
F-GPS-Identified Failure CuesExecutable Visual Programs81.5579.3092.3090.80

Table 13(a): Component ablation on CVTG (visual text rendering).

MethodEMNEMPSSWPWRWF1
Base24.227.077.950.150.948.6
Predefined Visual Perturbations + Executable Programs7.67.874.257.032.838.1
LLM-Suggested Failures + Model-Based Editing16.016.070.149.528.433.0
F-GPS-Identified Failure Cues + Model-Based Editing6.06.067.743.421.626.3
F-GPS-Identified Failure Cues + Executable Programs32.032.881.358.259.056.8

Table 13(b): Component ablation on MARIO-Eval (visual text rendering).

MethodEMNEMPSSWPWRWF1
Base0.80.839.83.13.73.1
Predefined Visual Perturbations + Executable Programs20.823.875.642.443.841.0
LLM-Suggested Failures + Model-Based Editing14.416.664.128.926.526.4
F-GPS-Identified Failure Cues + Model-Based Editing15.417.264.031.328.328.2
F-GPS-Identified Failure Cues + Executable Programs24.926.176.848.146.745.8

Model-Based Editing refers to instruction-guided editing with FLUX.2-klein-4B. All results use SDXL-Base-1.0 with Diffusion-DPO (Appendix I.5, Table 13).

The results demonstrate the importance of the proposed failure-guided construction. Using BLIP-Large, predefined executable perturbations achieve 71.71% on T2I-CompBench++ and 74.91% on TIFA, compared with 81.55% and 92.30% for F-GPS. For visual text rendering, CVTG Exact Match improves from 7.6% with predefined perturbations to 32.0% with F-GPS.

We also investigate the negative-construction mechanism using instruction-guided editing with FLUX.2-klein-4B. On T2I-CompBench++, editing with LLM-suggested failures achieves 43.84%, while editing with F-GPS-identified cues achieves 60.49%, compared with 81.55% for executable F-GPS programs. The different configurations exhibit some task-dependent behavior, but the complete F-GPS pipeline performs best among these alternatives across the reported benchmarks.

Furthermore, our evaluation of 500 matched preferred–rejected pairs per method and task shows that executable F-GPS programs realize all intended alterations in 80.8%/84.4% of image-composition/text-rendering pairs, compared with 3.6%/6.2% for editing with F-GPS-identified cues. Executable programs also introduce substantially fewer unrequested changes.

These experiments provide evidence that the contribution extends beyond generic image corruption: both the failure information extracted from the target model and its controlled operationalization affect the quality of the resulting preference supervision.

The complete results are presented in Appendix I.5 (Tables 12–14).

2. Continued Relevance of Direct Preference Optimization

We would like to emphasize that direct preference optimization (DPO) remains an active research direction for text-to-image alignment, as demonstrated by recent works such as SIPO (ICML 2026), BIDPO (CVPR 2026), and Linear-DPO (2026). These works continue to develop and improve preference-based optimization approaches, highlighting their relevance alongside emerging online RL methods.

Our work is complementary to these developments. Rather than proposing another optimization objective, F-GPS focuses on improving the construction of preferred–rejected pairs, which are fundamental to DPO-style training. Thus, advances in preference optimization further motivate the importance of constructing informative preference supervision.

3. Extension of F-GPS to online preference optimization

We thank the reviewer for this suggestion. We would like to clarify that F-GPS is primarily designed for offline preference-supervision construction, and its direct integration into online RL-based optimization is not straightforward.

In particular, methods such as GRPO typically sample a group of candidate images from the current policy, evaluate them using a reward model, and compute relative advantages within the group. This allows multiple candidates to receive positive or negative advantages based on their relative rewards. In contrast, F-GPS constructs rejected images by applying targeted failure programs to a given editable preferred example, rather than sampling and comparatively evaluating multiple candidates from the current policy. Therefore, the preference construction mechanism in F-GPS does not directly align with the group-relative optimization setting.

Furthermore, online optimization continuously updates the policy, potentially changing its failure distribution. Maintaining model-specific failure supervision would therefore require periodically revisiting failure discovery and program construction. Repeating these MLLM-based analyses as the policy evolves could introduce substantial computational overhead, while retaining a fixed failure-program library might not adequately capture newly emerging failures.

Consequently, integrating F-GPS into standard GRPO-style training would require additional methodological development, including adapting its supervision to group-relative optimization and determining how frequently failure programs should be updated. We have acknowledged these limitations in Appendix A and consider such integration an interesting direction for future work.

Thanks for providing the key comparison. On account of this, I am increasing my score.

We sincerely thank the reviewer for taking the time to consider our rebuttal and for updating their score. We greatly appreciate their constructive feedback and reconsideration of our work.

Reviewer_R3hHOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies the compositional generation capabilities of text-to-image diffusion models. Specifically, the paper hypothesizes that creating targeted negative examples for preference optimization can significantly improve model performance. To this end, the authors propose the F-GPS framework, which consists of three main stages: (1) Failure Discovery, which uses an LLM (GPT-4o) to identify compositional failures of a target model; (2) Failure Program Construction, which uses GPT-4o to create executable visual programs; and (3) Failure Program Application, which sequentially applies (r) failure programs to construct a negative sample. Specifically, these visual programs can modify images that have editable representations.

The paper evaluates F-GPS using three preference optimization algorithms—DPO, Linear-DPO, and DSPO—on two tasks: image composition and text rendering. The evaluation setup includes SD1.5, SDXL, and SD3.5 and compares F-GPS against existing baselines such as IterComp, SPO, and InPO, as well as the authors' introduced negative construction methods. The evaluation results indicate that F-GPS effectively improves text rendering and compositional generation in T2I models, outperforming other negative construction baselines across both tasks.

Critical strengths
  • The paper presents an approach to constrcut preference pairs by nverting observed model failures into reusable executable programs. This is approach can provide useful and effective pairs/data that is essential for sucessful preference optimization methods.

  • The experiments setup includes 2 tasks, multiple backbones. The reported improvements are generally consistent and shows that model is effective across different backbones and tested preference methods.

Critical weaknesses
  • [W1] Limited methodological novelty, inadequate/insufficient baselines & evidence to test importance visual programs and effectivness of us F-GPS components. One of the main claims of the paper is that targeted negative construction effectively improves the performance of base diffusion models. However, constructing negative examples through controlled image modifications is not particularly novel on its own. The main methodological distinction of F-GPS is the failure analysis using GPT-4o to construct executable visual programs. However, the main negative construction baselines, MG (generating images from prompts) and MG+RS (MG + random shuffle), do not clearly isolate whether the improvements come from targeted negative construction, model-specific failure analysis, or the use of executable visual programs. Therefore, two additional baselines are critiacal to establish the effectiveness of the components: (1) random corruption baseline. This applies similar corruptions to positive images without MLLM-based failure analysis (i.e. e.g., a fixed set of corropotions not tailored to the target image). One simiple approach could be to use LLM to suggest a list of corruptions. (2) image-editing baseline that generates negative examples by directly editing the pos. image. It remains unclear whether visual programs are necessary, especiall because current sota image-editing models can perform targeted and potentially more complex edits compared to visual programs. I suggest authors to conduct this test with comprable/fair models (commercial or public) at-least on a smaller set. Moreover, unlike F-GPS, these models do not necessarily require structured editable representations, raising further questions regarding the effectiveness, necessisity of the visual programs

  • [W2] The paper requires some additional ablations and error analsis to showcase its effectivness and reliability.

The proposed framework relies heavily relies multiple passes of MLLM (GPT-4o) for failure extraction, atomization, filtering, and program construction. Each stage is prone to have errors. However, the paper does not sufficiently evaluate the correctness of the identified failures or the sensitivity of the framework to the choice of MLLM. Errors in these stages could propagate into the failure programs creating weak or incorrect preference pairs. Since failure analysis is an essential component of F-GPS, some evaluation (I suggest small human evaluation) discovered failures and an ablation using different LLMs, including open-source models of different sizes, are important for establishing the robustness and generalizability of the framework. Additionally, in L246 it's mentioned that "we randomly sample r failure program". However, I couldn't find what the value of r is. I would appreciate if authors could provide insight how different values of "r" can impact the effectiveness of model. for instance do small values of r still create effective negative examples? are large values of r causing to create extremely hard and hence bad quality negative examples?

Questions for authors

In addition to above, i appreciate clarification on the following concerns and/or questions:

  1. comparisong against fair and more appropriate negative construction baselines (e.g. image editing and random)
  2. some error anlaysis on different stages (e.g. failrue discover or etc) using preferebly human evaluation or approrpirate automatic metrics. In particular, it would be useful to measure how often the generated negatives faithfully reflect the intended failure types and genuinely violate the corresponding prompts.
  3. I appreciate it if authors could clarify how related the studied failrues/edits and negatives to the teest set benchmarks. According to the prompt they seem to be highly related. While this is still okay as the method is studying compostional and text rendering generation, this could raise concern on generalization of the proposed method to setting beyond what the visual programs were construcrted. I appreciate if authors could clarify the generalization of the proposed method to unseen compositional scenarios or discuss this in the limitation section.
  4. Sensitivity to the MLLM model
  5. ablation on hyperparamter r
  6. I appreciate albation on library size. It seems like resutl show that increasing L consistenty increasing the performance. According to the plots I suggest testing larger values to clearly see the curve and potential of the proposed method. In other words, the current plot don't really show what library sizes saturates the performance.
  7. I appreciate that three different preference optimization methods were explored. However, all methods are used in offline setting. COuld authors please clarify whether F-GPS can be extended to online algorithms and provide quantitative evidence? If not, discussing this limitation would help clarify the scope of the framework
11 comments

Reviewer R3hH

1. Methodological novelty and contribution beyond existing controlled negative-example construction methods

We appreciate the opportunity to clarify our contribution. F-GPS does not introduce a new preference optimization objective; rather, it addresses how informative rejected examples should be constructed. Our central motivation is to provide the target model with preference supervision that explicitly reflects its own observed failures, enabling it to learn not only what a preferred image should look like, but also which specific visual deviations it should avoid.

Unlike methods that rely on predefined perturbations, model-generated negatives, or generic editing operations, F-GPS derives its negative-construction mechanism from failures observed in the target T2I model. Specifically, F-GPS (i) identifies sample-specific failures, (ii) atomizes and consolidates recurring failure patterns into reusable atomic visual cues, (iii) operationalizes these cues as parameterized executable programs, and (iv) applies compositions of these programs to editable preferred examples. This allows us to construct preferred–rejected contrasts that deliberately expose the model to the types of mistakes identified in its own generations, rather than relying on generic or potentially ambiguous negative examples.

Compared with approaches such as D-Fusion, OSPO, and RealAlign, our distinguishing contribution is the distillation of target-model failures into a reusable executable failure-program library, rather than relying on a fixed perturbation vocabulary or an independently selected editing mechanism. The new component ablations further demonstrate the importance of both the failure source and the negative-construction mechanism (Appendix I.5, Tables 12–14).

2. Comparison with non-failure-guided random corruption baselines

We have added a baseline using predefined visual perturbations inspired by existing augmentation approaches. These perturbations are implemented as executable programs and applied to the same type of preferred examples, but are not derived from the target model's observed failures.

Table 12 (revised manuscript): Ablation of perturbation source and negative construction — image composition.

Perturbation sourceNegative constructionT2I-CompBench++ BLIPT2I-CompBench++ GITTIFA BLIPTIFA GIT
Base—78.0875.3088.6087.20
Predefined Visual PerturbationsExecutable Visual Programs71.7174.0474.9173.03
LLM-Suggested FailuresInstruction-Guided Editing43.8430.8640.7833.22
F-GPS-Identified Failure CuesInstruction-Guided Editing60.4952.5461.7353.14
F-GPS-Identified Failure CuesExecutable Visual Programs81.5579.3092.3090.80

Table 13(a): Component ablation on CVTG (visual text rendering).

MethodEMNEMPSSWPWRWF1
Base24.227.077.950.150.948.6
Predefined Visual Perturbations + Executable Programs7.67.874.257.032.838.1
LLM-Suggested Failures + Model-Based Editing16.016.070.149.528.433.0
F-GPS-Identified Failure Cues + Model-Based Editing6.06.067.743.421.626.3
F-GPS-Identified Failure Cues + Executable Programs32.032.881.358.259.056.8

Table 13(b): Component ablation on MARIO-Eval (visual text rendering).

MethodEMNEMPSSWPWRWF1
Base0.80.839.83.13.73.1
Predefined Visual Perturbations + Executable Programs20.823.875.642.443.841.0
LLM-Suggested Failures + Model-Based Editing14.416.664.128.926.526.4
F-GPS-Identified Failure Cues + Model-Based Editing15.417.264.031.328.328.2
F-GPS-Identified Failure Cues + Executable Programs24.926.176.848.146.745.8

Model-Based Editing refers to instruction-guided image editing using FLUX.2-klein-4B. All results use SDXL-Base-1.0 with Diffusion-DPO (Appendix I.5, Table 13).

These results indicate that executable perturbations alone are insufficient: identifying and operationalizing failures relevant to the target model is an important component of the resulting preference supervision. The complete results are reported in Appendix I.5 (Tables 12 and 13).

3. Comparison with image-editing models and the role of executable visual programs and structured editable representations

We have added two instruction-guided image-editing configurations using FLUX.2-klein-4B: one based on LLM-suggested failures and another based on F-GPS-identified failure cues. Both use the same preferred images and conditioning prompts as F-GPS, and the resulting models are trained with identical SDXL Diffusion-DPO optimization settings.

On T2I-CompBench++, instruction-guided editing achieves 43.84% with LLM-suggested failures and 60.49% with F-GPS-identified failures, compared with 81.55% for the complete F-GPS pipeline.

To understand this difference, we additionally evaluated 500 matched preferred–rejected pairs per method and task. The intended alterations were fully realized in only 0.4%/2.6% of LLM-suggested edits and 3.6%/6.2% of F-GPS-guided edits, compared with 80.8%/84.4% for executable programs (image composition/visual text rendering).

Executable programs also introduced substantially fewer unrequested changes. These results suggest that structured editable representations and executable transformations provide more reliable control over intended visual deviations than instruction-guided image editing in the evaluated configurations (Appendix I.5, Tables 12–14).

We acknowledge that improved prompting strategies or more capable editing models could potentially enhance editing-based negative construction. Nevertheless, generative editing may introduce variability in the realized changes and does not inherently guarantee that only the intended visual factors are modified. In contrast, executable programs provide a more deterministic and reproducible mechanism for applying targeted transformations, with explicit control over the modified elements. We view this controllability as an important advantage for constructing consistent preference supervision.

4. Reliability and correctness of MLLM-based failure analysis

We assess the reliability of the resulting failure supervision through three complementary analyses. The original manuscript included image-composition failure-cue correspondence and program-execution reliability; the revision extends cue correspondence to visual text rendering and adds a direct single-program fidelity analysis.

Failure-cue correspondence. We reanalyze 100 F-GPS-constructed rejected images per task using the same failure analyst and compare the recovered failure cues against the original inventories. Considering explicit and implicit correspondences, we obtain 94.1% coverage for image composition (16/17 cues) and 100% for visual text rendering (17/17 cues) (Appendix I.6, Figure 28).

Execution reliability. We apply each executable failure program to 3,000 dataset samples from the corresponding task, recording execution success and runtime. The results indicate high execution reliability across both program libraries (Appendix I.7, Figure 29).

Failure-program fidelity. We evaluate 500 program-balanced, single-program preferred–rejected pairs per task using GPT-6.1 Sol with medium reasoning effort. The intended visual changes are successfully identified in 88.0% of image-composition and 91.2% of visual text-rendering accepted pairs (Figure 6; Appendix I.7).

These analyses provide evidence that the distilled failure cues can be operationalized into executable programs that reliably produce their intended visual effects.

5. Sensitivity of failure analysis to the choice and size of MLLM

We have added a sensitivity experiment extending failure discovery to four additional MLLMs, keeping the discovery seed prompts and corresponding SDXL-Base-1.0 outputs fixed. Together with GPT-4o, we evaluate five failure analysts:

Failure analystImage compositionVisual text rendering
GPT-4o1717
Gemini 3.5 Flash-Lite1410
Gemma 4 E4B78
GPT-6.1 Sol1921
Qwen3-VL-8B-Instruct2120

The models identify several recurring failure patterns, but differ in the granularity, specificity, and decomposition of those patterns. For example, Gemma 4 E4B tends to consolidate related failures into broader categories, whereas GPT-6.1 Sol and Qwen3-VL-8B-Instruct identify more fine-grained inventories.

These findings show that failure analysis is sensitive to the choice of MLLM. However, inventory size alone does not establish failure-discovery quality, and our experiments do not imply a monotonic relationship between model size and performance. We use GPT-4o as a practical choice considering the granularity of its identified cues, accessibility, and cost. The complete inventories are presented in Appendix I.8 (Figures 30 and 31).

6. Rationale for selecting number of composed failure programs (r)

We have further clarified the rationale for selecting the number of composed failure programs, r.

Using 100 images per task, we analyze CLIP and DINOv2 similarity between preferred images and their perturbed counterparts while varying r from the available library of 17 programs. Increasing r generally reduces image similarity. Small values may produce insufficiently visible deviations, whereas large values may substantially alter the preferred image and compromise its original content.

Based on this analysis and manual inspection, we uniformly sample r ∈ {3, 4, ..., 11}, selecting distinct failure programs without replacement during preference-pair construction. The similarity analysis and selection rationale are provided in Appendix G.1 (Figure 21).

7. Fidelity of generated rejected examples to intended failure types and prompt requirements

We have added a direct evaluation of negative-example construction using 500 matched preferred–rejected image pairs per method and task, evaluated by GPT-6.1 Sol with medium reasoning effort..

The evaluation considers four criteria: (i) whether all intended alterations are visibly realized, (ii) whether the rejected image introduces a clear new prompt violation, (iii) whether unrequested changes occur, and (iv) whether the rejected image is visually better than the preferred image independently of prompt adherence.

Table 14a: Image Composition.

Evaluation criterionNegative constructionYes (%)No (%)Unclear (%)
All intended changes visibly present?LLM-suggested failures + instruction-guided editing0.498.41.2
All intended changes visibly present?F-GPS cues + instruction-guided editing3.690.26.2
All intended changes visibly present?F-GPS cues + executable programs80.815.83.4
New clear prompt violation?LLM-suggested failures + instruction-guided editing2.463.833.8
New clear prompt violation?F-GPS cues + instruction-guided editing27.452.420.2
New clear prompt violation?F-GPS cues + executable programs58.220.421.4
Unrequested changes?LLM-suggested failures + instruction-guided editing82.018.00.0
Unrequested changes?F-GPS cues + instruction-guided editing78.220.41.4
Unrequested changes?F-GPS cues + executable programs10.289.20.6
Rejected image visually better?LLM-suggested failures + instruction-guided editing7.092.60.4
Rejected image visually better?F-GPS cues + instruction-guided editing0.299.60.2
Rejected image visually better?F-GPS cues + executable programs0.299.60.2

Table 14b: Visual Text Rendering.

Evaluation criterionNegative constructionYes (%)No (%)Unclear (%)
All intended changes visibly present?LLM-suggested failures + instruction-guided editing2.695.61.8
All intended changes visibly present?F-GPS cues + instruction-guided editing6.290.03.8
All intended changes visibly present?F-GPS cues + executable programs84.47.48.2
New clear prompt violation?LLM-suggested failures + instruction-guided editing26.434.039.6
New clear prompt violation?F-GPS cues + instruction-guided editing36.632.431.0
New clear prompt violation?F-GPS cues + executable programs51.623.025.4
Unrequested changes?LLM-suggested failures + instruction-guided editing54.845.20.0
Unrequested changes?F-GPS cues + instruction-guided editing71.628.20.2
Unrequested changes?F-GPS cues + executable programs0.0100.00.0
Rejected image visually better?LLM-suggested failures + instruction-guided editing17.482.60.0
Rejected image visually better?F-GPS cues + instruction-guided editing10.489.60.0
Rejected image visually better?F-GPS cues + executable programs0.0100.00.0

500 matched pairs per construction method and task. Values are percentages of Yes/No/Unclear judgments; lower Yes is desirable for unrequested changes and visually better negatives.

These results support the effectiveness of executable programs in constructing rejected examples with recognizable intended deviations while limiting unintended modifications. They complement our separate single-program fidelity analysis, which reports 88.0%/91.2% visible realization on accepted evaluation pairs.

The full analysis is presented in Appendix I.5 (Table 14), with evaluation instructions in Appendix J.2.

Comments on the paper

2 comments
AuthorsOpenReview ↗

Summary of Additional Experiments and Rebuttal Clarifications

Dear Area Chair,

We sincerely thank you for your time and consideration. We would like to summarize the additional experimental evidence and clarifications provided during rebuttal and respectfully request consideration of the full body of evidence in the final assessment.

1. Additional Experimental Evidence

In response to the reviewers' concerns, we conducted the following experiments:

  • Component ablations (R3hH, Be99, iki6): Compared predefined visual perturbations, LLM-suggested failures, and model-based editing against F-GPS, demonstrating the effectiveness of failure-guided executable negative construction (Appendix I.5, Tables 12–13).
  • Negative-example validation (R3hH, iki6): Evaluated 500 pairs per method and task, alongside single-program fidelity, execution reliability, and failure-cue correspondence analyses (Table 14, Figure 6, Appendix I.6–I.7).
  • MLLM sensitivity (R3hH, iki6): Extended failure discovery to five MLLMs from different families, identifying recurring failure patterns alongside differences in decomposition and granularity (Appendix I.8).
  • Program composition (R3hH): Analyzed CLIP/DINOv2 similarity and additionally evaluated larger values of r∈{13,15,17}r \in \{13,15,17\}, observing substantially lower downstream performance than the original configuration.
  • Discovery–evaluation overlap (R3hH): Conducted semantic-similarity analysis to investigate potential overlap between failure-discovery seeds and evaluation prompts (Appendix I.9).
  • Cross-backbone transferability (iki6): Evaluated SDXL-derived failure supervision on SD v1.5 for both image composition and visual text rendering, observing partial, benchmark-dependent transfer (Appendix I.10, Tables 15–16).

2. Clarifications Addressing Reviewer Concerns

We also provided methodological and conceptual clarifications addressing the following concerns:

  • Methodological novelty: Clarified that F-GPS converts observed target-model failures into reusable executable programs, providing preference supervision that explicitly targets the model's observed weaknesses rather than relying on generic negative examples.
  • Reliability of failure discovery: Highlighted qualitative overlap in failure patterns identified independently by five MLLMs from different families, along with their correspondence to failures documented in existing literature. We also provided inspectable examples of failure identification and atomization (Appendix D).
  • Executable programs vs. generative editing: Acknowledged that stronger editing models may improve negative construction, while emphasizing the controllability, reproducibility, and computational efficiency of reusable executable programs, particularly compared with large-scale generative editing.
  • Failure-based vs. success-based supervision: Clarified that preferred examples are often available through curated or SFT datasets, whereas constructing informative negatives remains challenging. We also explained that failure discovery analyzes the complete set of target-model outputs without initially separating successful and failed generations.
  • Preference optimization and online RL: Explained why ambiguous or potentially reversed model-generated preference pairs can degrade alignment, the continued relevance of DPO-based supervision, and why F-GPS cannot be directly integrated into GRPO-style online optimization without adapting its preference-construction mechanism.
  • Evaluation reliability: Clarified that Qwen3-VL-8B-Instruct assesses comparative preferences, while BLIP-Large, GIT-Large, and LLaVA-based evaluations assess different dimensions of prompt adherence and text-rendering accuracy.
  • Generalization and limitations: Distinguished generalization across held-out prompts and compositional scenarios from generalization to entirely unseen failure categories. We also clarified the scope of the prompt-similarity analysis, the effect of failure-program library size, and the limitations of cross-backbone transfer and alternative MLLM analysis.
  • Presentation improvements: Made several presentational and organizational revisions throughout the manuscript to improve clarity, readability, and overall presentation.

3. Acknowledged Strengths and Reviewer Assessments

The reviewers recognized the importance of informative preference supervision, the intuitive methodological motivation, significant empirical improvements, and consistent evaluation across multiple backbones and preference objectives.

  • Be99 increased their score from 2 to 3, explicitly acknowledging that our additional component ablations addressed their principal concern.
  • R3hH acknowledged that several major concerns were adequately addressed but maintained their original score. We subsequently provided further experimental evidence on larger program compositions and detailed clarifications addressing the remaining concerns.
  • iki6 recently confirmed their original score following our detailed rebuttal and additional experiments. We have requested clarification regarding the remaining concerns. Given the discussion timeline, further experiments are not feasible at this stage, but we remain open to incorporating specific editorial and methodological clarifications in the final manuscript.

4. Closing Request

We believe the revised manuscript and substantial additional evidence provide strong empirical support for the central contribution of F-GPS within its evaluated scope. We therefore respectfully request that the complete revised manuscript, additional experiments, and author–reviewer discussions receive due consideration in the final assessment, beyond what may be reflected in the numerical scores alone.

Thank you again for your time and consideration.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.