Learning from Model Failures Distilled as Executable Visual Programs

Text-to-image (T2I) models can often satisfy a generation requirement on some inputs while failing on closely related ones, suggesting that useful capabilities may already be present but are expressed inconsistently. Preference-based alignment provides a natural mechanism for reinforcing desired generations and discouraging undesirable ones. However, its effectiveness depends not only on how preferences are optimized, but also on the informativeness of the preferred–rejected contrast. We introduce Failure-Guided Preference Supervision (F-GPS), a framework that constructs preference supervision from failures observed in the target T2I model. Rather than treating failures only as diagnostic signals, F-GPS distills recurring failure patterns into reusable programs: it atomizes observed coarse failure instances into fine-grained visual factors, aggregates these factors across samples into a candidate inventory, filters the inventory into reusable atomic visual cues, and operationalizes the retained cues as parameterized failure programs. A randomly sampled composition of these failure programs is then applied to an editable preferred example to construct a rejected example with targeted deviations, yielding an informative preference contrast. We study F-GPS on image composition and visual text rendering, two settings exhibiting distinct forms and granularities of generation failure. Across multiple T2I backbones and preference-optimization objectives, F-GPS improves downstream alignment, with gains of up to 5.6 pp for image composition and 10.4 pp for visual text rendering over SFT. These results suggest that failure patterns observed in the target model can provide targeted supervision for preference-based T2I alignment.