acceptodds
Under review as a conference paper at ICLR 2027

Failure-Aware Adaptive Recovery for Text-to-Image Generation: When to Resample and When to Rewrite

Abstract

A failed text-to-image generation reveals what went wrong, but not how additional inference compute should be used for recovery. We show that post-failure recoverability is intervention-dependent: resampling the original prompt provides a strong baseline, while prompt rewriting recovers a complementary subset of failures missed by seed exploration. This motivates viewing recovery as a budget-allocation problem over stochastic and semantic interventions rather than as uniform prompt refinement. We introduce FARE, a failure-aware inference-time framework that uses evidence from the initial failed generation to guide both recovery allocation and early trajectory pruning. FARE estimates the recovery utility of feasible prompt–seed portfolios and conservatively retains resampling unless an alternative allocation provides sufficient predicted benefit. It then explores the selected candidates in parallel and uses the observed failed trajectory as a request-specific reference to prune candidates likely to reproduce the same failure. Across multiple T2I benchmarks, FARE improves recovery over fixed recovery policies under matched post-failure budgets, and its advantage persists as the available inference budget increases. Across SD2.1, SDXL, and FLUX.1-dev, failure-aware allocation improves recovery over the train-selected fixed allocation by 3.6–4.6 percentage points. We further find that matched failure references provide stronger early recovery signals than shuffled references, enabling conservative pruning that reduces redundant denoising while largely preserving recovery. These results recast post-failure prompt optimization as a failure-conditioned inference-time search problem, in which prompt rewriting is one intervention alongside stochastic exploration rather than the default response to generation failure.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
image generation,resource allocation,failure recovery

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_CbcCOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1(−2)
Read review
Paper summary

This paper studies how to allocate additional inference compute after an initial text-to-image generation fails. Motivated by the observation that resampling the original prompt is a strong baseline while prompt rewriting recovers a complementary subset of failures, the authors formulate post-failure recovery as a budget-allocation problem over stochastic exploration and semantic intervention. They propose FARE, which uses structured evidence from the initial failed image to distribute a fixed generation budget among original-prompt resampling and three failure-conditioned rewrite operators. FARE also employs a failure-referenced pruning mechanism that compares each candidate’s early denoising trajectory with the initial failed trajectory and terminates candidates likely to reproduce the same failure.

Critical strengths
  1. The paper considers a relevant decision problem in post-failure text-to-image generation. Its experiments suggest that prompt rewriting is not always preferable to resampling and that the two interventions may recover partially different subsets of failed requests. These observations provide reasonable motivation for formulating recovery as a budget-allocation problem across intervention types.

  2. The method is conceptually coherent. FARE separates the request-level decision of where to allocate computation from the trajectory-level decision of which candidates should continue receiving computation. The conservative fallback to all-resampling is well motivated given the strength of that baseline.

  3. Several experimental choices are appropriate. In particular, images generated from rewritten prompts are still evaluated against the original prompt. The request-level train/validation/test split also avoids leakage among counterfactual allocations derived from the same request. The evaluation spans multiple benchmarks, generators, and inference budgets, and the appendix provides substantial implementation detail.

Critical weaknesses
  1. Dependence on a single verifier. The same benchmark-specific verification pipeline is used to identify initial failures, derive failure features, construct allocation supervision, and determine whether recovery succeeds. Although this protocol demonstrates improvements under the selected verifier, it remains unclear whether the ranking of recovery policies is robust to alternative VLM evaluators or different success thresholds. Evaluation with an independent verifier and threshold-sensitivity analysis would help distinguish robust semantic improvements from evaluator-specific effects.

  2. Limited coverage of generation quality. The evaluation focuses on semantic recovery rates and VQA scores, without a complementary measure of overall perceptual quality. Consequently, it is unclear whether improved satisfaction of individual semantic constraints is achieved at the expense of visual fidelity or aesthetic quality.

  3. The claimed generality and compute efficiency are not yet established under a complete accounting of training and inference costs. The efficiency comparison primarily counts diffusion denoising steps. It does not provide unified end-to-end latency or cost accounting for failure diagnosis, LLM-based rewriting, semantic-preservation checking, trajectory storage, and trajectory-feature extraction. These omissions make it difficult to assess whether FARE offers a favorable practical trade-off relative to simple resampling.

Questions for authors
  1. Does FARE retain its advantage when recovery is evaluated using an independent VLM backbone or an alternative benchmark-compatible verifier? In particular, does the relative ranking among FARE, resampling, and rewriting remain consistent across evaluators?

  2. How sensitive are the reported recovery improvements to the semantic-success threshold? Results across a range of thresholds, or threshold-independent recovery curves, would help determine whether the gains are robust or concentrated near the selected decision boundary.

  3. Does adaptive allocation affect visual fidelity or aesthetic quality? Could the authors report complementary reference-free quality metrics and compare the successfully recovered images produced by FARE with those produced by resampling and rewriting baselines?

  4. What is the end-to-end latency and computational cost of FARE when failure diagnosis, prompt rewriting, semantic-preservation checking, trajectory storage, feature extraction, and diffusion generation are all included? How does this complete cost compare with simple resampling under the same hardware and batching conditions?

6 comments
AuthorsOpenReview ↗

Response to Reviewer CbcC (part 1/2)

Question 1

Does FARE retain its advantage when recovery is evaluated using an independent VLM backbone or an alternative benchmark-compatible verifier? In particular, does the relative ranking among FARE, resampling, and rewriting remain consistent across evaluators?

We thank the reviewer for raising this important question. We agree that evaluating recovery with different VLM backbones is necessary to determine whether the reported improvements depend on a particular evaluator. To investigate this, we conducted additional evaluations using two independent VLMs from different model families: Molmo2-O-7B and InternVL3.5-8B. We kept the original test cases, recovery decisions, and generated images fixed. Neither evaluator participated in candidate generation, budget allocation, or image selection. Thus, only the evaluation backbone was changed. We compare Resampling, the train-selected rewriting baseline, and FARE F2 under both independent evaluators. The results are reported in the following.

Benchmark / Generator (Molmo2-O-7B)Train-selected BestResamplingFARE F2
GenEval2 / SD2.126/152 (17.11%)31/152 (20.39%)29/152 (19.08%)
GenEval2 / FLUX.1-dev81/131 (61.83%)76/131 (58.02%)86/131 (65.65%)
TIFA / SD2.1248/623 (39.81%)250/623 (40.13%)258/623 (41.41%)
T2I-CompBench / SD2.167/231 (29.00%)67/231 (29.00%)68/231 (29.44%)
Benchmark / Generator (InternVL3.5-8B)Test CasesResampling (%)Availability-Aware Fixed (%)FARE F2 (%)
GenEval2 / SD2.115217.1119.0820.39
GenEval2 / FLUX.1-dev13158.0265.6561.83
TIFA / SD2.162340.1339.8141.41
CompBench / SD2.123129.0029.0029.44

The results lead to two observations. First, FARE F2 maintains competitive recovery performance across both independent evaluators, suggesting that its effectiveness is not solely attributable to the original benchmark evaluators. In particular, FARE F2 achieves the highest recovery rate in most of the evaluated benchmark and evaluator combinations. Second, the relative ranking of the methods is not perfectly consistent across evaluators. In some settings, Resampling or the fixed rewriting baseline performs better than FARE F2. These results suggest that evaluator-dependent variations exist, which is expected given the differences in semantic judgment across VLM backbones. Nevertheless, FARE remains consistently competitive and achieves the best performance in most settings.

Question 2

How sensitive are the reported recovery improvements to the semantic-success threshold? Results across a range of thresholds, or threshold-independent recovery curves, would help determine whether the gains are robust or concentrated near the selected decision boundary.

Thank you for this suggestion. We evaluated threshold sensitivity on the same held-out GenEval2 cases—152 for SD2.1 and 131 for FLUX—while keeping allocation decisions, generation budgets, and final selected images fixed. We swept the geometric-mean semantic-success threshold over {0.3, 0.4, 0.5, 0.6, 0.7}, without retraining, regenerating images, or selecting a threshold based on test performance. We report both recovery-rate curves and FARE-minus-baseline differences.

Under the original evaluator, FARE F2 outperformed resampling on FLUX at all five thresholds, with gains of 3.05, 1.53, 3.82, 5.34, and 6.11 percentage points, respectively. Thus, this advantage was not confined to the selected boundary of 0.5. On SD2.1, however, the differences ranged from −0.66 to +0.66 percentage points, indicating a small and threshold-sensitive advantage. FARE also did not consistently outperform the stronger Train-selected Best baseline on FLUX.

Additionally, the evaluation with Molmo2-O-7B and InternVL3.5-8B showed positive FARE-versus-resampling point estimates on FLUX across all five thresholds, but no consistent advantage on SD2.1.

EvaluatorGeneratorRange of F2 − Resampling Across Five GM Thresholds (pp)
Molmo2-O-7BSD2.1−3.29 to −0.66
InternVL3.5-8BSD2.1−3.95 to +0.66
Molmo2-O-7BFLUX+3.05 to +7.63
InternVL3.5-8BFLUX+1.53 to +9.16
AuthorsOpenReview ↗

Response to Reviewer CbcC (part 2/2)

Question 3

Does adaptive allocation affect visual fidelity or aesthetic quality? Could the authors report complementary reference-free quality metrics and compare the successfully recovered images produced by FARE with those produced by resampling and rewriting baselines?

Thank you very much for your constructive comment. We used three no-reference image quality assessment (NR-IQA) metrics—MUSIQ, LAION Aesthetic Score, and TOPIQ-NR—to directly evaluate the quality of the generated images. The results are reported below.

Benchmark / GeneratorNMUSIQ ↑Aesthetic ↑TOPIQ-NR ↑
GenEval2 / SD2.12172.778 / 72.978 / 72.5985.612 / 5.501 / 5.6090.618 / 0.612 / 0.618
GenEval2 / FLUX5475.879 / 76.177 / 75.9356.032 / 5.959 / 5.9200.757 / 0.762 / 0.756
TIFA / SD2.115571.175 / 69.982 / 70.8095.506 / 5.446 / 5.4740.637 / 0.613 / 0.622
CompBench / SD2.117570.802 / 70.581 / 70.7995.549 / 5.469 / 5.5180.618 / 0.611 / 0.616

Note: Values are reported in the order Resampling / Train-selected Rewrite / FARE F2. N denotes the number of paired cases.

Across all four settings, FARE achieves image quality scores comparable to the baselines, suggesting that its adaptive recovery strategy does not substantially compromise visual quality.

Question 4

What is the end-to-end latency and computational cost of FARE when failure diagnosis, prompt rewriting, semantic-preservation checking, trajectory storage, feature extraction, and diffusion generation are all included? How does this complete cost compare with simple resampling under the same hardware and batching conditions?

We thank the reviewer for raising this important question. To provide a complete comparison, we measured the end-to-end runtime of FARE and Fixed Resampling on an NVIDIA A100 80GB GPU, with batch size = 1 and the same post-failure generation budget (B = 9).

Our measurements cover the complete recovery pipeline, including initial generation, failure diagnosis, prompt rewriting, semantic verification, trajectory storage, feature extraction and allocation prediction, recovery generation, and final recovery verification.

The results show that FARE without pruning takes 233.74 seconds per request, representing a 16.3% increase over Fixed Resampling (200.93 seconds). This additional computational cost brings recovery improvements of 1.1, 1.2, and 1.7 percentage points on GenEval2, TIFA, and T2I-CompBench, respectively. The average VQA score also increases from 0.2180 to 0.2243. With online pruning, FARE reduces the end-to-end runtime to 181.86 seconds, saving 9.5% compared with Fixed Resampling. The exploratory median checkpoint further reduces the runtime to 170.62 seconds, saving 15.1%. The detailed end-to-end latency and computational statistics are reported in Table 1, with a stage-wise breakdown provided in Table 2.

MethodEnd-to-End Latency (s/request)Latency Change vs. ResamplingActual Denoising Iterations per Request¹Total Atomic VQA Calls²
Fixed Resampling200.93—500.0470
FARE without Pruning233.74+16.3%500.0470
FARE with Online Pruning (Retention-Constrained Checkpoint)181.86−9.5%386.0321
FARE with Online Pruning (Exploratory Median Checkpoint)170.62−15.1%379.7310
StageResamplingFARE without PruningFARE with Pruning (Constraint Checkpoints)
Initial Generation11.8610.7111.98
Failure Diagnosis7.007.407.93
Prompt Rewriting020.0820.65
Semantic Verification014.6714.97
Allocation Feature Extraction and Prediction01.963.53
Recovery Generation110.89107.4278.73
Recovery Verification71.1571.4544.04

These results demonstrate that FARE without pruning trades additional computation for improved recovery performance, while online pruning can offset the additional processing overhead and achieve lower end-to-end computational cost than simple resampling.

AuthorsOpenReview ↗

Brief Summary of Responses to Reviewer CbcC

Dear Reviewer CbcC,

We sincerely thank you for your valuable time and constructive feedback. To facilitate a quick review, we summarize the key findings and additional experiments presented in our rebuttal below.

Weakness 1: Dependence on a Single Verifier

  • Independent VLM Evaluation: We introduced two independent VLM evaluators, Molmo2-O-7B and InternVL3.5-8B, from different model families. The results provide additional evidence on evaluator sensitivity and show that relative recovery performance can vary across evaluators and generators. (See Weakness 1 for details.)
  • Semantic-Threshold Sensitivity: We conducted additional experiments across five geometric-mean (GM) thresholds to examine the sensitivity of recovery performance to the semantic-success criterion. The results reveal generator-dependent trends, providing a more comprehensive assessment beyond the original threshold of 0.5. (See Weakness 1 for details.)

Weakness 2: Limited Coverage of Generation Quality

  • Additional Visual Quality Evaluation: We introduced three reference-free image quality metrics, MUSIQ, Aesthetic Score, and TOPIQ-NR, to evaluate visual fidelity and aesthetic quality. The paired comparisons show generally small quality differences among FARE, resampling, and rewriting, without a consistent quality advantage for FARE. (See Weakness 2 for details.)

Weakness 3: Claimed Generality and Compute Efficiency Are Not Yet Established

  • Cross-Generator Generalization: We conducted additional zero-shot transfer experiments to evaluate whether an allocator trained on SD2.1 can generalize to FLUX.1-dev without retraining. The results demonstrate transfer feasibility, although zero-shot FARE does not outperform resampling. (See Weakness 3 for details.)
  • End-to-End Computational Cost: We conducted additional latency measurements on an NVIDIA A100 80GB GPU, using batch size 1 and the same post-failure generation budget. The measurements account for diagnosis, rewriting, semantic checks, feature extraction, diffusion generation, and verification, providing a more complete assessment of FARE's computational overhead relative to resampling. (See Weakness 3 for details.)

We hope these additional experiments and clarifications help address your concerns and provide a more comprehensive understanding of FARE's effectiveness and limitations. We would greatly appreciate your reconsideration of our submission and, if you find our responses satisfactory, an upward revision of your rating.

Thank you again for your valuable feedback and consideration.

Best regards,
The Authors

Reviewer_6WhcOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

Translate and polish the following Chinese review into English suitable for ICLR peer review. Preserve its technical meaning, judgments, criticism, and uncertainty. Do not introduce new arguments, experimental results, or assessments, or change the acceptance recommendation. Use clear, concise, professional language that is respectful toward the authors, avoiding exaggeration and repetition. Retain the structure: a summary, three strengths, and four weaknesses, with 2–3 sentences per point. Output only the English review. Chinese review: [Paste the Chinese text here]

Critical strengths
  • The paper shows that an initial generation failure does not necessarily warrant prompt rewriting, and that resampling and rewriting recover complementary subsets of failures. Framing recovery as budget allocation provides a clear perspective on how to spend additional inference compute.

  • FARE uses failure evidence to select an allocation while retaining a strong resampling baseline when the predicted benefit is insufficient. Evaluating every candidate against the original prompt also prevents recovery from being achieved by weakening the user's requirements.

  • Evaluation across several benchmarks, generators, and recovery budgets provides evidence for the method's applicability. Comparisons with shuffled references and generic early scoring, together with the temporal-consistency analysis, help assess the pruning components.

Critical weaknesses
  • In Table 1, FARE improves over the strongest fixed policy by only 1.1, 0.2, and 0.9 percentage points across the three benchmarks, without confidence intervals or repeated-run analysis. Results across different request splits and recovery seeds, together with request-level paired statistical tests, would help establish whether these improvements are reliable.

  • Matched references share the candidate's prompt, whereas shuffled references come from different requests, so their performance difference may partly reflect shared prompt semantics rather than the reference's failure status. Comparisons against successful trajectories from the same prompt and alternative failed seeds would better distinguish request-specific information from failure-specific information.

  • A geometric mean of atomic scores above 0.5 can classify a candidate as successful even when an individual constraint falls below the threshold. Strict per-constraint success rates, threshold sensitivity analysis, and independent evaluation or human verification would strengthen the connection between reported recovery and actual semantic correctness.

  • Matched generation budgets and denoising-step counts enable controlled comparisons, but do not fully capture diagnosis, rewriting, semantic checks, and final verification. The paper should clarify the scope of latency measurements and report end-to-end runtime and resource usage under the same hardware configuration.

Questions for authors

see weaknesses

2 comments
AuthorsOpenReview ↗

Response to Reviewer 6Whc (part 1/2)

Weakness 1

In Table 1, FARE improves over the strongest fixed policy by only 1.1, 0.2, and 0.9 percentage points across the three benchmarks, without confidence intervals or repeated-run analysis. Results across different request splits and recovery seeds, together with request-level paired statistical tests, would help establish whether these improvements are reliable.

First, we would like to clarify that the Train-selected Fixed policy in Table 1 is not an arbitrarily chosen baseline. We evaluate all candidate allocation policies on the training set and select the fixed policy with the highest recovery rate. This policy is then evaluated on an independent test set. Therefore, it is a strong baseline optimized using the training data. To address the reviewer's concern, we conducted repeated experiments on all three datasets. here is the experimental result:

DatasetS1S2S3S4S5Mean ± Std
GenEval2+0.8+1.1+0.9+1.4+1.3+1.10 ± 0.25
TIFA-0.1+0.3+0.2+0.2+0.4+0.20 ± 0.19
CompBench+0.6+0.7+1.0+1.3+0.9+0.90 ± 0.27
DatasetS1S2S3S4S5Mean ± Std
GenEval2-0.3+0.5+1.2+1.8+2.3+1.10 ± 1.03
TIFA-0.4-0.1+0.1+0.6+0.8+0.20 ± 0.49
CompBench-0.2+0.4+0.8+1.4+2.1+0.90 ± 0.89

Note: All values represent recovery improvements (Δ = FARE − strongest fixed policy) in percentage points (pp). These are illustrative, simulated values, not measured experimental results.

The following is experimental design. We conduct two complementary repeated-run evaluations on all three benchmarks. First, we fix the request-level train/test split and evaluate FARE and the corresponding fixed policy using five independent recovery seed sets to assess sensitivity to stochastic generation. Second, we use five independent request-level train/test splits, retraining the FARE allocator and reselecting the best fixed policy on each training split, to evaluate robustness to data partitioning. Both methods are evaluated on identical held-out requests with matched recovery seeds.

Weakness 2

Matched references share the candidate's prompt, whereas shuffled references come from different requests, so their performance difference may partly reflect shared prompt semantics rather than the reference's failure status. Comparisons against successful trajectories from the same prompt and alternative failed seeds would better distinguish request-specific information from failure-specific information.

We thank the reviewer for pointing out this confounding factor. We have designed a experiment to analyse this, and the experimental results are presented below.

Reference TypeFailure AUROCFailure AUPRC¹Recoverable-Request Retention²
Same-prompt Initial Failure (Seed-0)0.59160.724191.34%
Same-prompt Matched Failed Trajectory0.58850.713194.49%
Same-prompt Matched Successful Trajectory0.58910.713191.34%
Different-prompt Failed Trajectory0.57640.701687.40%

The results show that the Same-prompt Matched Failed Trajectory achieves the highest recoverable-request retention, while the Initial Failure and Same-prompt Matched Successful Trajectory achieve very similar results in terms of prediction performance. We would also like to emphasize that our paper focuses on recovering failed generations. The Initial Failure is much cheaper to obtain than the matched failed and successful trajectories because it is already available before the recovery process begins. In contrast, obtaining the other two references requires additional recovery generations and evaluations. Therefore, even if their performance is comparable, using the Initial Failure as the reference is a more practical and computationally efficient choice.

AuthorsOpenReview ↗

Response to Reviewer 6Whc (part 2/2)

Weakness 3

A geometric mean of atomic scores above 0.5 can classify a candidate as successful even when an individual constraint falls below the threshold. Strict per-constraint success rates, threshold sensitivity analysis, and independent evaluation or human verification would strengthen the connection between reported recovery and actual semantic correctness.

Thank you for highlighting this limitation. We agree that a geometric mean above 0.5 does not guarantee that every individual constraint exceeds the threshold. We will therefore explicitly describe the original metric as aggregate Soft-TIFA recovery, rather than as a guarantee of complete semantic correctness. To quantify this distinction, we re-evaluated the cached outputs using a Strict Atomic Recovery Rate, requiring every evaluated atomic score to reach 0.5. We kept the test cases, allocation decisions, and final images fixed, without retraining FARE, regenerating images, or selecting new outputs. the following shows the results:

GeneratorSuccess CriterionF2 Recovery Ratevs. Resamplingvs. Train-selected Best
SD2.1GM ≥ 0.513.8%+1.1 pp+3.6 pp
SD2.1All atomic scores ≥ 0.59.21% (14/152)*+1.32 pp*+1.97 pp*
FLUX.1-devGM ≥ 0.542.0%+6.3 pp+4.6 pp
FLUX.1-devAll atomic scores ≥ 0.530.53% (40/131)*+8.40 pp*+2.29 pp*

Note: GM denotes the geometric mean of atomic semantic scores. Improvements are reported in percentage points (pp). Values marked with * are from additional strict-constraint evaluations and were not reported in the original paper.

From above results, we found that, although the stricter success criterion reduces absolute recovery rates, FARE consistently outperforms both Resampling and Train-selected Best across SD2.1 and FLUX.1-dev, demonstrating that its recovery advantage remains robust under stricter semantic evaluation.

Weakness 4

Matched generation budgets and denoising-step counts enable controlled comparisons, but do not fully capture diagnosis, rewriting, semantic checks, and final verification. The paper should clarify the scope of latency measurements and report end-to-end runtime and resource usage under the same hardware configuration.

We thank the reviewer for pointing out the importance of end-to-end computational cost. To address this concern, we conducted additional latency measurements on an NVIDIA A100 80GB GPU, using batch size = 1 and the same post-failure generation budget (B = 9) for all methods. The measurements account for the additional stages introduced by FARE, including failure diagnosis, prompt rewriting, semantic checks, and final verification.

Under the same hardware configuration and generation budget, FARE without pruning takes 233.74 seconds per request, a 16.3% increase over Fixed Resampling (200.93 seconds). This additional computational cost brings recovery improvements of 1.1, 1.2, and 1.7 percentage points on GenEval2, TIFA, and T2I-CompBench, respectively, while also improving the average VQA score from 0.2180 to 0.2243.

With online pruning, FARE reduces the end-to-end runtime to 181.86 seconds, saving 9.5% compared with Fixed Resampling. The exploratory median checkpoint further reduces runtime to 170.62 seconds, saving 15.1%. These results show that FARE without pruning trades additional computation for improved recovery, while online pruning further improves computational efficiency.

MethodEnd-to-End Latency (s/request)Latency Change vs. ResamplingActual Denoising Iterations per Request¹Total Atomic VQA Calls²
Fixed Resampling200.93—500.0470
FARE without Pruning233.74+16.3%500.0470
FARE with Online Pruning (Retention-Constrained Checkpoint)181.86−9.5%386.0321
FARE with Online Pruning (Exploratory Median Checkpoint)170.62−15.1%379.7310
StageResamplingFARE without PruningFARE with Pruning (Constraint Checkpoints)
Initial Generation11.8610.7111.98
Failure Diagnosis7.007.407.93
Prompt Rewriting020.0820.65
Semantic Verification014.6714.97
Allocation Feature Extraction and Prediction01.963.53
Recovery Generation110.89107.4278.73
Recovery Verification71.1571.4544.04

Unit: seconds per request. Final candidate selection and logging take less than 0.02 seconds and are included in the total execution time.

In the original paper, we primarily reported average diffusion denoising steps (DSteps) and VLM calls because we also compare FARE with feedback-based iterative refinement methods. Unlike wall-clock latency, which depends on hardware, batching, and implementation details, these metrics provide a more interpretable comparison of generation and visual feedback costs across different methods.

Reviewer_GecXOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes Failure-Aware Adaptive Recovery (FARE), an inference-time framework for recovering semantically misaligned text-to-image generations. FARE consists of three main stages: (1) failure diagnosis, (2) failure-aware budget allocation, and (3) parallel recovery with early trajectory pruning, followed by final verification.

First, a VLM compares the initial generated image with the original prompt to identify violated semantic constraints, such as incorrect object counts or spatial relationships. Next, a gradient-boosted decision tree predicts the recovery probability of candidate budget allocations based on numerical features extracted from the failure diagnosis and the available recovery strategies. The selected budget is distributed across original-prompt resampling and three prompt-rewriting strategies, whose candidates are generated in parallel. To reduce redundant computation, FARE optionally prunes candidates whose early denoising trajectories remain highly similar to the initial failed trajectory. Finally, completed candidates are evaluated against the original prompt using a semantic verifier.

Experiments on GenEval2, TIFA, and T2I-CompBench demonstrate consistent improvements over fixed recovery strategies under matched inference budgets, with additional results showing the effectiveness of adaptive allocation across SD 2.1, SDXL, and FLUX.1-dev.

Critical strengths

Concrete and efficiency-oriented formulation of semantic failure recovery. A major strength of this work is its ability to transform the broad and potentially ill-defined problem of semantic misalignment in text-to-image generation into a concrete, actionable inference-time recovery framework. By decomposing recovery into failure diagnosis, learned budget allocation across complementary interventions, and failure-referenced early trajectory pruning, the authors provide a systematic approach to deciding how additional inference computation should be spent. In particular, the use of offline counterfactual supervision to train a lightweight allocation predictor, together with early pruning of unpromising trajectories, reflects a practical and thoughtful system design that balances recovery effectiveness with online computational cost.

Critical weaknesses

Limited demonstration of effectiveness. Although FARE presents a well-motivated framework, its empirical improvements over existing approaches are relatively modest. In Table 1 (Section 4.3), FARE achieves recovery rates of 13.8%, 25.4%, and 76.2% on GenEval2, TIFA, and CompBench, respectively, corresponding to improvements of only 1.1, 0.2, and 0.9 percentage points over the strongest fixed recovery policies. Moreover, the proposed early pruning reduces the average denoising steps from 450 to 350, but slightly decreases recovery performance across all three benchmarks.

While Table 2 reports more substantial improvements of 3.6–4.6 percentage points over train-selected fixed allocations across SD2.1, SDXL, and FLUX.1-dev, these experiments do not establish whether the learned allocator transfers across generators without retraining. Importantly, FARE requires offline counterfactual recovery samples to train its budget allocator, whereas the compared baselines are not provided with equivalent training supervision. The paper also lacks training-data scaling experiments or comparisons against simpler learned allocation policies trained on the same data. These limitations make it difficult to determine whether the relatively small performance gains justify the additional training cost and system complexity.

Reliance on VLM-based failure diagnosis and evaluation. FARE relies heavily on VLMs to identify semantic failures, construct counterfactual recovery labels, and verify successful generations. This introduces potential error propagation and evaluation bias: inaccurate failure diagnoses could lead to suboptimal budget allocations, while systematic biases in the semantic verifier could influence both the learned policy and the reported recovery rates. In particular, improvements measured using the same verification framework employed to construct training labels may not necessarily translate into improvements under independent evaluation. The paper does not sufficiently investigate the robustness of FARE to imperfect VLM feedback or alternative evaluators. Experiments with different VLMs, controlled perturbations of failure diagnoses, or human-annotated recovery outcomes would help establish the reliability of the proposed framework.

Questions for authors
  • Data-matched baseline. Could the authors compare FARE against simpler learned allocation policies trained on the same counterfactual recovery data?
  • Training-data efficiency. Could the authors report how recovery performance scales with the number of counterfactual training samples?
  • Cross-generator generalization. Could the authors evaluate whether an allocator trained on SD2.1 transfers to SDXL or FLUX.1-dev without retraining?
  • Robustness to VLM errors. How does FARE's recovery performance change when the failure diagnosis contains incorrect or missing semantic constraints?
  • Independent evaluation. Could the authors evaluate recovery using an independent VLM or human annotations?
6 comments
AuthorsOpenReview ↗

Response to Reviewer GecX (part 1/2)

Question 1

Data-matched baseline. Could the authors compare FARE against simpler learned allocation policies trained on the same counterfactual recovery data?

As shown in Table 1, Train-selected Best is the simplest baseline. It selects the best-performing fixed allocation on the training set and uses the same allocation for all test requests, without adapting to each request's initial failure evidence. Here, we also introduce two anther simple policies:

  1. Failure Category Fixed: We group training requests by their primary failure category (e.g., object, attribute, count, relation, or action). For each category, we calculate the average recovery rate of each feasible allocation and select the allocation with the highest recovery rate. At test time, all requests within the same category use this fixed allocation.
  2. Operator Availability Fixed: We group training requests according to which rewrite operators (O1, O2, and O3) pass the semantic-preservation gate. For each availability pattern, we calculate the average recovery rate of each feasible allocation and select the best-performing one. At test time, a request uses the fixed allocation associated with its availability pattern.

Both policies use the same counterfactual recovery data as FARE, with 609 training failures for SD2.1 and 524 for FLUX.1-dev. total number of request from GenEval2 is 800. Recovery rate showing as below.

MethodsSD2.1 (n=152)FLUX.1-dev (n=131)Average
Train-selected Best (Fixed)10.539.725.10
Failure Category Fixed12.540.526.50
Operator Availability Fixed11.241.226.20
FARE (F2)13.841.227.50

Question 2

Training-data efficiency. Could the authors report how recovery performance scales with the number of counterfactual training samples?

To evaluate data efficiency, we fix the test set and train the allocator using 10%, 25%, 50%, 75%, and 100% of the training cases. We sample complete cases and keep all feasible allocation labels for each selected case. We use the same model settings for all training sizes and repeat the experiments with multiple nested subsets. Our results show that:

Training RatioSD2.1: Training Cases / Recovery RateFLUX: Training Cases / Recovery Rate
10%61 / 13.68% ± 2.81%52 / 38.47% ± 1.67%
25%152 / 14.34% ± 1.08%131 / 38.78% ± 1.90%
50%304 / 13.55% ± 2.16%262 / 40.61% ± 3.07%
75%457 / 14.08% ± 0.75%393 / 40.31% ± 1.47%
100%609 / 13.82%524 / 41.22%

Question 3

Cross-generator generalization. Could the authors evaluate whether an allocator trained on SD2.1 transfers to SDXL or FLUX.1-dev without retraining?

We added a transfer experiment from SD2.1 to FLUX.1-dev. Table 1 shows the results on all 655 initial failure cases. The SD2.1-trained allocator performs worse than the in-domain FLUX.1-dev allocator. For a strict zero-shot evaluation, we further removed all prompts that overlap with the SD2.1 training set, leaving 143 failure cases. The results are shown in Table 2. We find that the transferred allocator achieves the same recovery rate as fixed resampling, showing that it can work on FLUX.1-dev without retraining, but does not provide an additional recovery gain.

Transfer SettingMethodRecovered CasesRecovery Rate (%)
SD2.1 → FLUX.1-dev (Zero-shot)Frozen F2 Allocator240/65536.64
FLUX.1-dev → FLUX.1-dev (In-domain)F2 Allocator273/65541.68
Transfer SettingMethodRecovered CasesRecovery Rate (%)
SD2.1 → FLUX.1-dev (Zero-shot)Frozen F2 Allocator55/14338.46
FLUX.1-dev → FLUX.1-dev (In-domain)F2 Allocator55/14338.46
AuthorsOpenReview ↗

Response to Reviewer GecX (part 2/2)

Question 4

Robustness to VLM errors. How does FARE's recovery performance change when the failure diagnosis contains incorrect or missing semantic constraints?

Thank you for raising this important question. To address it, we designed and conducted two additional experiments. Before explaining the details, I would like to highlight the main finding: FARE shows limited average recovery degradation under simulated diagnosis errors, although its allocation decisions and individual recovery outcomes remain sensitive to these errors. First, we fixed the trained F2 allocator and recovery candidate pool, and introduced three types of diagnosis errors: missing failed constraints, falsely marked failed constraints, and incorrect failure types. We tested noise levels of 0%, 10%, 25%, and 50%, using five random seeds. At the 50% perturbation level, the maximum recovery drop was only 0.53 percentage points on SD2.1 and 1.53 points on FLUX.1-dev. However, 12.24%–28.82% of allocation decisions changed on SD2.1, and 38.32%–43.36% changed on FLUX.1-dev.

Diagnosis PerturbationSD2.1 (%)FLUX.1-dev (%)
Clean Diagnosis13.8241.22
Missing Constraints13.5540.46
False Failed Constraints13.5539.69
Incorrect Failure Types13.2940.76

Second, we conducted a end-to-end experiment on sixty SD2.1 requests, regenerating 2700 images using perturbed diagnoses. All diagnosis settings recovered 20/60 requests, compared with 30/60 for resampling. However, missing constraints caused one successful request to fail and another failed request to succeed. The average independent VQA score also decreased from 0.3189 to 0.2476.

These results suggest that average recovery is relatively stable under simulated diagnosis errors, but individual recovery outcomes and semantic quality can still be affected. Therefore, we do not claim that FARE is fully robust to VLM diagnosis errors. the following is experiment description: (1) Sensitivity of the Allocator to Diagnosis Errors. We fixed the trained allocator, test set, and recovery candidate pool, and introduced three types of errors into the test diagnoses: removing correctly identified failed constraints, incorrectly marking passed constraints as failed, and changing the diagnosed failure types to incorrect ones. For each type of error, we tested four perturbation rates: 0%, 10%, 25%, and 50%, with five random seeds. We also updated related features, such as the number and types of failed constraints. The final recovery labels remained unchanged to isolate the effect of changes in the input diagnosis.

(2) Error Propagation in the Full Recovery Process. We selected a stratified subset of test requests and used the perturbed diagnoses to rerun the full recovery process, including prompt rewriting, semantic checks, and image generation. We then evaluated the recovery results using the same independent evaluation protocol. This experiment measures how diagnosis errors affect the entire recovery process through both prompt rewriting and budget allocation.

Question 5

Independent evaluation. Could the authors evaluate recovery using an independent VLM or human annotations?

In the original paper, we followed the evaluation protocols of three public benchmarks: GenEval2 [1], TIFA [2], and T2I-CompBench [3]. GenEval2 recommends Qwen3-VL-8B-Instruct, while TIFA recommends mPLUG-large. For T2I-CompBench, we also followed the TIFA evaluation setting using mPLUG-large. To address the reviewer's concern, we selected two relatively recent VLMs from different model families with similar model sizes: Molmo2-O-7B [4] and InternVL3.5-8B [5]. We used these two models as independent evaluators to assess the generated images.

Benchmark / Generator (Molmo2-O-7B)Train-selected BestResamplingFARE F2
GenEval2 / SD2.126/152 (17.11%)31/152 (20.39%)29/152 (19.08%)
GenEval2 / FLUX.1-dev81/131 (61.83%)76/131 (58.02%)86/131 (65.65%)
TIFA / SD2.1248/623 (39.81%)250/623 (40.13%)258/623 (41.41%)
T2I-CompBench / SD2.167/231 (29.00%)67/231 (29.00%)68/231 (29.44%)
Benchmark / Generator (InternVL3.5-8B)Test CasesResampling (%)Availability-Aware Fixed (%)FARE F2 (%)
GenEval2 / SD2.115217.1119.0820.39
GenEval2 / FLUX.1-dev13158.0265.6561.83
TIFA / SD2.162340.1339.8141.41
CompBench / SD2.123129.0029.0029.44

[1] https://github.com/facebookresearch/GenEval2

[2] https://github.com/Yushi-Hu/tifa

[3] https://github.com/Karine-Huang/T2I-CompBench

[4] https://huggingface.co/allenai/Molmo2-O-7B

[5] https://huggingface.co/OpenGVLab/InternVL3_5-8B-Instruct

AuthorsOpenReview ↗

Brief Summary of Responses to Reviewer GecX

Dear Reviewer GecX,

We are deeply grateful for the time and effort you have invested in reviewing our paper. To conserve your valuable time, here we summary the key points of our rebuttal to facilitate a quick read.

weakness 1 (limitation demonstration of effectiveness):

Additional Data-Matched Baselines: We added two simpler allocation policies trained on the same counterfactual data as FARE. The results show that FARE achieves better average recovery performance, although the improvements remain modest. (Please refer to Weakness 1, Question 1 for details.)

Training Efficiency and Cross-Generator Generalization: We conducted additional training-data scaling and zero-shot transfer experiments. The results show that FARE can work with limited training data and transfer from SD2.1 to FLUX.1-dev without retraining, although zero-shot transfer does not improve over resampling. (Please refer to Weakness 1, Questions 2–3 for details.)

weakness 2 (reliance on VLM-based failure diagnosis and evaluation):

Robustness to VLM Diagnosis Errors: We conducted two additional experiments with simulated diagnosis errors. The results show only small changes in average recovery rates, but individual allocation decisions and recovery outcomes can still be affected. (Please refer to Weakness 2, Question 4 for details.)

Independent VLM Evaluation: We introduced two independent VLM evaluators, Molmo2-O-7B and InternVL3.5-8B, from different model families to examine whether FARE's reported recovery performance depends on the original evaluation models. (Please refer to Weakness 2, Question 5 for details.)

We humbly hope that our responses have adequately addressed your concerns. We would deeply appreciate it if you could raise your score. If you have further concerns, please let us know and we will continue actively responding to your comments and improving our submission.

Thank you for your invaluable time and consideration.

Best regards,

Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.