acceptodds
Withdrawn from ICLR 2027

T2I-SEH: A Self-Evolving Text-to-Image Harness via Effect-Conditioned Harness Learning

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

21 views

Abstract

Text-to-image (T2I) generation has recently been exploring a shift from conventional single-shot model invocation toward agentic systems, but its runtime control relies heavily on certain predefined strategies. The self-evolving paradigm presents a promising direction; nevertheless, existing approaches still depend on accurate state interpretation and reliable experience transfer. We propose T2I-SEH, a self-evolving T2I harness that aims to determine when prior experience can be leveraged and to what extent it could influence subsequent actions. The core design is Effect-Conditioned Harness Learning (ECHL) which quantifies paired intervention gains, weights well-supported historical effects according to contextual similarity and uncertainty, and yields cost-aware diagnostic beliefs. This formulation emphasizes relevant and reliable experience, while preserving prior beliefs when sufficient evidence is unavailable. ECHL then integrates cost-adjusted utility bounds with Monte Carlo estimates to determine whether to intervene, acquire additional evidence, or retain the current image. Finally, diagnostic generations refine action-specific beliefs and consolidate observed effects for future reuse. Experiments on the compositional generation task across five architecturally diverse T2I generators demonstrate that T2I-SEH enables more effective intervention selection at substantially lower action cost. In addition, when the task aligns with available interventions, resulting benefits transfer into improved generation quality effectively as reflected in the official GenEval 2 scores. Further, evaluations on CoReBench and DPG-Bench show T2I-SEH’s robustness to the task–intervention mismatch by selectively exploiting residual cross-task utility when beneficial and otherwise defaulting to the null intervention at negligible action cost.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Self-evolving System, Harness Learning, Text-to-Image Generation

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_Q6j8OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper presents a controller (the author called it as "Harness") for t2i generation that decides whether to keep the first image or try a fix, such as rewriting the prompt or sampling again. It records how much each fix helped in similar past cases, weighs that experience by how relevant and reliable it is, and decides whether running another test is worth the cost.

Critical strengths
  1. Asking when past experience should be trusted seems timing and interesting.
  2. Measuring each intervention's improvement relative to the original image is a sensible way to separate a helpful intervention from an easy prompt.
  3. The method is tested on several different generators.
Critical weaknesses
  1. The main results are scored by the same vision model that the system optimizes against, so the gains may not reflect real quality.
  2. Improvements on external benchmarks seems small. For example, on GenEval 2, gains over Raw are often a fraction of a point. A simple baseline is missing, generate the same number of images and let the same verifier pick the best one.
  3. The intervention space is limited. Regenerating or rewriting the prompt feels trivial and does not show the potential of a self-evolving harness.
Questions for authors
  1. Results with different evaluators?
  2. Perform against a best-of-N baseline that generates the same number of images and uses the same verifier to choose?
9 comments

Dear Reviewer Q6j8,

Thank you for your efforts and time on our work, for your insightful comments and for your constructive suggestions. We are more than delighted that you consider the motivation of our work "timely and interesting", consider the proposed methodology "sensible", and recognize the evaluation across multiple generators as a key strength of our work. We fully agree with your constructive comments, which have helped us further improve our work. In the following, we provide detailed, point-by-point responses to each of them.

W1 and Q1

In Table 1, we employ Qwen2.5-VL-3B as both the internal observer and the atomic evaluator to assess the Task Success Rate and Constraint Satisfaction Rate, primarily to examine the effectiveness of internal decision-making. We fully agree with your suggestion to incorporate independent evaluators for a more comprehensive and robust assessment. We additionally introduce six vision models of varying parameter scales developed by different organizations as independent evaluators: HyperCLOVAX-SEED-Vision-Instruct-3B, InternVL3.5-4B, PLaMo 2.1-8B-VL, GLM-4V-9B, Llama-3.2-11B-Vision and Qwen3.8-27B. Although the absolute scores vary somewhat under independent evaluation, the overall trend remains consistent, confirming that T2I-SEH selects more appropriate interventions while incurring lower action costs.

Table 1. Task Success Rate (TSR) and Constraint Satisfaction Rate (CSR) evaluated with HyperCLOVAX-SEED-Vision-Instruct-3B.

BTSR RawTSR PSTSR SPTSR ERTSR ECHLCSR RawCSR PSCSR SPCSR ERCSR ECHL
SDXL126.9428.6121.94-30.8339.8640.3933.28-43.86
226.9428.6121.9432.7834.4439.8640.3933.2846.5846.67
426.9428.6121.9432.7836.3939.8640.3933.2846.5848.67
SD3.5M164.1756.9457.50-65.0077.5071.3970.14-78.17
264.1756.9457.5070.5667.7877.5071.3970.1481.3679.97
464.1756.9457.5070.5670.8377.5071.3970.1481.3681.69
FLUX.2 Klein174.7271.6767.78-75.5677.4475.1473.56-78.14
274.7271.6767.7876.6777.2277.4475.1473.5679.2279.75
474.7271.6767.7876.6777.7877.4475.1473.5679.2280.67
Z-Image-Turbo173.8971.9470.56-75.2881.8979.8977.53-82.92
273.8971.9470.5678.0678.3381.8979.8977.5384.8185.83
473.8971.9470.5678.0680.0081.8979.8977.5384.8186.83
Janus-Pro-7B149.7239.175.83-52.2264.0649.5017.11-66.78
249.7239.175.8355.0052.2264.0649.5017.1169.6967.00
449.7239.175.8355.0055.5664.0649.5017.1169.6969.89

Table 2. Task Success Rate (TSR) and Constraint Satisfaction Rate (CSR) evaluated with InternVL3.5-4B.

BTSR RawTSR PSTSR SPTSR ERTSR ECHLCSR RawCSR PSCSR SPCSR ERCSR ECHL
SDXL124.7227.5020.28-28.8935.2236.6129.33-38.78
224.7227.5020.2831.9432.2235.2236.6129.3342.5842.25
424.7227.5020.2831.9433.6135.2236.6129.3342.5843.50
SD3.5M156.1151.9450.56-58.8970.9766.8663.78-73.67
256.1151.9450.5662.7862.2270.9766.8663.7877.7576.83
456.1151.9450.5662.7864.7270.9766.8663.7877.7578.14
FLUX.2 Klein163.6160.8360.28-64.7273.5671.9271.56-74.22
263.6160.8360.2867.2267.5073.5671.9271.5675.9776.19
463.6160.8360.2867.2268.3373.5671.9271.5675.9777.39
Z-Image-Turbo175.8379.4467.78-79.7281.8385.1977.00-85.69
275.8379.4467.7879.7280.2881.8385.1977.0084.5886.00
475.8379.4467.7879.7280.5681.8385.1977.0084.5886.72
Janus-Pro-7B148.0643.336.11-52.2261.8153.1916.97-66.39
248.0643.336.1152.5052.2261.8153.1916.9767.1166.67
448.0643.336.1152.5054.7261.8153.1916.9767.1168.92

Table 3. Task Success Rate (TSR) and Constraint Satisfaction Rate (CSR) evaluated with PLaMo 2.1-8B-VL.

BTSR RawTSR PSTSR SPTSR ERTSR ECHLCSR RawCSR PSCSR SPCSR ERCSR ECHL
SDXL136.9440.0032.78-40.8347.8351.1440.81-51.97
236.9440.0032.7842.7843.6147.8351.1440.8153.0853.47
436.9440.0032.7842.7845.2847.8351.1440.8153.0855.58
SD3.5M162.7863.3363.06-64.7275.6775.5674.42-77.08
262.7863.3363.0663.6166.3975.6775.5674.4277.2578.39
462.7863.3363.0663.6167.2275.6775.5674.4277.2579.08
FLUX.2 Klein161.3960.0061.94-62.5076.4474.9476.89-77.11
261.3960.0061.9463.0663.6176.4474.9476.8978.0878.19
461.3960.0061.9463.0665.5676.4474.9476.8978.0879.56
Z-Image-Turbo168.8965.8362.22-69.1781.7277.6175.81-81.78
268.8965.8362.2268.8969.4481.7277.6175.8182.1182.31
468.8965.8362.2268.8969.1781.7277.6175.8182.1182.22
Janus-Pro-7B155.8345.838.06-56.1167.6954.3925.36-68.08
255.8345.838.0657.2256.1167.6954.3925.3668.5868.08
455.8345.838.0657.2257.5067.6954.3925.3668.5869.47

Table 4. Task Success Rate (TSR) and Constraint Satisfaction Rate (CSR) evaluated with GLM-4V-9B.

BTSR RawTSR PSTSR SPTSR ERTSR ECHLCSR RawCSR PSCSR SPCSR ERCSR ECHL
SDXL122.7819.1714.44-24.1733.0829.0824.03-34.33
222.7819.1714.4424.1724.7233.0829.0824.0334.7235.97
422.7819.1714.4424.1728.0633.0829.0824.0334.7238.03
SD3.5M151.1147.2249.17-52.5067.1163.0662.50-68.94
251.1147.2249.1755.0054.1767.1163.0662.5071.5070.58
451.1147.2249.1755.0055.5667.1163.0662.5071.5071.89
FLUX.2 Klein160.5657.7855.56-61.3972.7270.8665.50-72.92
260.5657.7855.5660.8362.7872.7270.8665.5074.3374.69
460.5657.7855.5660.8363.8972.7270.8665.5074.3375.44
Z-Image-Turbo161.9453.8949.17-63.3375.8369.7265.94-76.69
261.9453.8949.1765.2865.2875.8369.7265.9478.2878.56
461.9453.8949.1765.2866.1175.8369.7265.9478.2879.72
Janus-Pro-7B148.3339.722.50-50.8359.3648.5814.25-61.56
248.3339.722.5052.2250.8359.3648.5814.2563.1462.61
448.3339.722.5052.2253.3359.3648.5814.2563.1464.28

Table 5. Task Success Rate (TSR) and Constraint Satisfaction Rate (CSR) evaluated with Llama-3.2-11B-Vision.

BTSR RawTSR PSTSR SPTSR ERTSR ECHLCSR RawCSR PSCSR SPCSR ERCSR ECHL
SDXL125.0026.1121.11-29.4435.7236.9230.64-39.86
225.0026.1121.1130.5631.3935.7236.9230.6441.5642.08
425.0026.1121.1130.5634.1735.7236.9230.6441.5644.81
SD3.5M159.7259.1755.28-62.7873.0072.0068.78-75.44
259.7259.1755.2865.0065.0073.0072.0068.7878.0377.75
459.7259.1755.2865.0068.0673.0072.0068.7878.0379.64
FLUX.2 Klein160.2859.7260.56-60.8374.9773.8375.31-75.72
260.2859.7260.5662.2263.8974.9773.8375.3176.4477.72
460.2859.7260.5662.2265.5674.9773.8375.3176.4478.50
Z-Image-Turbo161.3961.9455.00-62.7876.7875.7269.69-77.64
261.3961.9455.0064.1764.7276.7875.7269.6979.2279.58
461.3961.9455.0064.1764.7276.7875.7269.6979.2280.03
Janus-Pro-7B148.8941.115.28-50.5661.7550.0019.25-63.78
248.8941.115.2852.7850.5661.7550.0019.2566.6963.78
448.8941.115.2852.7853.3361.7550.0019.2566.6966.81

Table 6. Task Success Rate (TSR) and Constraint Satisfaction Rate (CSR) evaluated with Qwen3.8-27B.

BTSR RawTSR PSTSR SPTSR ERTSR ECHLCSR RawCSR PSCSR SPCSR ERCSR ECHL
SDXL123.0626.1120.00-26.9436.0837.2530.50-39.56
223.0626.1120.0028.0628.3336.0837.2530.5040.3941.08
423.0626.1120.0028.0630.8336.0837.2530.5040.3943.42
SD3.5M154.7246.9446.39-55.5670.6163.3662.58-71.22
254.7246.9446.3960.0058.3370.6163.3662.5875.7273.28
454.7246.9446.3960.0060.5670.6163.3662.5875.7276.44
FLUX.2 Klein156.6756.9457.22-58.0668.2566.6168.42-68.75
256.6756.9457.2260.8361.1168.2566.6168.4270.2871.19
456.6756.9457.2260.8362.2268.2566.6168.4270.2871.94
Z-Image-Turbo170.5670.2861.67-72.2278.4479.0672.28-79.81
270.5670.2861.6773.3374.1778.4479.0672.2881.0681.89
470.5670.2861.6773.3375.0078.4479.0672.2881.0682.72
Janus-Pro-7B146.1139.445.28-49.1760.3349.4215.42-63.25
246.1139.445.2850.8349.1760.3349.4215.4263.7564.19
446.1139.445.2850.8351.3960.3349.4215.4263.7565.53

W2 (On GenEval 2)

We agree that the absolute improvements on external benchmarks are relatively modest, and would like to provide additional context to clarify their practical significance. Specifically, the achievable gains are inherently bounded by the capabilities of the frozen backbone models and the corrective capacity of the available intervention set. And the value of T2I-SEH, lies in unlocking these existing capabilities effectively through selective, budget-controlled intervention, without requiring generator retraining. It helps determine when additional computation is worthwhile, which intervention to apply, and when to retain the original image. Furthermore, to assess whether the observed improvements are consistent, we conduct additional experiments across a total of five independent seeds.

Table 7. Internal Success Rate (ISR), GenEval 2 Soft-TIFA and action cost across five independent random seeds, reporting the mean and standard deviation.

GeneratorBISR RawISR PSISR SPISR ERISR ECHLSoft-TIFA RawSoft-TIFA PSSoft-TIFA SPSoft-TIFA ERSoft-TIFA ECHLCost ECHL
SDXL161.15±0.8983.48±1.1151.08±1.41–75.95±3.847.97±0.396.93±0.375.35±0.39–8.38±0.360.15±0.08
261.15±0.8983.48±1.1151.08±1.4176.50±0.4689.03±0.687.97±0.396.93±0.375.35±0.398.15±0.508.59±0.270.28±0.01
461.15±0.8983.48±1.1151.08±1.4176.50±0.4691.95±0.737.97±0.396.93±0.375.35±0.398.15±0.508.73±0.370.34±0.02
SD3.5M179.98±0.6490.43±0.4972.35±2.54–92.10±1.6619.04±1.0616.31±0.4811.53±0.63–19.73±1.110.12±0.02
279.98±0.6490.43±0.4972.35±2.5491.25±0.8196.20±0.3419.04±1.0616.31±0.4811.53±0.6320.22±1.0419.92±1.020.16±0.01
479.98±0.6490.43±0.4972.35±2.5491.25±0.8197.80±0.4519.04±1.0616.31±0.4811.53±0.6320.22±1.0420.06±1.030.19±0.01
FLUX.2 Klein189.55±0.6597.35±0.3895.28±0.85–98.83±0.2925.48±0.8524.77±0.3023.51±0.54–26.78±0.900.09±0.01
289.55±0.6597.35±0.3895.28±0.8594.98±0.4399.63±0.1525.48±0.8524.77±0.3023.51±0.5426.14±0.8926.84±0.880.10±0.01
489.55±0.6597.35±0.3895.28±0.8594.98±0.4399.70±0.1425.48±0.8524.77±0.3023.51±0.5426.14±0.8926.84±0.880.10±0.01
Z-Image-Turbo194.83±0.3898.45±0.6097.23±0.45–99.05±0.3431.50±0.7928.74±0.8228.42±0.97–32.26±0.770.04±0.01
294.83±0.3898.45±0.6097.23±0.4597.13±0.3999.83±0.1431.50±0.7928.74±0.8228.42±0.9731.76±0.8432.27±0.790.05±0.00
494.83±0.3898.45±0.6097.23±0.4597.13±0.3999.90±0.0631.50±0.7928.74±0.8228.42±0.9731.76±0.8432.27±0.790.05±0.00
Janus-Pro-7B167.48±1.9280.25±0.9434.05±1.24–86.10±0.6413.26±0.3711.68±0.525.19±0.25–13.73±0.500.09±0.05
267.48±1.9280.25±0.9434.05±1.2478.85±1.1589.03±0.6913.26±0.3711.68±0.525.19±0.2514.24±0.2914.35±0.250.22±0.02
467.48±1.9280.25±0.9434.05±1.2478.85±1.1590.98±0.7213.26±0.3711.68±0.525.19±0.2514.24±0.2914.47±0.270.25±0.02
Reviewer_E9bnOpenReview ↗
Rating
2
Confidence
3
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

This paper proposes that existing self-evolving harnesses often suffer from negative experience transfer, blindly applying past corrective actions to mismatched contexts, which wastes compute or degrades images. To address this, the paper proposes T2I-SEH powered by Effect-Conditioned Harness Learning (ECHL). ECHL formulates intervention selection as a cost-aware Bayesian decision process. It models paired intervention gains relative to the raw image, maintains precision-weighted Gaussian beliefs over action efficacy, and uses Monte Carlo estimates of Value of Information (VOI) per unit cost to decide whether to intervene, run diagnostic test generations, or simply retain the raw image. The system is evaluated across five generators (SDXL, SD3.5M, FLUX.2 Klein, Z-Image-Turbo, Janus-Pro-7B) on a controlled dataset, GenEval 2, CoReBench, and DPG-Bench.

Critical strengths

1. Focus on cost and knowing when to stop. Most agent papers ignore generation cost and keep calling tools endlessly. Having a clear rule that chooses to do nothing when an edit is unlikely to help is very practical.

2. Good generality. The authors test on five distinct models (diffusion, flow, and autoregressive) instead of tuning for just one, showing good generality across backbones.

Critical weaknesses

1. The action space is too simple for such a complex method. The paper builds a heavy Bayesian framework with Gaussian beliefs and Monte Carlo estimation, but only uses it to pick among three simple options: rewrite prompt, make a plan, or resample. Real-world image agents need to use actual tools like inpainting or ControlNet. Building such a complex setup for a 3-way choice feels like over-engineering.

2. Small quality gains. On the main GenEval 2 benchmark, the score only improves by about +0.5 to +1.1 points over the raw baseline. While it does save cost, the actual boost in image quality is very marginal.

Questions for authors
  1. Can this framework scale to real-world image tools like inpainting or regional editing, or does the VOI search become too slow as you add more tools?

  2. The gains on GenEval 2 are quite small (around +0.5 to +1.1 points). Were these results from a single run or averaged over multiple random seeds? Can you provide standard deviations or repeated test results to show that these small gains are statistically significant rather than random noise?

  3. How does the memory scale over time? Does searching past experiences get slow when you have thousands of requests?

3 comments

Dear Reviewer E9bn,

Thank you for your efforts and time on our work, for your insightful comments and for your constructive suggestions. We are more than delighted that you consider the proposed methodology is "very practical" and have "good generality across backbones". We fully agree with your constructive comments, which have helped us further improve our work. In the following, we provide detailed, point-by-point responses to each of them.

W1 and Q1

We sincerely appreciate your valuable suggestion. Following the constructive feedback from you, we further extend the ECHL framework to image editing tasks to assess its broader application. We evaluate ECHL on single-turn image editing tasks from MagicBrush, using an eight-action space organized into four tool families, each with two parameter presets: (1) Global Editing (GE), with denoising strengths of 0.45 (P0) and 0.70 (P1), allowing different degrees of modification to the source image; (2) Local Editing (LE), using either the original editing mask (P0) or a mask dilated by 16 pixels to vary the editable region (P1); (3) ControlNet Global Editing (CN_GE), combining the same two denoising strengths with Canny edge guidance; and (4) ControlNet Local Editing (CN_LE), combining structural guidance with either the original or 16-pixels dilated mask. Lower denoising strength favors source-image preservation, whereas higher strength permits more substantial modifications. The original mask targets the annotated editing region, while dilation provides additional flexibility for boundary adjustments. ControlNet encourages structural consistency with the source image, with edge constraints relaxed inside regions requiring structural changes. ECHL therefore jointly selects the editing tool and its operating regime, balancing editing intensity, spatial locality, structural preservation, and invocation cost under a limited tool-call budget. We employ Qwen3.5-4B as the internal observer to guide the intervention selection process.

Table 1. Results of single-turn image editing tasks from MagicBrush with eight actions.

L1↓L2↓CLIP-I↑DINO↑CLIP-T↑
SDXLGE P00.0780890.0208460.8841070.8266960.288192
GE P10.1073970.0296160.8259770.7229920.290616
LE P00.0540320.0169150.9357480.9003620.299288
LE P10.0545840.0170780.9352620.8982170.300191
CN_GE P00.0705630.0184770.8950620.8491950.288602
CN_GE P10.0887990.0222920.8663420.8068190.292044
CN_LE P00.0454320.0153640.9358970.8931680.296688
CN_LE P10.0472170.0157580.9321600.8866620.297665
ECHL0.0438680.0153510.9359110.8933960.299221
SD3.5MGE P00.0763470.0203460.8901510.8292530.292248
GE P10.1085480.0303160.8350950.7240240.296579
LE P00.0552590.0215230.9285200.8747170.304173
LE P10.0600240.0238260.9219280.8593090.304432
CN_GE P00.0727010.0189480.8979760.8394020.291071
CN_GE P10.0963700.0249940.8572750.7727100.295701
CN_LE P00.0527070.0192900.9317810.8828170.301682
CN_LE P10.0576620.0218630.9266720.8688350.301264
ECHL0.0460540.0181250.9319370.8893800.300235
FLUX.2 KleinGE P00.0919560.0262500.9205380.8698250.308368
GE P10.0641800.0187700.9313390.8847410.309277
LE P00.0394730.0153900.9388630.9044030.304442
LE P10.0401120.0155670.9382530.9009640.304427
CN_GE P00.0824670.0228790.9220320.8748630.304917
CN_GE P10.0605510.0179940.9320310.8889950.306933
CN_LE P00.0403450.0160220.9369820.8995200.303490
CN_LE P10.0408690.0161590.9363030.9003600.303082
ECHL0.0394310.0152090.9391210.9096950.302017
Z-Image-TurboGE P00.0903170.0236600.8681530.7950980.290465
GE P10.1347040.0384470.8002970.6714150.295975
LE P00.0519630.0212450.9251700.8631950.302954
LE P10.0575970.0239170.9198290.8469940.302678
CN_GE P00.2682840.1162160.7720930.6128360.295007
CN_GE P10.2343330.0912010.8021110.6897520.292025
CN_LE P00.0607490.0150530.9280390.8798290.301380
CN_LE P10.0624420.0158350.9271370.8733220.303038
ECHL0.0392400.0154640.9318620.8993330.291932
Janus-Pro-7BGE P00.1379300.0419370.7495750.6058910.256327
GE P10.2168080.0827470.6500260.3338750.223877
LE P00.0470920.0196970.9241700.8786200.285011
LE P10.0524580.0221620.9204680.8687310.284132
CN_GE P00.1003630.0255330.8582160.8003660.279331
CN_GE P10.1318090.0375450.8064960.7183380.267074
CN_LE P00.0462140.0191300.9269680.8841540.287380
CN_LE P10.0503580.0212370.9234890.8740970.286392
ECHL0.0392200.0154510.9315430.8992110.291647

Additionally, as VOI search is parallelized across multiple CPU cores, we did not observe a noticeable increase in VOI search latency when expanding the action space from 3 to 8 interventions in the experiments.

W2 and Q2

We agree that the absolute improvements on external benchmarks are relatively modest, and would like to provide additional context to clarify their practical significance. Specifically, the achievable gains are inherently bounded by the capabilities of the frozen backbone models and the corrective capacity of the available intervention set. And the value of T2I-SEH, lies in unlocking these existing capabilities effectively through selective, budget-controlled intervention, without requiring generator retraining. It helps determine when additional computation is worthwhile, which intervention to apply, and when to retain the original image. Furthermore, to assess whether the observed improvements are consistent, we conduct additional experiments across a total of five independent seeds.

Table 2. Internal Success Rate (ISR), GenEval 2 Soft-TIFA and action cost across five independent random seeds, reporting the mean and standard deviation.

GeneratorBISR RawISR PSISR SPISR ERISR ECHLSoft-TIFA RawSoft-TIFA PSSoft-TIFA SPSoft-TIFA ERSoft-TIFA ECHLCost ECHL
SDXL161.15±0.8983.48±1.1151.08±1.41–75.95±3.847.97±0.396.93±0.375.35±0.39–8.38±0.360.15±0.08
261.15±0.8983.48±1.1151.08±1.4176.50±0.4689.03±0.687.97±0.396.93±0.375.35±0.398.15±0.508.59±0.270.28±0.01
461.15±0.8983.48±1.1151.08±1.4176.50±0.4691.95±0.737.97±0.396.93±0.375.35±0.398.15±0.508.73±0.370.34±0.02
SD3.5M179.98±0.6490.43±0.4972.35±2.54–92.10±1.6619.04±1.0616.31±0.4811.53±0.63–19.73±1.110.12±0.02
279.98±0.6490.43±0.4972.35±2.5491.25±0.8196.20±0.3419.04±1.0616.31±0.4811.53±0.6320.22±1.0419.92±1.020.16±0.01
479.98±0.6490.43±0.4972.35±2.5491.25±0.8197.80±0.4519.04±1.0616.31±0.4811.53±0.6320.22±1.0420.06±1.030.19±0.01
FLUX.2 Klein189.55±0.6597.35±0.3895.28±0.85–98.83±0.2925.48±0.8524.77±0.3023.51±0.54–26.78±0.900.09±0.01
289.55±0.6597.35±0.3895.28±0.8594.98±0.4399.63±0.1525.48±0.8524.77±0.3023.51±0.5426.14±0.8926.84±0.880.10±0.01
489.55±0.6597.35±0.3895.28±0.8594.98±0.4399.70±0.1425.48±0.8524.77±0.3023.51±0.5426.14±0.8926.84±0.880.10±0.01
Z-Image-Turbo194.83±0.3898.45±0.6097.23±0.45–99.05±0.3431.50±0.7928.74±0.8228.42±0.97–32.26±0.770.04±0.01
294.83±0.3898.45±0.6097.23±0.4597.13±0.3999.83±0.1431.50±0.7928.74±0.8228.42±0.9731.76±0.8432.27±0.790.05±0.00
494.83±0.3898.45±0.6097.23±0.4597.13±0.3999.90±0.0631.50±0.7928.74±0.8228.42±0.9731.76±0.8432.27±0.790.05±0.00
Janus-Pro-7B167.48±1.9280.25±0.9434.05±1.24–86.10±0.6413.26±0.3711.68±0.525.19±0.25–13.73±0.500.09±0.05
267.48±1.9280.25±0.9434.05±1.2478.85±1.1589.03±0.6913.26±0.3711.68±0.525.19±0.2514.24±0.2914.35±0.250.22±0.02
467.48±1.9280.25±0.9434.05±1.2478.85±1.1590.98±0.7213.26±0.3711.68±0.525.19±0.2514.24±0.2914.47±0.270.25±0.02

Q3

Memory Growth and Retrieval Scalability. Our methodology maintains an aggregated experience memory indexed by context-intervention pairs, where each context is represented by a compact signature comprising the generator, capability, failure pattern, and severity. When a new request matches an existing context, its outcome updates the corresponding intervention's effect statistics rather than creating an additional searchable record. Given NN requests, CC distinct contexts, and AA available interventions, the number of searchable records is bounded by M≤min⁡(AN,AC)M\leq\min(AN,AC). For example, 10,000 requests spanning 20 distinct contexts and 3 interventions yield at most 60 searchable records. Thus, accumulated experience progressively refines the statistical priors without proportionally expanding the searchable memory.

Regarding Retrieval Efficiency. Our implementation scans the MM aggregated records, evaluates their fixed-length context signatures, and ranks the HH eligible matches, resulting in O(M+Hlog⁡H)O(M+H\log H) retrieval complexity, followed by O(AH)O(AH) posterior fusion. Notably, processing thousands of requests does not necessarily increase retrieval latency, as recurring contexts enrich existing statistical estimates without increasing MM. Retrieval cost therefore depends primarily on the number of distinct contexts rather than the total request volume.

Finally, we would like to thank you once again for the constructive suggestions and insightful comments on our work.

AuthorsOpenReview ↗

Sincerely Looking Forward to your Feedback

Dear Reviewer E9bn,

We would like to thank you once again for efforts and time on our work, for your insightful comments and for your constructive suggestions. As the discussion period draws to a close, we are reaching out to solicit your thoughts on the rebuttal responses. We have gone through your points one-by-one and tried to address them carefully. We eagerly anticipate your feedback and hope our efforts align with the needs of the community and the rigour of the conference.

Reviewer_wNPFOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes T2I-SEH, a self-evolving harness for text-to-image generation that improves the inference-time decisions of frozen generative models. Its central component, Effect-Conditioned Harness Learning (ECHL), estimates the effectiveness of different generation interventions using their paired improvements relative to the original output. ECHL retrieves relevant historical observations based on contextual similarity and empirical support, and combines them with Gaussian priors through precision-weighted updates. It then uses cost-adjusted utility bounds and Monte Carlo estimates of the value of information (VOI) to decide whether to retain the original image, commit to an intervention, or perform additional diagnostic generations. The resulting observations are stored in persistent memory to guide future decisions. Experiments are conducted on five text-to-image generators, including diffusion-based and autoregressive models, using a controlled compositional generation dataset, GenEval 2, CoReBench, and DPG-Bench. The results show improvements in generation quality over the original generator with relatively low additional action costs.

Critical strengths
  1. The paper addresses an interesting problem: how to reuse historical experience to guide generation-time interventions without modifying the underlying generative model. Representing intervention effectiveness through paired improvements rather than absolute generation scores is well motivated, as it can partially account for differences in prompt difficulty. The integration of uncertainty estimates and intervention costs also provides a reasonable approach to avoiding unnecessary generation operations.
  2. The experiments cover five generators with different architectures and several evaluation settings. The controlled dataset allows the authors to examine different types of compositional failures. The results suggest that adaptive intervention selection can improve generation quality while reducing the number of additional generation operations.
Critical weaknesses
  1. This paper compares ECHL against only fixed intervention strategies, while ECHL involves a complicated uncertainty-aware decision mechanism. However, several simpler adaptive strategies could provide stronger baselines than fixed ones. For example, we can use the same historical observations but selects the intervention with the highest estimated cost-adjusted utility without additional diagnostic testing. This paper lacks the analysis of the contribution of each compoents.
  2. The improvements on external benchmarks are limited given the complicated designs and unjustified hand-chosen hyperparameters. On GenEval 2, even with the largest diagnostic budget B=4B=4, ECHL improves the official Soft-TIFA score by only 0.52–1.17 points over Raw across the five generators. Moreover, ECHL does not consistently outperform the strongest fixed-intervention baseline in generation quality. The paper also does not provide confidence intervals, making it difficult to determine whether these small gains are real. The Internal Success Rate improvement is significant though. Since ISR is evaluated using the same model (Qwen2.5-VL-3B) that guides ECHL's intervention selection, improvements in ISR may partly reflect optimization toward the internal evaluation signal rather than independent improvements in generation quality. The relatively limited gains on the external Soft-TIFA metric raise concerns about how well these improvements generalize beyond the internal model.
Questions for authors
  1. Could the authors provide ablation studies that separately examine the contributions of historical experience retrieval, uncertainty-weighted belief fusion, and VOI-based sequential diagnosis?
  2. Could the authors provide statistical uncertainty estimates for the benchmark results? The GenEval 2 improvements over Raw in Table 2 are relatively small for some generators, and the changes on CoReBench and DPG-Bench are often close to zero.
  3. In Appendix G, the same intervention is tested multiple times on the same prompt, and each observation is used to update the Gaussian posterior. However, Appendix C says only one candidate per prompt-intervention pair. Are these repeated tests based on independently generated samples or repeated evaluations of the same candidate?
  4. How sensitive is the method to the hand-designed similarity coefficients, retrieval threshold, support threshold, Gaussian prior, and VOI stopping threshold?
  5. Can the authors explain the significant discrepancy between ISR and external evaluation scores? Large ISR improvements may not be independent evidence of improved image quality.
11 comments

Dear Reviewer wNPF,

Thank you for your efforts and time on our work, for your insightful comments and for your constructive suggestions. We are more than delighted that you recognize the problem as "interesting", the proposed methodology as "well-motivated" and "reasonable", and "five generators with different architectures and several evaluation settings" as a key strength of our study. We fully agree with your constructive comments, which have helped us further improve our work. In the following, we provide detailed, point-by-point responses to each of them.

W1 and Q1 (Ablation Studies)

We further conduct ablation studies on the controlled dataset and the GenEval 2 to systematically assess the individual contributions of three key components: historical experience retrieval, uncertainty-weighted belief fusion, and VOI-based sequential diagnosis. Here, HR denotes historical experience retrieval, and UF denotes uncertainty-weighted fusion; moreover, "w/o UF" fixes the variance of retrieved records at 0.25, while "w/o VOI" selects diagnostic tests based on the ratio of posterior standard deviation to testing cost.

Table 1. Ablation Studies of Task Success Rate (TSR) and Constraint Satisfaction Rate (CSR) on the Controlled Dataset.

BTSR w/o HRTSR w/o UFTSR w/o VOITSR ECHLCSR w/o HRCSR w/o UFCSR w/o VOICSR ECHL
SDXL138.3340.5637.7841.9446.0847.3945.3949.17
242.5044.1741.1145.0050.4451.6948.6952.67
448.3350.5646.9451.1155.7557.1454.1957.92
SD3.5M162.7863.8961.1165.0075.5676.8973.6978.06
266.6768.8965.8369.7279.3180.6977.4781.83
471.9473.6170.8374.1783.1784.7281.3185.67
FLUX.2 Klein171.9473.0670.8374.4476.5077.5675.3378.33
277.5078.3376.1179.7280.2581.1778.9782.14
478.6181.1178.6181.3981.6482.8680.7583.67
Z-Image-Turbo183.8984.7283.8986.1185.3386.1484.5886.83
291.1191.3990.0092.5091.2292.0690.1492.72
491.9492.2290.2893.6192.4292.9291.5693.83
Janus-Pro-7B162.2263.8960.5665.2871.0072.8669.5373.67
262.2264.4461.1165.5671.2272.9769.8974.00
470.2870.2868.6172.2277.8679.3976.2280.61

Table 2. Ablation Studies of Internal Success Rate(ISR), GenEval2 Soft-TIFA on the GenEval 2 Benchmark.

BISR w/o HRISR w/o UFISR w/o VOIISR ECHLSoft-TIFA w/o HRSoft-TIFA w/o UFSoft-TIFA w/o VOISoft-TIFA ECHL
SDXL170.0072.2568.3873.007.838.017.678.07
285.8887.5083.3889.258.078.277.898.34
488.2590.3886.1391.258.258.398.058.41
SD3.5M188.3890.7586.8891.5019.6519.8719.4720.01
294.1395.5092.5096.7519.7620.0619.6020.10
496.0097.6395.0098.2519.9820.2119.7720.28
FLUX.2 Klein197.0097.7595.5098.3825.5325.8025.2925.96
298.2598.7596.6399.6325.7025.9525.4626.08
498.1399.0097.1399.7525.8526.0025.5926.08
Z-Image-Turbo197.3898.8896.0099.5031.4231.6631.1231.84
298.2598.7596.5099.6331.5231.7331.2231.85
499.0099.7597.5099.8831.5831.8331.3631.85
Janus-Pro-7B185.0086.5083.6387.8813.8814.1613.7614.33
286.8888.5084.2589.1314.2614.5113.9914.62
488.1390.3886.3891.0014.3914.6714.1614.78

W2 and Q2 (On Table 2 of paper, GenEval 2)

We would like to provide additional context to clarify their practical significance. Specifically, the achievable gains are inherently bounded by the capabilities of the frozen backbone models and the corrective capacity of the available intervention set. And the value of T2I-SEH, lies in unlocking these existing capabilities effectively through selective, budget-controlled intervention, without requiring generator retraining. It helps determine when additional computation is worthwhile, which intervention to apply, and when to retain the original image. Furthermore, to assess whether the observed improvements are consistent, we conduct additional experiments across a total of five independent seeds.

Table 2. Internal Success Rate (ISR), GenEval 2 Soft-TIFA and action cost across five independent random seeds, reporting the mean and standard deviation.

GeneratorBISR RawISR PSISR SPISR ERISR ECHLSoft-TIFA RawSoft-TIFA PSSoft-TIFA SPSoft-TIFA ERSoft-TIFA ECHLCost ECHL
SDXL161.15±0.8983.48±1.1151.08±1.41-75.95±3.847.97±0.396.93±0.375.35±0.39-8.38±0.360.15±0.08
261.15±0.8983.48±1.1151.08±1.4176.50±0.4689.03±0.687.97±0.396.93±0.375.35±0.398.15±0.508.59±0.270.28±0.01
461.15±0.8983.48±1.1151.08±1.4176.50±0.4691.95±0.737.97±0.396.93±0.375.35±0.398.15±0.508.73±0.370.34±0.02
SD3.5M179.98±0.6490.43±0.4972.35±2.54-92.10±1.6619.04±1.0616.31±0.4811.53±0.63-19.73±1.110.12±0.02
279.98±0.6490.43±0.4972.35±2.5491.25±0.8196.20±0.3419.04±1.0616.31±0.4811.53±0.6320.22±1.0419.92±1.020.16±0.01
479.98±0.6490.43±0.4972.35±2.5491.25±0.8197.80±0.4519.04±1.0616.31±0.4811.53±0.6320.22±1.0420.06±1.030.19±0.01
FLUX.2 Klein189.55±0.6597.35±0.3895.28±0.85-98.83±0.2925.48±0.8524.77±0.3023.51±0.54-26.78±0.900.09±0.01
289.55±0.6597.35±0.3895.28±0.8594.98±0.4399.63±0.1525.48±0.8524.77±0.3023.51±0.5426.14±0.8926.84±0.880.10±0.01
489.55±0.6597.35±0.3895.28±0.8594.98±0.4399.70±0.1425.48±0.8524.77±0.3023.51±0.5426.14±0.8926.84±0.880.10±0.01
Z-Image-Turbo194.83±0.3898.45±0.6097.23±0.45-99.05±0.3431.50±0.7928.74±0.8228.42±0.97-32.26±0.770.04±0.01
294.83±0.3898.45±0.6097.23±0.4597.13±0.3999.83±0.1431.50±0.7928.74±0.8228.42±0.9731.76±0.8432.27±0.790.05±0.00
494.83±0.3898.45±0.6097.23±0.4597.13±0.3999.90±0.0631.50±0.7928.74±0.8228.42±0.9731.76±0.8432.27±0.790.05±0.00
Janus-Pro-7B167.48±1.9280.25±0.9434.05±1.24-86.10±0.6413.26±0.3711.68±0.525.19±0.25-13.73±0.500.09±0.05
267.48±1.9280.25±0.9434.05±1.2478.85±1.1589.03±0.6913.26±0.3711.68±0.525.19±0.2514.24±0.2914.35±0.250.22±0.02
467.48±1.9280.25±0.9434.05±1.2478.85±1.1590.98±0.7213.26±0.3711.68±0.525.19±0.2514.24±0.2914.47±0.270.25±0.02

W2 (On Table 1 of paper, Controlled Dataset, Additional Evaluation)

In Table 1, we employ Qwen2.5-VL-3B as both the internal observer and the atomic evaluator to assess the Task Success Rate and Constraint Satisfaction Rate, primarily to examine the effectiveness of internal decision-making. We fully agree with your suggestion to incorporate independent evaluators for a more comprehensive and robust assessment. We additionally introduce six vision models of varying parameter scales developed by different organizations as independent evaluators: HyperCLOVAX-SEED-Vision-Instruct-3B, InternVL3.5-4B, PLaMo 2.1-8B-VL, GLM-4V-9B, Llama-3.2-11B-Vision and Qwen3.8-27B. Although the absolute scores vary somewhat under independent evaluation, the overall trend remains consistent, confirming that T2I-SEH selects more appropriate interventions while incurring lower action costs.

Table 3. Task Success Rate (TSR) and Constraint Satisfaction Rate (CSR) evaluated with HyperCLOVAX-SEED-Vision-Instruct-3B.

BTSR RawTSR PSTSR SPTSR ERTSR ECHLCSR RawCSR PSCSR SPCSR ERCSR ECHL
SDXL126.9428.6121.94-30.8339.8640.3933.28-43.86
226.9428.6121.9432.7834.4439.8640.3933.2846.5846.67
426.9428.6121.9432.7836.3939.8640.3933.2846.5848.67
SD3.5M164.1756.9457.50-65.0077.5071.3970.14-78.17
264.1756.9457.5070.5667.7877.5071.3970.1481.3679.97
464.1756.9457.5070.5670.8377.5071.3970.1481.3681.69
FLUX.2 Klein174.7271.6767.78-75.5677.4475.1473.56-78.14
274.7271.6767.7876.6777.2277.4475.1473.5679.2279.75
474.7271.6767.7876.6777.7877.4475.1473.5679.2280.67
Z-Image-Turbo173.8971.9470.56-75.2881.8979.8977.53-82.92
273.8971.9470.5678.0678.3381.8979.8977.5384.8185.83
473.8971.9470.5678.0680.0081.8979.8977.5384.8186.83
Janus-Pro-7B149.7239.175.83-52.2264.0649.5017.11-66.78
249.7239.175.8355.0052.2264.0649.5017.1169.6967.00
449.7239.175.8355.0055.5664.0649.5017.1169.6969.89

Table 4. Task Success Rate (TSR) and Constraint Satisfaction Rate (CSR) evaluated with InternVL3.5-4B.

BTSR RawTSR PSTSR SPTSR ERTSR ECHLCSR RawCSR PSCSR SPCSR ERCSR ECHL
SDXL124.7227.5020.28-28.8935.2236.6129.33-38.78
224.7227.5020.2831.9432.2235.2236.6129.3342.5842.25
424.7227.5020.2831.9433.6135.2236.6129.3342.5843.50
SD3.5M156.1151.9450.56-58.8970.9766.8663.78-73.67
256.1151.9450.5662.7862.2270.9766.8663.7877.7576.83
456.1151.9450.5662.7864.7270.9766.8663.7877.7578.14
FLUX.2 Klein163.6160.8360.28-64.7273.5671.9271.56-74.22
263.6160.8360.2867.2267.5073.5671.9271.5675.9776.19
463.6160.8360.2867.2268.3373.5671.9271.5675.9777.39
Z-Image-Turbo175.8379.4467.78-79.7281.8385.1977.00-85.69
275.8379.4467.7879.7280.2881.8385.1977.0084.5886.00
475.8379.4467.7879.7280.5681.8385.1977.0084.5886.72
Janus-Pro-7B148.0643.336.11-52.2261.8153.1916.97-66.39
248.0643.336.1152.5052.2261.8153.1916.9767.1166.67
448.0643.336.1152.5054.7261.8153.1916.9767.1168.92

Table 5. Task Success Rate (TSR) and Constraint Satisfaction Rate (CSR) evaluated with PLaMo 2.1-8B-VL.

BTSR RawTSR PSTSR SPTSR ERTSR ECHLCSR RawCSR PSCSR SPCSR ERCSR ECHL
SDXL136.9440.0032.78-40.8347.8351.1440.81-51.97
236.9440.0032.7842.7843.6147.8351.1440.8153.0853.47
436.9440.0032.7842.7845.2847.8351.1440.8153.0855.58
SD3.5M162.7863.3363.06-64.7275.6775.5674.42-77.08
262.7863.3363.0663.6166.3975.6775.5674.4277.2578.39
462.7863.3363.0663.6167.2275.6775.5674.4277.2579.08
FLUX.2 Klein161.3960.0061.94-62.5076.4474.9476.89-77.11
261.3960.0061.9463.0663.6176.4474.9476.8978.0878.19
461.3960.0061.9463.0665.5676.4474.9476.8978.0879.56
Z-Image-Turbo168.8965.8362.22-69.1781.7277.6175.81-81.78
268.8965.8362.2268.8969.4481.7277.6175.8182.1182.31
468.8965.8362.2268.8969.1781.7277.6175.8182.1182.22
Janus-Pro-7B155.8345.838.06-56.1167.6954.3925.36-68.08
255.8345.838.0657.2256.1167.6954.3925.3668.5868.08
455.8345.838.0657.2257.5067.6954.3925.3668.5869.47

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.