- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes VeriPixel, a self-improving framework for pixel-level prediction that improves test-time generation prompts through verification feedback without updating model parameters. Its key idea is code-as-verification: instead of directly relying on a VLM to judge prediction quality, the verifier generates and executes image-processing code to construct proxy references, compute pixel-level measurements, and identify localized errors. The method first uses annotated training data to optimize verification strategies offline, and then progressively refines the prompt for each test sample based on verification feedback. VeriPixel is evaluated on referring segmentation, semantic segmentation, and image restoration, achieving improvements of 18.5 cIoU, 11.7 mIoU, and 3.4 dB PSNR, respectively, over direct GPT-Image-2 prediction.
Critical strengths
- The method operationalizes feedback without test-time ground truth using executable proxy construction and localized error attribution, and clearly distinguishes offline strategy optimization from online prompt revision. This provides a concrete approach to a difficult verification problem.
- The paper compares verifier variants and shows that removing offline optimization or code execution lowers held-out RefCOCOg judgment accuracy and destabilizes online refinement. This provides more mechanistic evidence than the final task scores alone.
Critical weaknesses
- Verifier reliability is only thoroughly validated on RefCOCOg. Cityscapes and FoundIR use different proxy construction and metrics, and most Cityscapes classes allow prediction-conditioned proxy initialization. Without corresponding held-out verifier-accuracy and proxy-fidelity analysis, the claim of reliable cross-task verification is not fully supported.
- The contribution of additional test-time computation is not fully disentangled. VeriPixel uses an average of 5.38 GPT-Image-2 generation calls per sample under a maximum 15-round refinement budget, whereas the direct baseline uses a single generation. The RefCOCOg ablations already compare against prompt refinement with a no-code VLM-as-a-judge verifier and provide evidence for the value of code-based feedback. However, comparisons with best-of-N sampling and diverse prompting under matched generation-call counts and comparable total inference budgets are still missing, particularly on Cityscapes and FoundIR.
Questions for authors
- Can the authors report held-out verifier accuracy and proxy-to-ground-truth agreement on Cityscapes and FoundIR, including an ablation without prediction-conditioned proxies?
- Building on the existing RefCOCOg no-code ablation, can the authors compare VeriPixel with best-of-N sampling, diverse prompting, and no-code VLM-based refinement under matched GPT-Image-2 call counts and comparable total inference budgets across all three tasks?
- Evaluation uncertainty is under-reported. Some external baselines use different evaluation protocols, while no confidence intervals or variability across stochastic generations are reported, making small performance differences difficult to interpret. Can the authors report exact evaluation sample counts and uncertainty across independent generation runs?
- The paper should discuss Test-time Prompt Refinement for Text-to-Image Models, Idea2Img, GenPilot, and GenArtist, and clarify how VeriPixel differs, especially in its focus on dense pixel prediction and executable pixel-level verification.
Convincing answers to these questions would make me consider raising my score.
5 comments
Response to Reviewer LDzy (2/3)
W2 & Q2. Comparison with best-of-N under matched budgets.
At inference, no ground truth is available to identify the Best-of-N samples, so oracle best-of-N cannot be realized in practice. A practical best-of-N method must rely on a label-free signal to select or combine the samples. We therefore compare our code-as-verification with two such label-free alternatives, pixel-level majority voting and no-code VLM-as-a-judge, under the same number of generation calls. The results show that code-as-verification provides a substantially more reliable selection signal.
-
Setup. We fix the number of GPT-Image-2 generation calls N and compare the three methods on all three datasets. Pixel-level majority voting combines the N predictions pixel by pixel: the majority label at each pixel for segmentation, and the per-pixel mean for restoration. Note that with N = 2, no majority exists for segmentation, so we use the second-round prediction. All methods use the same set of diverse prompts, i.e., the prompts produced by our refinement framework, so they differ only in how they choose or combine the N predictions.
-
Results. With the same budget, selection by code-as-verification performs clearly best on all three tasks, and its advantage grows with N. Majority voting improves over a single generation but soon plateaus. On image restoration (FoundIR) it can even lower PSNR, because averaging pixels from different predictions introduces inconsistencies between neighbouring pixels. No-code VLM-as-a-judge gives less accurate feedback, so the result it selects can even become worse as N increases (e.g., 64.5 → 58.5 cIoU on RefCOCOg from N = 8 to N = 12). Visual comparisons are provided in
best-of-n.pngin the supplementary material.Dataset Selection method N = 2 N = 4 N = 6 N = 8 N = 10 N = 12 code-as-verification 53.5 56.4 62.7 62.4 63.8 64.1 Cityscapes (mIoU) majority voting 53.6 56.1 57.4 57.5 59.8 58.8 VLM-as-a-judge 53.3 55.2 54.0 53.1 52.8 52.0 code-as-verification 57.8 71.6 73.2 73.9 74.1 74.3 RefCOCOg (cIoU) majority voting 54.9 62.8 67.3 67.4 66.5 67.1 VLM-as-a-judge 54.6 60.3 64.1 64.5 61.7 58.5 code-as-verification 19.6 21.8 22.2 22.5 22.6 22.9 FoundIR (PSNR) majority voting 19.4 19.7 20.3 20.4 20.1 20.3 VLM-as-a-judge 19.7 20.7 21.1 20.7 20.8 20.1
Q3. Analysis of uncertainty.
The table below provides the number of evaluation samples for each dataset, and the mean ± standard deviation over three independent runs of the full online loop. The variation across runs is small (std ≤ 0.7). Each final prediction is selected from multiple candidates by the verifier, which reduces the variance of a single stochastic generation. We will add these statistics to Table 1 in the revision.
| Cityscapes (mIoU) | RefCOCOg (cIoU) | FoundIR (PSNR) | |
|---|---|---|---|
| Evaluation samples | 500 | 2,573 | 1,500 |
| Mean ± std (3 runs) | 64.3 ± 0.7 | 74.4 ± 0.5 | 22.5 ± 0.5 |
Response to Reviewer LDzy (3/3)
Q4. Discussion of representative related works.
We will add the following discussion to the related work in the revised version:
Recent work combines generative and understanding models into self-improvement frameworks for image generation. Idea2Img [1] and Test-time Prompt Refinement [2] use a VLM to iteratively refine prompts. Agentic frameworks further decompose the tasks and verify intermediate results. GenPilot [3] designs a multi-agent system for structured error attribution, and GenArtist [4] supports object addition, removal, and replacement through task planning and tool composition. These methods focus on creative generation and mainly assess semantic alignment between the image and the text. VeriPixel instead applies generative models to dense pixel prediction, such as segmentation and image restoration, which requires spatially precise, pixel-level quality assessment. To this end, we propose code-as-verification, which turns single-pass VLM judgment into an iterative perception–execution–observation process, and grounds verification in code-computed metrics and localized pixel evidence, providing more reliable and actionable feedback.
References
[1] Yang Z, Wang J, Li L, et al. Idea2Img: Iterative self-refinement with GPT-4V for automatic image design and generation. In ECCV, 2024: 167–184.
[2] Khan M A H, Jain Y, Bhattacharyya S, et al. Test-time prompt refinement for text-to-image models. In ICCV, 2025: 6565–6575.
[3] Ye W, Liu Z, Gui Y, et al. GenPilot: A multi-agent system for test-time prompt optimization in image generation. In Findings of EMNLP, 2025: 929–958.
[4] Wang Z, Li A, Li Z, et al. GenArtist: Multimodal LLM as an agent for unified image generation and editing. In NeurIPS, 2024, 37: 128374–128395.
Response to Reviewer LDzy (1/3)
We thank the reviewer for the careful reading and the precise, constructive questions. We appreciate your recognition that code-as-verification provides a concrete way to obtain feedback without test-time ground truth, and that our verifier ablations offer mechanistic evidence beyond final task scores. We have conducted all the requested experiments and provide a point-by-point response below. We hope these results address your concerns and merit a positive reconsideration of the score.
We report held-out verifier accuracy and proxy fidelity on all three datasets, together with the requested ablation on prediction-conditioned proxies. The results show that verifier reliability is not specific to RefCOCOg, the same framework yields reliable verification across tasks. Besides, prediction-conditioned initialization on Cityscapes is a choice made for efficiency, not a requirement for accuracy.
1. Held-out accuracy and proxy fidelity on all three tasks. Following the protocol in Sec. 4.4, we evaluate the verifier on held-out predictions for each dataset. Proxy fidelity, measured on the test set, is the quality of the proxy measured against ground truth (mIoU for Cityscapes, cIoU for RefCOCOg, PSNR for FoundIR). On all three tasks, the strategies that the verifier proposes and validates in the offline loop (proxy construction, metric selection, and thresholds) clearly improve both verification accuracy and proxy fidelity over the variant without the offline loop. This shows that the framework generalizes across tasks with different proxies and metrics.
2. Ablation without prediction-conditioned proxies. The prediction-conditioned proxies are used for efficiency. On Cityscapes, building every class's proxy from scratch shows comparable accuracy and proxy fidelity (table below). However, with 19 classes, building all proxies from scratch takes about 6× more time (26.9 s vs 4.6 s per sample). Therefore, we allow the offline loop to consider prediction conditioning for multi-class datasets. The verifier then decides per class, using the labelled offline data, and conditions only classes where the generator's prediction is at least as accurate as the code-initialized proxy. The prediction serves only as an initialization and is still validated against image evidence (Sec. 3.2).