acceptodds
Under review as a conference paper at ICLR 2027

VeriPixel: Code-as-Verification for Self-Improving Pixel Prediction

Abstract

Generative foundation models attract increasing attention for pixel prediction, as the image-to-image formulation offers the potential to unify image generation and pixel-level understanding tasks. However, existing prompt-based approaches typically rely on fixed, handcrafted prompts for single-round inference. They struggle with the limited semantic understanding capability of generative models, leaving a substantial performance gap relative to tuning-based methods. To this end, we propose VeriPixel, a verification-driven agentic framework for self-improving pixel prediction, which uses verification feedback to progressively enrich generation guidance with semantic information missing from the generative model. First, to provide reliable feedback for self-improvement, we propose code-as-verification, which uses a vision-language model (VLM) to generate executable code, transforming the vanilla single-pass VLM-as-a-judge into an iterative perception-execution-observation process. Instead of image-level VLM judgment, it provides iterative, pixel-grounded verification process. Nevertheless, producing reliable feedback through code execution requires an effective verification strategy. Therefore, instead of a single improvement loop, we propose bootstrapped self-improvement that first uses verifiable feedback from annotated training data to optimize verification strategies offline. With these strategies as instructions, the verifier provides reliable feedback and pixel evidence to bootstrap online self-improvement. Experiments show that VeriPixel improves upon direct prediction by 18.5 cIoU, 11.7 mIoU, and 3.4 dB PSNR across three representative pixel tasks, demonstrating significant gains from self-improvement and achieving competitive performance against tuning-based methods.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Pixel Prediction, Segmentation, Image Restoration

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_LDzyOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes VeriPixel, a self-improving framework for pixel-level prediction that improves test-time generation prompts through verification feedback without updating model parameters. Its key idea is code-as-verification: instead of directly relying on a VLM to judge prediction quality, the verifier generates and executes image-processing code to construct proxy references, compute pixel-level measurements, and identify localized errors. The method first uses annotated training data to optimize verification strategies offline, and then progressively refines the prompt for each test sample based on verification feedback. VeriPixel is evaluated on referring segmentation, semantic segmentation, and image restoration, achieving improvements of 18.5 cIoU, 11.7 mIoU, and 3.4 dB PSNR, respectively, over direct GPT-Image-2 prediction.

Critical strengths
  1. The method operationalizes feedback without test-time ground truth using executable proxy construction and localized error attribution, and clearly distinguishes offline strategy optimization from online prompt revision. This provides a concrete approach to a difficult verification problem.
  2. The paper compares verifier variants and shows that removing offline optimization or code execution lowers held-out RefCOCOg judgment accuracy and destabilizes online refinement. This provides more mechanistic evidence than the final task scores alone.
Critical weaknesses
  1. Verifier reliability is only thoroughly validated on RefCOCOg. Cityscapes and FoundIR use different proxy construction and metrics, and most Cityscapes classes allow prediction-conditioned proxy initialization. Without corresponding held-out verifier-accuracy and proxy-fidelity analysis, the claim of reliable cross-task verification is not fully supported.
  2. The contribution of additional test-time computation is not fully disentangled. VeriPixel uses an average of 5.38 GPT-Image-2 generation calls per sample under a maximum 15-round refinement budget, whereas the direct baseline uses a single generation. The RefCOCOg ablations already compare against prompt refinement with a no-code VLM-as-a-judge verifier and provide evidence for the value of code-based feedback. However, comparisons with best-of-N sampling and diverse prompting under matched generation-call counts and comparable total inference budgets are still missing, particularly on Cityscapes and FoundIR.
Questions for authors
  1. Can the authors report held-out verifier accuracy and proxy-to-ground-truth agreement on Cityscapes and FoundIR, including an ablation without prediction-conditioned proxies?
  2. Building on the existing RefCOCOg no-code ablation, can the authors compare VeriPixel with best-of-N sampling, diverse prompting, and no-code VLM-based refinement under matched GPT-Image-2 call counts and comparable total inference budgets across all three tasks?
  3. Evaluation uncertainty is under-reported. Some external baselines use different evaluation protocols, while no confidence intervals or variability across stochastic generations are reported, making small performance differences difficult to interpret. Can the authors report exact evaluation sample counts and uncertainty across independent generation runs?
  4. The paper should discuss Test-time Prompt Refinement for Text-to-Image Models, Idea2Img, GenPilot, and GenArtist, and clarify how VeriPixel differs, especially in its focus on dense pixel prediction and executable pixel-level verification.

Convincing answers to these questions would make me consider raising my score.

5 comments
AuthorsOpenReview ↗

Response to Reviewer LDzy (1/3)

We thank the reviewer for the careful reading and the precise, constructive questions. We appreciate your recognition that code-as-verification provides a concrete way to obtain feedback without test-time ground truth, and that our verifier ablations offer mechanistic evidence beyond final task scores. We have conducted all the requested experiments and provide a point-by-point response below. We hope these results address your concerns and merit a positive reconsideration of the score.

W1 & Q1. Held-out verifier accuracy and proxy fidelity on all tasks, including the ablation without prediction-conditioned proxies.

We report held-out verifier accuracy and proxy fidelity on all three datasets, together with the requested ablation on prediction-conditioned proxies. The results show that verifier reliability is not specific to RefCOCOg, the same framework yields reliable verification across tasks. Besides, prediction-conditioned initialization on Cityscapes is a choice made for efficiency, not a requirement for accuracy.

1. Held-out accuracy and proxy fidelity on all three tasks. Following the protocol in Sec. 4.4, we evaluate the verifier on held-out predictions for each dataset. Proxy fidelity, measured on the test set, is the quality of the proxy measured against ground truth (mIoU for Cityscapes, cIoU for RefCOCOg, PSNR for FoundIR). On all three tasks, the strategies that the verifier proposes and validates in the offline loop (proxy construction, metric selection, and thresholds) clearly improve both verification accuracy and proxy fidelity over the variant without the offline loop. This shows that the framework generalizes across tasks with different proxies and metrics.

DatasetAbs. Acc. (Ours)Rel. Acc. (Ours)Proxy Fidelity (Ours)Abs. Acc. (w/o offline)Rel. Acc. (w/o offline)Proxy Fidelity (w/o offline)
Cityscapes77.379.555.358.263.647.9
RefCOCOg82.889.164.165.573.053.2
FoundIR79.680.120.759.557.419.1

2. Ablation without prediction-conditioned proxies. The prediction-conditioned proxies are used for efficiency. On Cityscapes, building every class's proxy from scratch shows comparable accuracy and proxy fidelity (table below). However, with 19 classes, building all proxies from scratch takes about 6× more time (26.9 s vs 4.6 s per sample). Therefore, we allow the offline loop to consider prediction conditioning for multi-class datasets. The verifier then decides per class, using the labelled offline data, and conditions only classes where the generator's prediction is at least as accurate as the code-initialized proxy. The prediction serves only as an initialization and is still validated against image evidence (Sec. 3.2).

SettingAbs. Held-out Acc.Rel. Held-out Acc.Proxy FidelityAvg. Time (s)
From scratch76.180.453.726.9
Prediction-conditioned77.379.555.34.6
AuthorsOpenReview ↗

Response to Reviewer LDzy (2/3)

W2 & Q2. Comparison with best-of-N under matched budgets.

At inference, no ground truth is available to identify the Best-of-N samples, so oracle best-of-N cannot be realized in practice. A practical best-of-N method must rely on a label-free signal to select or combine the samples. We therefore compare our code-as-verification with two such label-free alternatives, pixel-level majority voting and no-code VLM-as-a-judge, under the same number of generation calls. The results show that code-as-verification provides a substantially more reliable selection signal.

  • Setup. We fix the number of GPT-Image-2 generation calls N and compare the three methods on all three datasets. Pixel-level majority voting combines the N predictions pixel by pixel: the majority label at each pixel for segmentation, and the per-pixel mean for restoration. Note that with N = 2, no majority exists for segmentation, so we use the second-round prediction. All methods use the same set of diverse prompts, i.e., the prompts produced by our refinement framework, so they differ only in how they choose or combine the N predictions.

  • Results. With the same budget, selection by code-as-verification performs clearly best on all three tasks, and its advantage grows with N. Majority voting improves over a single generation but soon plateaus. On image restoration (FoundIR) it can even lower PSNR, because averaging pixels from different predictions introduces inconsistencies between neighbouring pixels. No-code VLM-as-a-judge gives less accurate feedback, so the result it selects can even become worse as N increases (e.g., 64.5 → 58.5 cIoU on RefCOCOg from N = 8 to N = 12). Visual comparisons are provided in best-of-n.png in the supplementary material.

    DatasetSelection methodN = 2N = 4N = 6N = 8N = 10N = 12
    code-as-verification53.556.462.762.463.864.1
    Cityscapes (mIoU)majority voting53.656.157.457.559.858.8
    VLM-as-a-judge53.355.254.053.152.852.0
    code-as-verification57.871.673.273.974.174.3
    RefCOCOg (cIoU)majority voting54.962.867.367.466.567.1
    VLM-as-a-judge54.660.364.164.561.758.5
    code-as-verification19.621.822.222.522.622.9
    FoundIR (PSNR)majority voting19.419.720.320.420.120.3
    VLM-as-a-judge19.720.721.120.720.820.1

Q3. Analysis of uncertainty.

The table below provides the number of evaluation samples for each dataset, and the mean ± standard deviation over three independent runs of the full online loop. The variation across runs is small (std ≤ 0.7). Each final prediction is selected from multiple candidates by the verifier, which reduces the variance of a single stochastic generation. We will add these statistics to Table 1 in the revision.

Cityscapes (mIoU)RefCOCOg (cIoU)FoundIR (PSNR)
Evaluation samples5002,5731,500
Mean ± std (3 runs)64.3 ± 0.774.4 ± 0.522.5 ± 0.5
AuthorsOpenReview ↗

Response to Reviewer LDzy (3/3)

Q4. Discussion of representative related works.

We will add the following discussion to the related work in the revised version:

Recent work combines generative and understanding models into self-improvement frameworks for image generation. Idea2Img [1] and Test-time Prompt Refinement [2] use a VLM to iteratively refine prompts. Agentic frameworks further decompose the tasks and verify intermediate results. GenPilot [3] designs a multi-agent system for structured error attribution, and GenArtist [4] supports object addition, removal, and replacement through task planning and tool composition. These methods focus on creative generation and mainly assess semantic alignment between the image and the text. VeriPixel instead applies generative models to dense pixel prediction, such as segmentation and image restoration, which requires spatially precise, pixel-level quality assessment. To this end, we propose code-as-verification, which turns single-pass VLM judgment into an iterative perception–execution–observation process, and grounds verification in code-computed metrics and localized pixel evidence, providing more reliable and actionable feedback.

References

[1] Yang Z, Wang J, Li L, et al. Idea2Img: Iterative self-refinement with GPT-4V for automatic image design and generation. In ECCV, 2024: 167–184.

[2] Khan M A H, Jain Y, Bhattacharyya S, et al. Test-time prompt refinement for text-to-image models. In ICCV, 2025: 6565–6575.

[3] Ye W, Liu Z, Gui Y, et al. GenPilot: A multi-agent system for test-time prompt optimization in image generation. In Findings of EMNLP, 2025: 929–958.

[4] Wang Z, Li A, Li Z, et al. GenArtist: Multimodal LLM as an agent for unified image generation and editing. In NeurIPS, 2024, 37: 128374–128395.

Reviewer_cNc5OpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

The paper proposes VeriPixel, a test-time framework for pixel prediction that keeps the generator frozen and improves its outputs by iteratively refining the prompt rather than fine-tuning the model. A prompt-revision agent rewrites sample-specific guidance based on feedback from a VLM verifier, which writes and executes code to build a proxy ground truth, computes metrics against it, and returns both an absolute and a pairwise judgment together with error attribution. The verification strategy is selected offline on a small set of annotated training images and then kept fixed during the online loop. Across referring segmentation, semantic segmentation and image restoration, the framework improves cIoU, mIoU and PSNR over direct GPT-Image-2 prediction and is competitive with tuning-based methods. Further experiments compare different agent and verifier models, ablate offline optimization and code execution, and report time and token costs.

Critical strengths
  1. The paper proposes a well-designed agent flow that it is insightful. Splitting the framework into an offline loop that tunes the verification strategy on annotated training data and an online loop that refines the generation prompt for each test sample is a reasonable design.
  2. The experiments are comprehensive. The main results cover three different pixel tasks and show large gains over direct GPT-Image-2 prediction. The ablations examine both the agent model and the verifier, including variants without offline optimization or code execution, as well as a ground-truth oracle. The paper also analyzes time and token usage for both the offline and online stages, which matters for a multi-round agentic method.
Critical weaknesses
  1. All three default components are closed-source API models. In addition, the verification strategies are proposed by the agent itself during the offline loop, and the outputs of the online loop are stochastic. As a result, it will likely be difficult for others to reproduce the exact numbers in Table 1, especially once these model versions are updated or retired. This does not undermine the main claims, but it limits how far the reported numbers can serve as a reference for future work.
  2. The transferability of the offline-calibrated verifier is not tested, and its reliability is evaluated only in-distribution. The verification strategy, including the proxy operators and numeric thresholds, is selected for each task on annotated training images from the same dataset that is later used for evaluation. The held-out samples used to measure verifier accuracy are also drawn from the training splits of these datasets, and the verifier analysis is reported only on RefCOCOg. A more convincing test would apply a strategy calibrated on one dataset to another without re-calibration.
Questions for authors

The offline loop selects the verification strategy and its thresholds by maximizing agreement with labels derived from ground truth on annotated training images of the target datasets. This stage is supervised calibration on target-domain data, and only the online loop improves without labels. How do you see the offline stage fitting the "self-improvement" framing?

4 comments
AuthorsOpenReview ↗

Response to Reviewer cNc5 (1/2)

We sincerely thank the reviewer for the positive assessment, and for finding the offline/online design insightful and the experiments comprehensive. Your concerns on reproducibility, cross-dataset transfer, and the self-improvement framing have helped us strengthen the paper, and we respond to each of them in detail below. We hope these responses further strengthen your positive assessment of our work.

W1. Reproducibility affected by model versions.

We provide results with open-source models to ensure reproducibility independent of generative model versions. In addition, our response to Reviewer LDzy's Q3 reports the standard deviation over 3 independent runs of the online loop, showing that the stochasticity of this process is very small.

  1. We ensure the reproducibility of our results and key conclusions in five aspects:
  • Transferring code-as-verification to the open-source image generation model Open-Vision-Banana and reporting results, ensuring that the generative model's reproducibility is unaffected by model versions (see our response to Reviewer QH2n, Q2).
  • Reporting results with open-source VLMs as verifiers (DeepSeek and Kimi), ensuring that the VLM's reproducibility is unaffected by model versions.
  • Reporting results with open-source VLM-driven agents for online prompt refinement (DeepSeek and Kimi), ensuring that prompt refinement's reproducibility is unaffected by model versions.
  • Providing details of the current offline search results in Table 4, such as prediction conditioning and commonly used operators, to ensure reproducibility.
  • Further, we will open-source the complete verification configuration and experiment code, to ensure the reproducibility of the full framework.
  1. According to the experimental results in our response to LDzy (Q3), due to the multiple predictions in the self-improvement process, the standard deviations over 3 independent online-loop runs are all below 1, demonstrating very low stochasticity.

W2. Transferability in cross-dataset setting.

To verify the transferability of the offline-calibrated verifier, we conduct cross-dataset experiments. We find that code is the key factor for cross-dataset transfer, especially in transfer scenarios with different semantic categories.

  • Setup: We run the offline loop on each of the three datasets used in the paper as the source, apply the result directly to a different target dataset, and measure held-out accuracy and online optimization performance. In the table, the number before "→" is the result of testing directly on the target dataset without the offline loop (the w/o offline baseline). The number after "→" is the result of first running offline optimization on each of the three source datasets and then transferring to the new dataset.
  • Observations:
  1. Comparing before and after "→", all three cross-dataset transfers improve consistently when code is used. In particular, for Cityscapes, the experience accumulated in the offline loop includes class-specific colors and thresholds (see Table 4), which are not valid for the new classes in VOC. Therefore, the improvement should come from the experience regarding code operators, metric selection, etc., which is transferable.

  2. Without code, the benefit of cross-dataset transfer is very limited and can even degrade VOC performance. Without code, the verifier stores too much class-specific experience during the offline loop and lacks operators that generalize across datasets, which in turn harms the verifier's reliability.

    SettingHeld-out Acc. AbsoluteHeld-out Acc. RelativePerformanceHeld-out Acc. Absolute (w/o code)Held-out Acc. Relative (w/o code)Performance (w/o code)
    Cityscapes → PASCAL VOC (20)60.1 → 78.667.5 → 82.282.3 → 87.836.4 → 27.144.9 → 42.580.4 → 77.9
    RefCOCOg → RefCOCO+62.9 → 79.368.0 → 87.562.5 → 69.947.2 → 50.856.7 → 56.257.3 → 58.6
    FoundIR → MiO [2]57.2 → 76.560.3 → 79.419.2 → 21.543.3 → 42.549.4 → 52.817.6 → 18.8
AuthorsOpenReview ↗

Response to Reviewer cNc5 (2/2)

Q1. Calibration on target-domain data for self-improvement.

Since self-improvement relies on verification signals that are reliable and aligned with the target task, most existing methods have to conduct calibration with target-domain data, and this target-domain feedback usually comes from existing training data, as illustrated by the following examples:

  • Self-Harness [2] directly provides feedback through held-in trajectories to drive harness improvement.
  • Meta-Harness [3] selects harnesses using a target-domain search set before evaluation on the test set.
  • SKILLOPT [4] uses a target-domain Dsel\mathcal{D}_{\text{sel}} to decide whether modified skills can be accepted.

Following this setting, our method utilizes target-domain signals through code, which substantially improves cross-dataset transferability.

References

[1] Zhu K, Gu J, You Z, et al. An intelligent agentic system for complex image restoration problems[C], ICLR, 2025: 57985-58013.

[2] Zhang H, Zhang S, Li K, et al. Self-Harness: Harnesses that improve themselves. arXiv preprint arXiv:2606.09498, 2026.

[3] Lee Y, Nair R S, Zhang Q, et al. Meta-harness: End-to-end optimization of model harnesses. In Proceedings of the Third Conference on Language Modeling, 2026.

[4] Yang Y, Gong Z, Huang W, et al. SKILLOPT: Executive strategy for self-evolving agent skills. arXiv preprint arXiv:2605.23904, 2026.

Thanks for the detailed responses. I think the most of the rebuttals in response to all the reviewers are convincing. So I kept my score.

AuthorsOpenReview ↗

Thank You for Your Positive Feedback

Thank you very much for the kind follow-up and for maintaining your positive score. We are greatly encouraged that you found our responses convincing, and we will incorporate all the new results and clarifications into the revised version.

Reviewer_QH2nOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

VeriPixel is a weight‑free agentic framework for improving prompt‑based pixel‑level prediction (segmentation, image restoration). To fix unreliable feedback from naive VLM‑as‑judge, it introduces code‑as‑verification: a VLM generates executable image‑processing code to build proxy ground‑truth, compute pixel‑wise metrics and localize errors via a perception‑execution‑observation loop.

An offline bootstrapping step optimizes verification strategies on labelled training data. At test time, the agent iteratively refines prompts using verification feedback while keeping the generative model frozen. It achieves large gains over direct generation and competes with tuning‑based methods on 2D tasks, yet cannot overcome the base model’s inherent failures on 3D geometry prediction (depth, surface normal).

Critical strengths
  1. No fine‑tuning needed, compatible with closed‑source generative models.

  2. Code‑driven verification delivers concrete pixel‑level error evidence, outperforming subjective VLM‑only judging.

  3. Offline strategy optimization stabilizes online self‑improvement; comprehensive ablations are provided.

Critical weaknesses
  1. Performance ceiling limited by the base generative model; ineffective for 3D geometric tasks.

  2. Substantial compute overhead from repeated VLM calls, code execution and image generation.

  3. Relies on artificially built proxy ground‑truth; faulty proxies produce misleading feedback.

  4. Needs task‑specific verification design; verification quality depends heavily on the VLM verifier.

Questions for authors
  1. Can code‑as‑verification be adapted for 3D geometry or novel pixel‑level tasks?

  2. Does VeriPixel’s improvement transfer to open‑source image‑generation backbones?

  3. Can we combine prompt refinement with light‑weight test‑time adaptation for better results?

4 comments
AuthorsOpenReview ↗

Response to Reviewer QH2n (1/3)

We sincerely thank the reviewer for the thoughtful review, and for recognizing that VeriPixel requires no fine-tuning and remains compatible with closed-source generators, code-driven verification delivers concrete pixel-level evidence beyond subjective VLM-only judging, and that offline strategy optimization stabilizes online self-improvement. Your concerns about the performance ceiling, computational cost, and proxy reliability are important, and we have addressed each of them below with new experiments and analysis. We hope our responses resolve these concerns.

W1 & Q1. Transferability to 3D geometric tasks.

  1. Actually, our proposed code-as-verification can be generalized to 3D geometric tasks. The limitation reported in the paper stems from GPT-Image-2's insufficient 3D capability (see Page 19 of the paper).

  2. To demonstrate this, we replace GPT-Image-2 with the open-source Open-Vision-Banana (OVB, zero-shot, 9B) [1], which has been trained with 3D capability, as the generative model. Adding our code-as-verification yields consistent improvements on both depth estimation and surface-normal estimation on NYU.

    MethodAligned depth ↑Normals mean error ↓
    OVB97.419.9
    OVB+code98.718.4

W2. Substantial compute overhead from repeated VLM calls, code execution and image generation.

  1. The overhead is crucial for reliable verification. The additional computation grounds verification in pixel-level evidence rather than relying solely on unreliable single-pass judgments. As shown in Table 1 and Figure 4 in paper, this additional computation yields substantial improvements in pixel prediction.

  2. Under the same number of generation rounds, our method substantially outperforms other verification methods, including pixel-level majority voting and VLM-as-a-Judge. The experiment results are shown below, where N denotes the number of generation calls. For majority voting at N = 2, we use the second-round prediction, since no majority exists between two masks.

    DatasetVerification methodN=2N=4N=6N=8N=10N=12
    Code-as-verification53.556.462.762.463.864.1
    CityscapesMajority voting53.656.157.457.559.858.8
    VLM-as-a-Judge53.355.254.053.152.852.0
    Code-as-verification57.871.673.273.974.174.3
    RefCOCOgMajority voting54.962.867.367.466.567.1
    VLM-as-a-Judge54.660.364.164.561.758.5
    Code-as-verification19.621.822.222.522.622.9
    FoundIRMajority voting19.419.720.320.420.120.3
    VLM-as-a-Judge19.720.721.120.720.820.1
  3. Overhead can be substantially reduced through practical engineering strategies, including:

  • Overhead of API cost: Both the VLM and the generative model can be replaced with open-source alternatives while maintaining comparable performance, as demonstrated in Figure 4 in paper, and our responses to Q2 respectively. This reduces cost on commercial APIs.
  • Overhead of VLM-calls: As shown in the above experiment results, replacing the VLM with pixel-level majority voting still improves pixel prediction through self-improvement (above table), alleviating the overhead of VLM calls.
AuthorsOpenReview ↗

Response to Reviewer QH2n (2/3)

W3. Relies on artificially built proxy ground-truth; faulty proxies produce misleading feedback.

Actually, our code-as-verification is robust to faulty proxy ground-truth. In code-as-verification, the proxy serves as a reference to compare against the prediction, and points the VLM to key regions that require careful analysis. This is why we compute multiple metrics rather than making decisions directly from the IoU between the prediction and the proxy. We discuss this in detail below, supported by experiments.

  1. Our method is highly robust to faulty proxy ground truth. We report the proxy GT quality, the initial GPT-Image-2 performance, and the final optimized GPT-Image-2 performance on three datasets in the table below. The proxy GT is clearly worse than the final GPT-Image-2 performance. In other words, better predictions can be found using a worse proxy, which quantitatively demonstrates the robustness to faulty proxies.

    Cityscapes (mIoU)RefCOCOg (cIoU)FoundIR (PSNR)
    Proxy GT55.364.120.7
    GPT-Image-2 (Step 1)52.955.719.5
    GPT-Image-2 (Final)64.674.222.9
  2. The proxy serves as a reference for comparison with the prediction, recommending key regions for the VLM to inspect, rather than directly determining prediction quality. As shown in Table 4 and Fig. 6 of the paper, code-as-verification typically extracts metrics such as precision, recall, and false positives to locate key regions. The VLM then crops these regions for closer inspection. This substantially improves robustness to faulty proxies.

  3. The verification process continually refines the initial proxy, making verification robust to faulty proxies. During verification, the VLM inspects the key regions and revises the proxy when it conflicts with image evidence. Therefore, errors in the initial proxy are progressively corrected rather than carried into the final judgment. To quantify this, we instruct the VLM to save the proxy at the end of verification and compare it with the initial one. As shown below, the final proxy is consistently more accurate on all three tasks, +2.5 mIoU on Cityscapes, +5.1 cIoU on RefCOCOg, and +0.4 dB PSNR on FoundIR.

    Cityscapes (mIoU)RefCOCOg (cIoU)FoundIR (PSNR)
    Proxy GT, Initial55.364.120.7
    Proxy GT, Final57.869.221.1
  4. Verification errors caused by faulty proxy GT usually lie near quality-decision boundaries, which substantially alleviates their impact. For example, in absolute quality assessment, predictions that are actually OK but misjudged as GOOD mostly have true IoU near the OK/GOOD boundary (see held_out_{dataset}.png in the supplementary material). As a result, predictions with erroneous GOOD judgments still perform well.

W4. Needs task-specific verification design; Verification quality depends heavily on the VLM verifier.

In fact, our method requires no manual per-task design, and thanks to the generalizability of code-as-verification, verification does not depend heavily on the VLM's capability.

  1. No manually designed task-specific framework. To achieve this, we propose the Offline Loop with Verification Refinement module, using a small amount of existing training data, the verifier proposes multiple task-specific verification schemes, and reliable schemes are selected according to the ground-truth annotations of the training data.
  2. Verification does not heavily depend on the VLM's capability. As shown in Fig. 4 (left), even when GPT-5.5 is replaced with DeepSeek-V4.1-Flash, the weakest of the six VLM verifiers, pixel prediction still improves steadily during online optimization. In contrast, even with the strongest VLM, removing code from verification yields almost no improvement in pixel prediction. This shows that the most critical factor in verification is whether code is used, rather than heavy reliance on the VLM.

Q2. Does VeriPixel's improvement transfer to open-source image-generation backbones?

VeriPixel's improvements can be transferred to open-source image models and improve pixel prediction.

  1. We replace GPT-Image-2 with an open-source image generation backbone, Open-Vision-Banana (OVB). The results on four datasets are shown below. With our code-as-verification, OVB improves consistently on all four datasets. Compared with GPT-Image-2, it achieves better semantic segmentation performance and extends to 3D geometry estimation.

  2. In principle, our verification method and self-improvement framework are complementary to any image generation model and can improve its pixel prediction performance.

    RefCOCOg ↑Cityscapes ↑NYU (Depth) ↑NYU (Normal) ↓
    OVB57.465.697.419.9
    OVB + code63.968.198.718.4
AuthorsOpenReview ↗

Response to Reviewer QH2n (3/3)

Q3. Can we combine prompt refinement with light-weight test-time adaptation for better results?

  1. Lightweight test-time adaptation is an optional way to reduce overhead. We adopt pixel-level majority voting as a lightweight test-time adaptation, which can replace code-as-verification and be combined with prompt refinement to obtain better results, as the experiment results provided in our response to W2. Compared with the first-round result, majority voting over multi-round results improves performance.
  2. However, lightweight test-time adaptation still lags considerably behind our code-as-verification. As shown below, code-as-verification has a clear advantage at every setting from round 4 onward, owing to its more accurate and reliable verification.

References

[1] Jadon A, Pariza V, Ruthardt J, et al. Open Vision Banana: A Fully Open Recipe for Generative Visual Perception. arXiv preprint arXiv:2611.15844, 2026.

AuthorsOpenReview ↗

Looking Forward to Your Feedback

Dear Reviewer QH2n, thank you again for your valuable comments. We have provided detailed responses with new experiments addressing each of your concerns, and we would greatly appreciate your feedback. We look forward to further discussion with you, and we hope our responses merit a reconsideration of your score.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary of Reviews and Responses

Dear Area Chair,

We thank you and all reviewers for the time and constructive feedback. Below we summarize the strengths the reviewers recognized, their main concerns, and how we addressed them.

1. Contributions recognized by the reviewers

  • A concrete solution to verification without test-time ground truth. Code-as-verification turns subjective VLM-as-a-judge into executable proxy construction, pixel-level measurement, and localized error attribution (QH2n, LDzy).
  • A well-designed framework. The split into an offline loop that calibrates the verification strategy and an online loop that refines prompts for each sample was described as "insightful" and "reasonable" (cNc5, LDzy). It requires no fine-tuning and is compatible with closed-source generators (QH2n).
  • Comprehensive experiments with mechanistic evidence. The evaluation covers three pixel tasks, with large gains over direct prediction (+18.5 cIoU, +11.7 mIoU, +3.4 dB PSNR). Reviewers noted the ablations over agents, verifiers, offline optimization, code execution, and a ground-truth oracle, as well as the cost analysis (cNc5, LDzy, QH2n).

2. Main concerns and our responses

  • Verifier reliability beyond RefCOCOg (LDzy, QH2n). We added held-out verifier accuracy and proxy fidelity on all three tasks. Offline optimization consistently improves both (e.g., absolute accuracy 77.3 / 82.8 / 79.6 vs. 58.2 / 65.5 / 59.5 without it). An ablation shows that prediction-conditioned proxies are used for efficiency (6× faster) and are not needed for accuracy. We also showed that verification is robust to faulty proxies: the proxy is refined during verification, and the final predictions exceed the proxy's own accuracy.
  • Gains from extra compute (LDzy, QH2n). Under the same number of generation calls, code-as-verification outperforms pixel-level majority voting and VLM-as-a-judge on all three tasks (e.g., 74.3 vs. 67.1 vs. 58.5 cIoU on RefCOCOg), and the gap widens with more calls.
  • Transferability and reproducibility (cNc5, QH2n). Strategies calibrated on one dataset transfer to another without re-calibration (Cityscapes → VOC, RefCOCOg → RefCOCO+, FoundIR → MiO) and improve all three. The framework also transfers to the open-source generator Open-Vision-Banana, improving it on four datasets, including 3D depth and surface-normal estimation. Three independent runs give a standard deviation below 1 on every task. We will release all code and verification configurations.

3. Reviewer feedback after the rebuttal

  • Reviewer cNc5 found "most of the rebuttals in response to all the reviewers convincing" and maintained their positive score.
  • Reviewer LDzy confirmed that "the main concerns have been solved".
  • For Reviewer QH2n, we provided new experiments for each concern (3D tasks, compute, proxy reliability, open-source backbones) but have not yet received a reply.

We will incorporate all new results and clarifications into the revised version. We sincerely appreciate your consideration.

Best regards, The Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.