acceptodds
Under review as a conference paper at ICLR 2027

UniEvo-VL: Test-Time Training for Self-Evolving Unified Multimodal Models

Abstract

Modern multimodal models bring generation and understanding into a single unified system, which enables them to provide and learn from their own feedback. Motivated by this unified capacity, we introduce UniEvo-VL, a self-evolving framework for multimodal models to learn from this constructive self-correction feedback during test-time compute. Instead of relying on a separate, often larger, teacher, we leverage their self-critiques as privileged information and ask a single multimodal model to act as both teacher and student with different contexts. The student only sees the vanilla question, while the teacher conditions on the privileged critique. Then training minimizes the per-state divergence between their denoising diffusion distributions over the student's own sampling trajectories. Experiments demonstrate that UniEvo-VL improves the image generation capabilities of multimodal models, while maintaining their sensitivity to additional reflection information. Specifically, we build on top of the open-source Qwen-image-2512 and observe a significant performance gain from 0.747 to 0.808 on GenEval and from 32.97 to 35.53 on GenEval2 Soft-TIFA. Moreover, attempts with more powerful external critics (e.g., GPT5.6-Luna) show that multimodal models with strong judge capabilities can anticipate a higher self-evolving ceiling. Last but not least, mixed text-rendering outcomes show that our self-improvements may not be uniform across different tasks. We hope our study can shed light on the current hot recursive self-improvement research line to enhance the user experience when using multimodal models without external supervision or guidance.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Multimodel LLMs, Diffusion Distillation, Self-evolving

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_KkZWOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes UniEvo-VL, a critique-conditioned on-policy self-distillation framework for improving text-to-image generation. It distills corrective information from revised prompts into a generator while preserving the original prompt as the student’s conditioning. Experiments on GenEval, GenEval2, and OCR demonstrate improvements under certain configurations, with additional gains from stronger critics and inference-time reflection. However, the extent to which the proposed objective itself accounts for the observed improvements remains insufficiently established, particularly in the absence of stronger controlled baselines and comprehensive ablation studies.

Critical strengths

The paper introduces a promising approach that conditions an EMA teacher on critique-derived revised prompts and distills its predictions along the student’s own sampling trajectories. This provides dense intermediate supervision without requiring corrected-image targets, offering a meaningful application of on-policy self-distillation to feedback-driven image generation. Besides, the experiments cover compositional generation and text rendering, distinguishing direct generation from reflection-assisted generation. The paper also analyzes performance across prompt difficulty and acknowledges regressions and stylistic biases, providing a balanced assessment of the method’s potential and limitations.

Critical weaknesses

The primary novelty appears to lie in constructing privileged teacher conditioning from image-specific visual critiques, while the underlying on-policy prediction-matching mechanism follows existing distillation approaches. However, the paper does not provide sufficiently strong, controlled comparisons against generic prompt rewriting, image-independent prompt expansion or off-policy distillation. Consequently, the reported improvements cannot be attributed confidently to the proposed distillation mechanism rather than to prompt rewriting, data filtering, or other aspects of the training recipe.

Besides, the main results do not clearly establish variance across independent training runs or provide confidence intervals. Furthermore, only small part of acquisition attempts are accepted when post-revision verification is enabled, yet the reported learning curves primarily use optimizer updates rather than total end-to-end computational cost. Finally, the implementation uses fixed external critic and synthesis models and separately trained task-specific generator adapters. The evidence therefore supports task-specific feedback-guided adaptation more directly than general, continual self-improvement of a unified multi-modal model.

Questions for authors

A key concern is whether the reported improvements stem from the proposed on-policy distillation objective or from prompt rewriting and other training factors. A controlled comparison against generic prompt expansion, image-independent rewriting, and off-policy distillation under matched data and computational budgets would help clarify the contribution of each component. In addition, reporting means and standard deviations across independent training runs is important for establishing the robustness of the results.

The reliability of the evaluation also requires further clarification. The authors should describe how training and evaluation prompts are separated, including the detection of semantically similar and template-level near-duplicates. Furthermore, since Qwen-family models provide training feedback and are also used in GenEval2 evaluation, independent validation using evaluators from different model families or human assessments would help determine whether the reported improvements reflect genuine gains in generation quality rather than alignment with Qwen’s judging preferences.

2 comments
AuthorsOpenReview ↗

Response to Reviewer KkZW (1/2)

Thank you for the detailed review. Your questions separate three things we had not separated clearly enough: the source of the teacher's conditioning, the on-policy objective, and the rest of the training recipe. We answer each below and state plainly which controls we have and which we do not.

1. What the improvement can be attributed to

Scope of the claim. The on-policy prediction-matching objective follows prior work (OPD, DiffusionOPD, D-OPSD) and is not our contribution. Our contribution is the teacher's conditioning: a critique of the student's own draft, synthesized into a revised prompt. The controls below are organized around that distinction.

Control A — image-independent rewriting (same objective, different conditioning). We distilled from the official Qwen prompt enhancer, which expands the request without seeing any generated image. This covers both generic prompt expansion and image-independent rewriting. The objective, backbone (Qwen-Image-2.1), seed (43), evaluation prompts and scorer are identical; only the source of the teacher prompt differs. Scores are official GenEval2 Soft-TIFA ×100 under direct generation, with 800 images and 6,012 checks per checkpoint.

StepPrompt-only GMFeedback GMPrompt-only AMFeedback AM
013.4513.2759.3059.14
2012.3125.6258.3570.59
6018.5332.1463.7075.61
10027.2635.3169.9177.26
15032.3939.3772.5378.86

Two conclusions follow. First, rewriting alone can be distilled into better direct generation, so part of the gain does come from prompt rewriting, as you suspected. Second, image-specific feedback is higher at every post-training checkpoint and ends +6.98 GM and +6.33 AM ahead.

Limits of this control: it is a single seed, and it is matched in optimizer updates, not in compute. The prompt-only pipeline skips drafting, critique and regeneration, so it is much cheaper per update. We will add a compute-matched comparison to the revision.

Control B — off-policy training on corrected images (same data source, different objective). We trained on the regenerated corrected images with standard supervised flow matching, which does not use the student's own trajectories. Both runs use 100 development prompts, 750 checks and Gemini-2.5-Flash scoring.

StepSFT checks (%)OPSD checks (%)SFT strict (%)OPSD strict (%)SFT pref. /10OPSD pref. /10
080.6780.5328.0028.006.646.63
3078.0080.1327.0028.006.516.62
5072.5382.5311.0034.004.746.90

SFT falls below its own starting point while OPSD finishes above it. A second SFT run at the OPSD learning rate degraded faster (80.40% → 66.40% checks by step 20). This is preliminary evidence only: the two runs differ in learning rate (10⁻⁴ vs 3×10⁻⁴), the evaluation set is small, and this is not a full off-policy distillation baseline with a prompt-conditioned teacher. We will add that baseline under matched data and compute.

Data filtering. The submitted verified and unverified configurations differ in learning rate, acquisition seeds per prompt and run seed as well as in verification. We therefore cannot attribute their difference to filtering alone. We will rerun the unverified configuration under matched settings and report it.

2. Variance across independent training runs

We ran three seed-labelled training runs (42, 43, 44) on Qwen-Image-2.1. Values are mean ± sample SD, official GenEval2 Soft-TIFA ×100.

StepDirect GMDirect AM+1 reflection GM+1 reflection AM
013.30 ± 1.0458.61 ± 0.6036.89 ± 1.4374.82 ± 0.61
6031.85 ± 0.9275.14 ± 0.6045.15 ± 1.9280.66 ± 0.47
10035.71 ± 1.0877.16 ± 0.0847.56 ± 1.1781.95 ± 0.26
15040.37 ± 0.8979.46 ± 0.5250.27 ± 1.4383.06 ± 0.83

The direct-generation gain is positive in every seed. The mean within-seed improvement from step 0 to 150 is +27.07 ± 1.49 GM (95% CI [23.4, 30.8], t-interval, n = 3) and +20.86 ± 1.07 AM (95% CI [18.2, 23.5]).

What this does not cover: the Qwen-Image-2512 results in Tables 1 and 2 (GenEval, GenEval2, OCR) and the prompt-only control above are still single runs. We will add repeated runs for these in the revision.

3. End-to-end computational cost

GPU-hours are elapsed time multiplied by allocated GPUs and include rejected acquisitions.

AuthorsOpenReview ↗

Response to Reviewer KkZW (2/2)

UpdatesTotal GPU-hAcquisition GPU-hOPSD GPU-hAttemptedAcceptedAcceptance
2022.2621.151.1154016129.81%
4039.2637.042.2292832034.48%
6061.5458.203.341,50048032.00%
8081.3276.864.452,02464131.67%
100105.3699.805.552,66480030.03%

Cost is linear at about 1.05 GPU-hours per update. As you noted, about 30% of acquisition attempts pass verification; acquisition accounts for about 95% of total cost and the distillation updates for about 5%. Because cost is linear in updates, the learning curves keep their shape when re-plotted against GPU-hours, and we will add that axis to Figure 3.

For reference against inference-time reflection: direct generation takes 4.49 s per GenEval prompt and base + one reflection takes 51.60 s. The 105.36 GPU-hours are recovered after about 8,000 requests served without reflection, assuming single-GPU inference.

4. Scope: task-specific adaptation, not general continual self-improvement

We agree with your reading. In the submitted experiments the critic and synthesis models are fixed and separate from the generator, and a separate generator LoRA is trained for each task. We have no multi-task or continual-learning experiment. The evidence supports task-specific, feedback-guided adaptation of a generator, and we will describe it that way. Concretely, we will:

  • replace "a single multimodal model to act as both teacher and student" with wording that names the fixed vision-language critic, and reserve "self-distillation" for the student generator and its EMA teacher;
  • replace "without external supervision or guidance" with "without paired ground-truth images, human annotation, or task-specific reward functions";
  • write the critic as MϕundM^{\text{und}}_\phi with φ fixed, instead of sharing θ with the generator;
  • state in Section 3.1 that adapters are trained per task, and list unified and continual adaptation as future work.

5. Train/evaluation separation

GenEval2. We construct 50,000 adaptation prompts and a separate 800-prompt development set, and reserve the official 800-prompt benchmark for evaluation. A candidate is excluded if it matches an official or previously accepted example by normalized prompt text or by canonical annotated-scene identity (objects, counts, attributes, relations). The scene comparison canonicalizes entity order, symmetric relations and inverse-equivalent spatial relations, so paraphrases and reorderings of the same scene are removed, not only identical strings.

GenEval. There is zero exact or normalized prompt overlap between the training collection and the 553 evaluation prompts. GenEval is template-based ("a photo of X and Y"), so training and evaluation prompts share templates by construction; what is held out is the specific object, count, color and position combination.

OCR. We use the publicly released Flow-GRPO splits (19,653 training rows, 1,018 evaluation prompts) unchanged.

6. Evaluator independence

The concern is that Qwen-family models supply the training feedback and also score GenEval2. Four pieces of evidence do not depend on a Qwen judge:

  1. GenEval native score is computed by the official detector-based pipeline, not a vision-language model. It improves from 0.748 to 0.818 under direct generation.
  2. OCR native score is edit similarity on recognized text. It improves from 0.769 to 0.790.
  3. Gemini-2.5-Flash atomic accuracy, from a different model family, improves on both compositional benchmarks: 95.41% → 97.41% on GenEval and 81.47% → 82.75% on GenEval2 (Table 2).
  4. A non-Qwen critic also raises the Qwen-scored metric. With GPT-5.6-Luna feedback, GenEval2 Soft-TIFA improves from 32.97 to 35.53, so the gain on that metric does not require Qwen-generated feedback.

Two limits. The new results in Sections 1 and 2 above are reported only in GenEval2 Soft-TIFA; we will add Gemini scores for them. We have no human evaluation, and we will label HumanPref explicitly as an automated model judgment, not a human rating.

Closing

Thank you again for the time and care you put into this review. Your requests for controlled baselines and repeated runs have made the paper's claims more precise, and the revisions above will all appear in the updated manuscript. If these responses address your concerns, we would be grateful if you would consider raising your score. We are happy to answer any further questions during the discussion period.

Reviewer_LgveOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper aims to retain image-correction experience in model parameters, reducing reliance on inference-time reflection. UniEvo-VL rewrites prompts from feedback and distills an EMA teacher conditioned on those rewrites into a student that sees only the original prompt, using the student's denoising trajectories. Experiments train separate Qwen-Image-2512 LoRA adapters for three tasks. Gains are clear on GenEval, while GenEval2 and text-rendering results depend more on the configuration.

Critical strengths
  1. Using textual corrections to train the generator has practical value. The procedure turns feedback into denoising supervision without ground-truth target images or scalar rewards.

  2. The GenEval gains have clear experimental support. In Table 2, the verified model's direct-generation score rises from 0.748 to 0.818, showing that the improvement is not merely due to additional inference-time correction.

Critical weaknesses
  1. The added value of the main design choice is not established. As described in Related Work, feedback-conditioned self-distillation and OPD for diffusion models already have precedents. The main distinction here is constructing teacher prompts from critiques of generated images. However, the experiments do not separate image-specific corrections from ordinary prompt rewriting, leaving the contribution of inspecting and correcting the model's own outputs unclear.

  2. The claim of test-time training lacks direct experimental support. The paper describes continual updates across test requests, but Appendix C uses training prompt collections, and Appendix A.3 evaluates saved adapters. These results support improvements after training, but do not clearly demonstrate the benefit of updating parameters while processing test requests. Inference-time prompt rewriting and regeneration do not establish that benefit either.

Questions for authors
  1. With unverified Qwen feedback, the overall GenEval2 Gemini rating increases only from 6.14 to 6.23, while the native score falls from 32.86 to 32.37 (Tables 3 and 9). How do the authors explain these limited or negative gains? Compared with GenEval, the evidence for reliable improvement on more complex tasks is weaker.

  2. Tables 4 and 5 show that the verified and unverified experiments reported in Table 1 use different training settings. Specifically, on GenEval2, adding verification also changes the learning rate from 10−410^{-4} to 3×10−43\times10^{-4} and the number of acquisition seeds per prompt from eight to one. The performance difference therefore cannot be attributed to verification alone.

  3. Appendix E reports degradation in the SFT attempts. Could the authors provide quantitative results and training settings for these existing experiments? The advantage of OPSD over directly fine-tuning on corrected images remains difficult to assess.

  4. How many training examples were ultimately accepted for each task, and what was the end-to-end training time? Appendix A.4 reports acceptance rates of only 10%-21%, so optimizer-step counts omit substantial generation and critique costs. The practical cost of adaptation remains unclear.

4 comments
AuthorsOpenReview ↗

Response to Reviewer Lgve (1/2)

Thank you for the precise review. You identified two claims that the submitted evidence does not support: that verification explains the verified/unverified gap, and that the experiments demonstrate test-time training. We agree with both and say below exactly what we will change. We also add the rewriting control, the SFT numbers and the cost accounting you asked for.

W1. Image-specific correction versus ordinary prompt rewriting

We ran the control you describe. The teacher prompt comes from the official Qwen prompt enhancer, which rewrites the request without seeing the generated image or any critique. The student is trained with the same objective and evaluated by direct generation from the original prompt.

Setup: Qwen-Image-2.1, seed 43, official GenEval2 Soft-TIFA ×100, same evaluation prompts and generation seeds, 800 images and 6,012 checks per checkpoint. Note that this uses the newer backbone, not the Qwen-Image-2512 of the submission.

StepPrompt-only GMFeedback GMPrompt-only AMFeedback AM
013.4513.2759.3059.14
2012.3125.6258.3570.59
6018.5332.1463.7075.61
10027.2635.3169.9177.26
15032.3939.3772.5378.86

Both students start from the same untrained baseline. Ordinary rewriting does distill into better direct generation (13.45 → 32.39 GM), so part of the benefit is attributable to rewriting, as you suspected. Image-specific feedback is higher at every post-training checkpoint and ends +6.98 GM and +6.33 AM ahead.

Limits: this is one seed; the two pipelines are matched in optimizer updates, not compute; and they differ in the rewriting model as well as in image access. A cleaner control uses our own synthesis model with the image and critique withheld, and we will add it.

W2. Test-time training is not demonstrated — we will withdraw the claim

You are right. All experiments adapt on predefined training prompt collections and evaluate saved adapters on held-out benchmarks. They show that improvement is stored in the generator (GenEval direct generation 0.748 → 0.818 without reflection) and how it develops across checkpoints. They do not show a benefit from updating parameters while serving test requests, and inference-time reflection does not show it either.

We will remove the test-time framing:

  • Abstract. Old: "learn from this constructive self-correction feedback during test-time compute." New: "learn from this self-correction feedback through iterative on-policy adaptation."
  • Section 2.3. Old: "carries its updated parameters forward to subsequent requests, allowing corrective experience to accumulate across the request stream during test time." New: "carries its updated parameters forward to the next acquisition round, so later drafts and critiques come from the updated student."
  • Section 3.1. We will state that adaptation uses training prompt collections disjoint from the evaluation prompts, and that online adaptation over a live request stream is not evaluated.

What Algorithm 1 does support is the on-policy property: each round's drafts are generated by the current student, so the supervision tracks its current errors. We will describe the method in those terms.

Q1. Limited or negative GenEval2 results with unverified feedback

We read these numbers the same way you do. For the unverified Qwen configuration, the native score moves from 32.86 to 32.37 and the Gemini rating from 6.14 to 6.23. Neither is a measurable improvement.

One fact bears on why. Appendix A.4 reports that only 10–21% of revised prompts pass post-revision verification, meaning most revisions do not fix the draft when regenerated. The unverified configuration trains on all of them. This is consistent with a diluted training signal, but because of the confound in Q2 we cannot claim it as the cause.

On whether improvement on the harder benchmark is reliable:

  • Submitted backbone. The verified configuration improves matched direct generation from 31.92 to 35.07 (+3.15). This is a single run.
  • New backbone, three seeds. On Qwen-Image-2.1 the gain is positive in every seed.
StepDirect GMDirect AM+1 reflection GM+1 reflection AM
013.30 ± 1.0458.61 ± 0.6036.89 ± 1.4374.82 ± 0.61
6031.85 ± 0.9275.14 ± 0.6045.15 ± 1.9280.66 ± 0.47
10035.71 ± 1.0877.16 ± 0.0847.56 ± 1.1781.95 ± 0.26
15040.37 ± 0.8979.46 ± 0.5250.27 ± 1.4383.06 ± 0.83

Values are mean ± sample SD over seeds 42, 43 and 44. The mean within-seed gain from step 0 to 150 is +27.07 ± 1.49 GM and +20.86 ± 1.07 AM.

AuthorsOpenReview ↗

Response to Reviewer Lgve (2/2)

Q2. Verification is confounded with other settings

Correct. The unverified experiments were run before we introduced verification and the accompanying hyperparameter sweep. On GenEval2 the two configurations differ in learning rate, acquisition seeds per prompt (eight vs one) and run seed, in addition to verification. The difference in Table 1 cannot be attributed to verification alone.

We will:

  • rerun the unverified configuration with the verified configuration's learning rate, acquisition seeds and run seeds, and report the controlled comparison;

  • relabel the two rows of Table 1 as two configurations, and remove the statements in Section 3.2 that attribute their difference to verification.

Q3. Quantitative SFT results and settings

Below is every available direct-generation evaluation of the two continuations, each with its own step-0 baseline. Both use 100 development prompts, 750 checks, the same generation seeds and Gemini-2.5-Flash scoring.

StepSFT checks (%)OPSD checks (%)SFT strict (%)OPSD strict (%)SFT pref. /10OPSD pref. /10
080.6780.5328.0028.006.646.63
3078.0080.1327.0028.006.516.62
5072.5382.5311.0034.004.746.90

SFT falls below its own starting point; OPSD finishes above it.

The learning rates in this table differ: 10⁻⁴ for SFT and 3×10⁻⁴ for OPSD. A separately initialized SFT run at 3×10⁻⁴ degraded faster: 80.40% checks, 28% strict and 6.52 preference at step 0, against 66.40%, 3% and 3.48 at step 20. So matching the learning rate does not help SFT.

This evidence is limited: the evaluation set is small, the runs are short, and each setting is a single run. It shows that our SFT attempts were unstable in this regime, not that SFT cannot work. We will report these numbers and settings in Appendix E with that qualification.

Q4. Accepted examples and end-to-end cost

Cost over training. GPU-hours are elapsed time multiplied by allocated GPUs and include rejected acquisitions.

UpdatesTotal GPU-hAcquisition GPU-hOPSD GPU-hAttemptedAcceptedAcceptance
2022.2621.151.1154016129.81%
6061.5458.203.341,50048032.00%
100105.3699.805.552,66480030.03%

Your point is confirmed: optimizer steps understate the cost. Acquisition (drafting, critique, synthesis, regeneration, verification) takes about 95% of the total and the distillation updates about 5%. Cost is linear at about 1.05 GPU-hours per update, with 8 accepted examples per update.

Per task.

We provide the missing per-task breakdown below. The training-cost table in our earlier response described GenEval; we now also report OCR and GenEval2.

Table 2. OCR: cumulative acquisition and training statistics.

\text{Step} & \text{Attempted} & \text{Accepted} & \text{Accept. (\\%)} & \text{Acq.} & \text{Train} & \text{Total} & \text{Avg.} \\\\ \hline 20 & 792 & 160 & 20.20 & 50.74 & 1.11 & 51.85 & 2.59 \\\\ 60 & 2{,}208 & 480 & 21.74 & 139.69 & 3.31 & 143.00 & 2.38 \\\\ 100 & 3{,}748 & 801 & 21.37 & 231.48 & 5.52 & 237.01 & 2.37 \end{array}$$ **Table 3. GenEval2: cumulative acquisition and training statistics.** $$\begin{array}{r|rrr|rrrr} \text{Step} & \text{Attempted} & \text{Accepted} & \text{Accept. (\\%)} & \text{Acq.} & \text{Train} & \text{Total} & \text{Avg.} \\\\ \hline 20 & 628 & 160 & 25.48 & 39.52 & 1.12 & 40.65 & 2.03 \\\\ 60 & 1{,}840 & 480 & 26.09 & 118.00 & 3.33 & 121.35 & 2.02 \\\\ 100 & 3{,}068 & 803 & 26.17 & 203.80 & 5.64 & 209.46 & 2.09 \end{array}$$ **Against inference-time reflection.** Direct generation takes 4.49 s per GenEval prompt; base + one reflection takes 51.60 s. Each request served without reflection saves 47.11 GPU-seconds, so 105.36 GPU-hours are recovered after about 8,000 requests, assuming single-GPU inference. On GenEval the adapted model nearly matches reflection quality (0.818 vs 0.826). On GenEval2 with Qwen-Image-2512 it does not (35.07 vs 42.52), so there the lower inference cost comes with part of the reflection gain given up. ### Closing Thank you again for the time and care you put into this review. Your points on the verification confound and the test-time framing have made the paper's claims more accurate, and the revisions above will all appear in the updated manuscript. If these responses address your concerns, we would be grateful if you would consider raising your score. We are happy to answer any further questions during the discussion period.

Thank you for the substantial additional effort. The prompt-only control addresses my concern and provides useful evidence that image-specific feedback substantially outperforms ordinary prompt rewriting. However, this control and the much stronger three-seed results use a new backbone, whereas the submitted backbone shows only modest or negative gains under several configurations. Together with the simultaneous changes in training settings, this raises a new concern about robustness across backbones and configurations. Withdrawing the test-time training claim makes the paper more accurate, but also narrows its positioning to iterative on-policy adaptation and weakens its novelty relative to existing work. Given this substantial reframing, the configuration sensitivity, and the controlled verification experiment that remains pending, I do not believe the paper is yet fully substantiated, and I maintain my original score.

Thank you for recognizing the value of the prompt-only control.

(1) We have now completed an additional matched verification ablation on Qwen-Image-2.1, holding the run seed, learning rate, acquisition seeds per prompt, optimization settings, and evaluation protocol fixed while removing post-revision verification and its acceptance filter. Without verification, direct-generation GenEval2 GM improves from 13.45 to 37.94 after 150 updates, while the verified variant reaches 39.37. The effectiveness of this verifier depends on how difficult it is to correct a mistake for a given model and dataset. For Qwen-Image-2512, the verifier provided more stable training at the cost of additional rollout compute. We observe a smaller benefit on Qwen-Image-2.1.

Selected checkpoints are compared below. Scores are native GenEval2 Soft-TIFA GM ×100\times 100, evaluated on 800 images and 6,012 checks with seed 43.

\text{Step} & \text{Without verification} & \text{With verification} \\\\ \hline 0 & 13.45 & 13.27 \\\\ 40 & 27.69 & \mathbf{29.94} \\\\ 80 & 33.21 & \mathbf{36.51} \\\\ 120 & 36.36 & \mathbf{36.91} \\\\ 150 & 37.94 & \mathbf{39.37} \end{array}$$ (2) Regarding configuration changes, the Qwen-Image-2.1 runs largely preserve the optimization recipe already used for verified Qwen-Image-2512: learning rate $3\times10^{-4}$, one acquisition seed per prompt, 20 sampling steps, $1024^2$ resolution, LoRA rank/alpha 16/16, EMA decay 0.999, and local batch/gradient accumulation 1/2. **The new within-backbone ablation holds the relevant optimization quantities fixed.** (3) The reduction from CFG 4 to CFG 1 follows Qwen-Image-2.1’s intended sampling regime. Its official Diffusers implementation states: “Qwen-Image 2.1 is meant to be sampled without guidance, hence the default of 1.0.” We therefore use **CFG 1 consistently for the unadapted baseline, feedback adaptation, and prompt-only control**. (4) We respectfully disagree that this clarification constitutes a substantial reframing. The original paper already centered on acquiring corrective feedback without supplied target images and internalizing it through on-policy self-distillation. **On-policy adaptation was the original training mechanism, not a new positioning introduced during rebuttal.** (5) We discussed test-time adaptation as a potential application because the method can obtain supervision from incoming prompts and its own generated drafts. **Correcting the scope of that application claim leaves the proposed method and its central contribution unchanged.** The additional controls directly test this same contribution.
Reviewer_EdNsOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces UniEvo-VL, a self-evolving framework for multimodal models to learn from this constructive self-correction feedback during test-time compute. Instead of relying on a separate teacher, UniEvo-VL leverages the self-critiques as privileged information and ask a Qwen-Image to act as both teacher and student with different contexts. The student only sees the original prompts, while the teacher conditions on the prompts with privileged critique. Then the student is trained based on On-Policy Self-Distillation. The idea of critique-derived privileged conditioning, making the model can be trained without paired supervision. However, there are strong conceptual and methodological overlap with previous methods like D-OPSD and Dreaming in Flow. And my main concern is the overclaim of without larger teacher, and without external supervision or guidance. The use of Qwen3-VL+Qwen-image creates somewhat a large teacher with guidance.

Critical strengths
  • Clear motivation. The paper addresses an important problem on how to leverage the benefits of inference-time reflection into the model's generation capabilities, rather than relying on repeated critique and refinement at inference time. This is a meaningful direction toward self-improving generative models.

  • Effective conversion of visual critique into supervision. UniEvo-VL provides a reasonable way to convert critiques into dense supervision through on-policy self-distillation. By matching teacher and student predictions along the student's own denoising trajectories, the method enables the generator to learn from corrective feedback without relying on scalar rewards or explicit image targets.

  • Self-Improvement without groundtruth Images. UniEvo-VL does not require paired groundtruth images or manually annotated correction targets. It leverages generated samples and model-produced critiques to construct training signals, offering a promising approach to reducing dependence on external supervision.

Critical weaknesses
  • Limited methodological novelty. The core technical modules of on-policy self-distillation, privileged conditioning, and teacher-student prediction matching along denoising trajectories, have been explored in prior works such as D-OPSD and Dreaming in Flow. The main novelty lies in constructing privileged conditions from self-generated critiques rather than introducing a fundamentally new learning mechanism. The paper would benefit from a clearer differentiation from these closely related methods.

  • Overclaim of single-model self-improvement. Although the paper emphasizes self-improvement within a single unified multimodal model, the actual implementation relies on a separate Qwen3-VL for critique and prompt synthesis. These components are not jointly optimized, and the feedback models remain fixed throughout training. This raises concerns about whether the experiments truly demonstrate self-improvement within a single model, as claimed (e.g. Instead of relying on a separate, often larger, teacher, we leverage their self-critiques as privileged information and ask a single multimodal model to act as both teacher and student with different contexts, without external supervision or guidance), or rather a multi-model feedback-driven training pipeline. The paper should clarify this distinction and better justify its characterization as single-model self-improvement.

  • Missing computational analysis. The proposed framework introduces substantial computational overhead through repeated image generation, critique, revision, verification, and on-policy distillation. The paper lacks a systematic analysis of training time, computational efficiency, and the computational comparison between the proposed method and direct reflection pass.

Questions for authors
  • Please differentiate UniEvo-VL from D-OPSD and Dreaming in Flow from the perspective of methodological novelty.
  • Please clarify the distinction between single-model self-improvement and teacher with extra model Qwen3-VL for guidance, and justify its characterization as single-model self-improvement.
  • Please give the computational analysis
4 comments
AuthorsOpenReview ↗

Response to Reviewer EdNs (1/2)

Thank you for the careful review. We agree with two of your central points: the distillation mechanism is not our contribution, and our "single-model" wording overstated the implemented setting. Below we (1) state the contribution precisely, (2) give the exact text we will change, and (3) provide the compute analysis.

W1 / Q1. Difference from D-OPSD and Dreaming in Flow

What we do not claim. On-policy self-distillation with an EMA teacher and transition matching along student trajectories follows prior work (OPD, DiffusionOPD, D-OPSD). We will state this explicitly in Sections 1 and 2.3.

What we claim. Our contribution is how the teacher's privileged condition is obtained when no reference image exists.

Source of the teacher's privileged informationNeeds a reference image per promptDepends on the current student's output
D-OPSDA provided reference image (content, appearance, style)YesNo
Dreaming in Flow (v1)Image- and repair-conditioned flow feedback, plus semantic replayNoYes
UniEvo-VLA text critique of the student's own draft, synthesized into a revised promptNoYes
  • D-OPSD fixes the privileged information per image–text pair. UniEvo-VL derives it from the errors of the current generator, so the supervision changes as the student improves.
  • Dreaming in Flow is the closest work and is discussed in Section 4. Its first version appeared on 8 September 2026, shortly before the submission deadline, so we regard it as concurrent. The difference is the explicit intermediate diagnosis: a critic names a specific failure against the evaluation criteria, and the teacher prompt is written for that draft. Dreaming's v2 uses a generic repair instruction.

New control: is image-specific feedback needed? We distilled from the official Qwen prompt enhancer, which rewrites the request without seeing any generated image. Both students are evaluated by direct generation from the original prompts (Qwen-Image-2.1, seed 43, official GenEval2 Soft-TIFA ×100, 800 images and 6,012 checks per checkpoint).

StepPrompt-only GMFeedback GMPrompt-only AMFeedback AM
013.4513.2759.3059.14
2012.3125.6258.3570.59
6018.5332.1463.7075.61
10027.2635.3169.9177.26
15032.3939.3772.5378.86

The feedback pipeline is higher at every post-training checkpoint, ending +6.98 GM and +6.33 AM ahead. Two caveats: this is a single seed, and it compares the two implemented pipelines rather than isolating image access alone. It is not a direct comparison with Dreaming in Flow.

W2 / Q2. "Single-model self-improvement" is an overclaim — we will correct it

You are right. In the submitted Qwen-Image-2512 experiments the generator's conditioning encoder is Qwen2.5-VL, while critique and prompt synthesis use a separate, fixed Qwen3-VL. Belonging to the same model family does not make them one jointly adapting model. We will make the following changes.

Abstract, sentence 1.

  • Old: "Instead of relying on a separate, often larger, teacher, we leverage their self-critiques as privileged information and ask a single multimodal model to act as both teacher and student with different contexts."
  • New: "Instead of relying on a separately trained, stronger generator as teacher, we use critiques of the generator's own outputs, produced by a fixed vision-language critic, as privileged information. The generator and its EMA copy act as student and teacher under different contexts."

Abstract, last sentence.

  • Old: "without external supervision or guidance"
  • New: "without paired ground-truth images, human annotation, or task-specific reward functions"

Notation. Eq. 1–2, Algorithm 1 and Figure 2 write the critic as MθundM^{\text{und}}_\theta, which shares θ with the generator. We will write it as MϕundM^{\text{und}}_\phi with φ fixed, and state that only the generator LoRA is updated.

Terminology. "Self-distillation" will refer only to the student generator and its EMA teacher. The GPT-5.6-Luna runs will be labelled as an external-critic variant.

On "Qwen3-VL + Qwen-Image forms a larger teacher." We agree the teacher side has access to more than the student. Three facts bound what that extra access is:

  1. The teacher's generator is the EMA of the student's own weights. No stronger or separately trained generator is involved.
  2. The critic contributes only text. It produces no image targets, no gradients and no scalar rewards.
  3. The gain is not limited to reproducing reflection. With one reflection pass given to both models, the trained generator still exceeds the base model: 0.848 vs 0.826 on GenEval, 46.23 vs 42.52 on GenEval2, and 0.803 vs 0.783 on OCR (Table 2).

We will describe the demonstrated setting as generator adaptation from model-generated corrections with a fixed critic.

AuthorsOpenReview ↗

Response to Reviewer EdNs (2/2)

New evidence closer to the unified setting. Qwen-Image-2.1 uses a Qwen3-VL conditioning encoder, the same model family and version as our critic. The critic remains frozen and is not jointly optimized, so this narrows the gap to the unified setting but does not close it. Results are mean ± sample SD over seeds 42, 43, and 44 (official GenEval2 Soft-TIFA ×100).

StepDirect GMDirect AM+1 reflection GM+1 reflection AM
013.30 ± 1.0458.61 ± 0.6036.89 ± 1.4374.82 ± 0.61
6031.85 ± 0.9275.14 ± 0.6045.15 ± 1.9280.66 ± 0.47
10035.71 ± 1.0877.16 ± 0.0847.56 ± 1.1781.95 ± 0.26
15040.37 ± 0.8979.46 ± 0.5250.27 ± 1.4383.06 ± 0.83

Direct generation at step 150 (40.37) exceeds the base model with one reflection pass (36.89).

W3 / Q3. Computational analysis

Training cost. GPU-hours are elapsed time multiplied by allocated GPUs, and include rejected acquisitions.

UpdatesTotal GPU-hAcquisition GPU-hOPSD GPU-hAttemptedAcceptedAcceptance
2022.2621.151.1154016129.81%
4039.2637.042.2292832034.48%
6061.5458.203.341,50048032.00%
8081.3276.864.452,02464131.67%
100105.3699.805.552,66480030.03%

Cost grows linearly at about 1.05 GPU-hours per update. Acquisition (draft, critique, synthesis, regeneration, verification) accounts for about 95% of it; the distillation updates take about 5%. Roughly 30% of acquisition attempts pass verification.

Comparison with a direct reflection pass. On the GenEval prompts, direct generation takes 4.49 s per prompt and base + one reflection takes 51.60 s, an 11.5× difference.

Break-even. Each request served by the adapted model instead of a reflection pass saves 47.11 GPU-seconds. The 105.36 GPU-hours of training are therefore recovered after about 8,000 requests, assuming single-GPU inference.

What this does and does not buy. On GenEval the adapted model nearly matches reflection quality at direct-generation cost (0.818 vs 0.826). On GenEval2 with Qwen-Image-2512 it does not (35.07 vs 42.52), so there the method trades part of the reflection gain for the lower inference cost. On Qwen-Image-2.1 the adapted model exceeds base + reflection (40.37 vs 36.89). Reflection also remains usable after training and adds further gains in every setting.

We will add this analysis, including the break-even calculation, to Appendix A.4.


Closing

Thank you again for the time and care you put into this review. Your comments on the single-model wording and on the missing compute analysis have made the paper more precise, and the revisions above will all appear in the updated manuscript. If these responses address your concerns, we would be grateful if you would consider raising your score. We are happy to answer any further questions during the discussion period.

Thanks for the authors' detailed response. The clarifications alleviate my concerns about overstated claims and insufficient computational cost. The paper now more supports a narrower conclusion: corrective prompts produced by a fixedb VLM critic can be distilled through an existing on-policy self-distillation mechanism to improve the generator’s performance.

However, using a VLM critic from the same model family and version does not establish that they constitute a single, jointly self-evolving multimodal model. My main remaining concern is the limited methodological novelty relative to existing approaches. While the rebuttal clarifies the contribution, it does not yet sufficiently resolve this concern, so I am maintaining my original score for now.

Since I'm not an expert in this area, I would also appreciate the other reviewers’ post-rebuttal assessments, particularly regarding novelty, and remain open to reconsidering my assessment in light of the discussion.

AuthorsOpenReview ↗

Thank for your reply

Thank you for the follow-up and for stating clearly what remains open.

  1. We agree with your summary of what the experiments establish. The generator improves while the critic remains fixed. Using a critic from the same model family and version does not make the system a single, jointly self-evolving model. We will make that distinction explicit in the revision.

  2. Our contribution is how we obtain the corrective prompt. The on-policy distillation mechanism follows existing work. What we would add to your summary is that the critic inspects the student's current draft against the original request and identifies what needs to change. This gives the teacher a correction specific to that draft, without requiring a supplied target image. The student learns to generate from the original request alone.

  3. The prompt-only control helps quantify what that distinction adds. With the same Qwen-Image-2.1 backbone, distillation objective, and evaluation protocol, distilling an image-independent rewrite reaches 32.39 GenEval2 GM after 150 updates on seed 43. Distilling image-specific corrections reaches 39.37 GM (+6.98 points) and scores higher at every evaluated post-training checkpoint. Prompt rewriting alone produces a substantial gain. The additional gain with feedback is the evidence behind our claim that it matters how the corrective prompt is obtained.

Whether this constitutes a sufficient contribution is a fair question. We appreciate the opportunity to clarify both the contribution and the evidence for it.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary of Additional Experiments and Revisions

Dear Area Chair and Reviewers,

Thank you for the careful reviews and constructive discussion. We summarize the additional evidence for UniEvo-VL’s central contribution: acquiring image-specific corrective conditioning without supplied target images and internalizing it into direct generation through on-policy self-distillation.

  • Generalization to Qwen-Image-2.1. Across three training seeds, direct-generation GenEval2 Soft-TIFA GM improves from 13.30±1.0413.30\pm1.04 to 40.37±0.89\mathbf{40.37\pm0.89} after 150 updates, with a positive gain in every seed. One additional reflection reaches 50.27±1.4350.27\pm1.43. These results demonstrate that corrective adaptation also benefits the stronger Qwen-Image-2.1 backbone. Values are mean ± sample SD; GM scores are multiplied by 100.

  • Prompt-only control. On Qwen-Image-2.1, seed 43, distilling image-specific feedback reaches 39.37 GM, compared with 32.39 for distilling ordinary prompt rewriting. Feedback scores higher at every evaluated post-training checkpoint.

  • Matched verification ablation. Holding the optimization settings, acquisition seeds per prompt, run seed, and evaluation protocol fixed, training without post-revision verification improves GM from 13.45 to 37.94. Verification reaches 39.37. Substantial improvement therefore persists without verification, with a quantified benefit from verification in this matched run.

  • Computational cost. We report task-specific acquisition counts and GPU-hours through 100 updates for GenEval, GenEval2, and OCR, including rejected attempts. On GenEval, direct inference averages 4.08 s, versus 50.55 s with reflection, including prompt rewriting and regeneration.

The revised manuscript includes the Qwen-Image-2.1 learning curves, checkpoint tables, configuration details, quantitative SFT results, and compute breakdowns.

We are grateful for the time and attention you have devoted to our submission. We hope the additional experiments and revisions clarify the contribution and address the remaining questions about its empirical support.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.