- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper studies whether improvements obtained by a text-to-image agent harness can be transferred into the generator’s weights. Its Auto Skill Evolver (ASE) converts task episodes into insights and reusable prompt-rewriting skills. Diffusion On-Policy Context Distillation (D-OPCD) then trains a query-conditioned student to match an EMA teacher conditioned on both the query and the harness-produced prompt, using velocity matching on detached student-generated trajectory states.
Experiments use Z-Image-Turbo with benchmark-specific LoRA adaptation on GenEval, GenEval2, WISE, and R2I-Bench. D-OPCD improves the average direct-generation score from 60.52 to 65.09 and outperforms the tested SFT, Diffusion-DPO, and D-OPSD baselines. The paper also examines teacher conditioning, context filtering and diversity, OOD transfer, and a subsequent round of skill evolution after updating the generator and resetting the harness.
Critical strengths
-
A useful interface between agent experience and generator learning, with consistent direct-generation improvements. Using the original query and the harness-selected prompt avoids requiring the full agent at deployment. Table 2 reports improvements over the base generator on all four benchmarks, with an average gain of 4.57 points and a 1.71-point advantage over SFT. These results support the practical possibility of transferring some prompt-mediated harness benefit into generator weights.
-
The experimental protocol separates several important sources of evidence. The paper distinguishes training, selection, and final evaluation tasks; freezes the selected harness during replay; and compares training methods using the same retained task records. Table 3 tests matched versus shuffled teacher context, while Table 5 evaluates separately held-out OOD categories. The explicit discussion of filtering confounds and the limited number of co-evolution rounds also helps delimit what the experiments establish.
Critical weaknesses
-
The comparative advantage is not yet sufficiently established, particularly against D-OPSD. Appendix D.3 describes D-OPSD as using a detached clean-output endpoint target from Qwen3-VL-4B-Instruct. The original method instead specifies velocity matching and explicitly replaces the VLM’s language-model weights with Qwen3-4B weights to mitigate conditioning-feature mismatch. The manuscript does not establish whether these implementation details were preserved or whether the endpoint formulation includes equivalent timestep weighting. This matters because D-OPSD is the closest algorithmic comparator. See Jiang et al., Eq. 7 and Appendix C.
-
Moreover, Appendix D.4 specifies one training seed, and in-distribution evaluation uses one image per task without uncertainty estimates. Some advantages are small: the WISE difference between D-OPCD and D-OPSD corresponds to four additional successes among 350 tasks. Matching 2,000 optimizer steps also does not establish matched computation across objectives involving different numbers of model evaluations; effective batch size and actual training costs are not reported.
-
The experiments do not isolate internalization of accumulated skills from ordinary prompt enhancement, or establish that resetting old skills is warranted. All distillation records come from the evolved harness. There is no corresponding distillation control using prompts from the skill-free harness or an experience-free rewriter. Matched-versus-shuffled context establishes the importance of task relevance, but does not isolate the contribution of ASE’s accumulated experience. This distinction is consequential because the objective closely follows existing on-policy distillation constructions, making the experience-specific findings central to the contribution. Likewise, Eq. 9 compares the updated skill-free harness with the original skill-equipped harness, but does not test whether the original skills remain beneficial on the updated generator.
Questions for authors
-
Baseline clarification: Did the D-OPSD implementation use the original paper’s Qwen3 language-weight replacement within the multimodal encoder? Please specify its exact loss and timestep weighting, and explain how the “clean-output endpoint target” in Appendix D.3 relates to the original velocity-matching objective.
-
Attribution clarification: Are there existing comparisons that distill prompts from the skill-free harness or an experience-free rewriter under the same protocol? If not, what evidence distinguishes learning from accumulated ASE experience from learning ordinary task-specific prompt enhancement?
-
Reset clarification: Was the original skill library evaluated with the D-OPCD-updated generator? This comparison could determine whether the old skills are actually redundant and whether resetting them is preferable to retaining them.
5 comments
Q3: Equal optimizer updates do not ensure matched computational costs, and effective batch sizes and actual training costs are not reported.
Thank you for pointing this out. Using the same 2,000 optimizer updates does not ensure matched computational costs across methods. However, we do not believe that matching compute necessarily provides a fairer comparison. Different methods have different computational requirements and use the training data with different levels of efficiency. For example, given the same dataset, SFT may begin to overfit after consuming less compute. We therefore set a common, sufficiently large upper limit on optimizer updates and use a held-out validation set to select the best checkpoint for each method before evaluating it on the test set. This allows us to compare the performance each method achieves within the same training horizon.
Representative related work also adopts similar experimental controls. For example, Diffusion-DPO [1] uses the same schedule length when comparing SFT with DPO. D-OPSD [2] likewise specifies a fixed training horizon of 1,000 iterations for its LoRA setting and, in Appendix C, reports training costs of 5 GPU hours for SFT and 9 GPU hours for D-OPSD, comparing their performance despite the difference in compute. These examples show that using a common training schedule and comparing methods with different training costs are both established practices; matching actual compute is not a uniform prerequisite for all performance comparisons.
We also apologize for not clearly documenting the effective batch size and training costs in the appendix. For a configuration with an effective batch size of 4, using four NVIDIA A100 80GB GPUs for a complete 2,000-update training run, the cost per run is as follows:
| Method | Batch size | Training cost per run (GPU hours) |
|---|---|---|
| SFT | 4 | 4 |
| DPO | 4 | 16 |
| D-OPSD | 4 | 18 |
| D-OPCD | 4 | 17 |
We will add the training configurations, the basis for reporting training costs, and the validation-based checkpoint selection protocol to the experimental section and Appendix D.4 of the revised manuscript.
[1] Diffusion Model Alignment Using Direct Preference Optimization. https://arxiv.org/abs/2311.12908
[2] D-OPSD: On-Policy Self-Distillation for Continuously Tuning Step-Distilled Diffusion Models. https://arxiv.org/abs/2605.05204v3
Q4: Does D-OPCD internalize accumulated ASE experience beyond ordinary task-specific prompt enhancement?
Thank you for raising this attribution question. We added a skill-free scaffold distillation control and reused the second generator-update results reported in our response to Reviewer 1LwT, Q3. We will include these results in the revised manuscript. Our experiments indicate that D-OPCD internalizes the agent’s overall experience, benefiting from both ordinary scaffold refinement and the additional experience accumulated through ASE. We believe that the value of ASE lies not only in providing transferable experience beyond scaffold refinement but also in enabling renewed harness adaptation that supports further generator improvement.
| Experience used for internalization | GenEval | GenEval2 | WISE | R2I-Bench | Avg. |
|---|---|---|---|---|---|
| Skill-free scaffold | 78.16 | 81.42 | 51.14 | 46.27 | 64.25 |
| ASE: first round | 81.28 | 82.24 | 50.00 | 46.83 | 65.09 |
| ASE: second round | 89.74 | 83.74 | 51.43 | 47.26 | 68.04 |
Specifically, distilling skill-free scaffold experience achieves an average score of 64.25, while first-round ASE experience raises it to 65.09, with gains on three of four benchmarks. This supports an additional transferable benefit from accumulated ASE experience beyond ordinary scaffold refinement. After the second update described in our response to Reviewer 1LwT, Q3, the average reaches 68.04, improving by 2.95 points with gains on all four benchmarks. While this gain includes additional optimization, it supports the value of renewed harness adaptation in providing experience for further generator improvement.
Q5: Was the original skill library evaluated with the D-OPCD-updated generator to assess whether the old skills are redundant and resetting is preferable to retention?
Thank you for raising this question. We added an experiment evaluating the D-OPCD-updated generator with and without the original skill library.
| Benchmark | With old skills | Without old skills | Difference (pp) |
|---|---|---|---|
| GenEval | 91.24 | 85.57 | +5.67 |
| GenEval2 | 88.52 | 88.38 | +0.14 |
| WISE | 81.43 | 80.86 | +0.57 |
| R2I-Bench | 66.79 | 70.18 | −3.39 |
| Overall | 82.00 | 81.25 | +0.75 |
The results show that the old skills retain utility after generator internalization. However, removing them yields comparable overall performance, with a difference of 0.75 percentage points, and their effects depend on the dataset.
Importantly, our motivation for resetting the skill library was to enable continued evolution after generator internalization, rather than to improve immediate inference performance over retaining the old skills. The original library was the best snapshot selected by held-out performance during the first ASE stage, and further evolution during that stage resulted in lower performance. After internalizing the accumulated experience into the generator, resetting provides a fresh starting point for learning skills around the updated generator’s capabilities and remaining limitations. This design aims to move beyond the previously saturated evolutionary trajectory and sustain further improvement, as supported by our subsequent skill-evolution and second generator-update results. Reset is not required in practical deployment, where useful old skills can still be retained.
We agree that “now-redundant” is imprecise and could misleadingly suggest that the old skills have become useless. We will replace it with “previously saturated” to clarify that skill evolution had reached a plateau with the original generator, while the old skills can still provide value after the update.
We appreciate your detailed feedback and constructive questions.
Thank you for requesting these implementation details. We apologize for not explaining the implementation clearly enough in the original manuscript. Our implementation uses the same loading helper as the official D-OPSD code to load the generator’s Qwen3 text-encoder weights into Qwen3-VL’s language-model component, retaining its vision encoder and connector weights.
The loss also follows clean-endpoint matching in the official released training code. On shared student states,
Lendpoint(t)=(1−t)2∥vs−sg(vT)∥22
It therefore corresponds to timestep-weighted velocity matching, rather than the paper’s unweighted objective. At t=0,0.1,0.25,0.5, the weights are 1,0.81,0.5625,0.25, respectively. We will clarify the encoder replacement, exact loss, and timestep weighting in Appendix D.3, and document the official code version and adaptations to our task records.
To further assess the sensitivity of our results to this objective choice, we additionally trained D-OPCD using the same endpoint-loss formulation as D-OPSD. We include the reported D-OPSD results below for reference.
With endpoint matching, D-OPCD achieves an average score of 65.67 and still exceeds the reported D-OPSD scores on all four benchmarks. These results suggest that D-OPCD’s advantage is not specific to the unweighted velocity objective. We will also include this comparison in Appendix D.3.
Thank you for raising this point. We have expanded our experiments to examine variation in both training and image generation, and added uncertainty estimates and significance tests.
We retain the original results obtained with seed 42 and add training seeds 43 and 44. For each training seed, we select a checkpoint using the validation set and evaluate it with generation seeds 42, 43, and 44. Each checkpoint therefore produces three images per test task, giving nine training–generation combinations for each method on each benchmark.
We first average the three generation-seed scores for each trained model, then report the mean and sample standard deviation across training seeds. The full results are shown below, with standard deviations in parentheses. Avg gives equal weight to the four benchmarks.
D-OPCD has the highest mean score on all four benchmarks. Its overall score is 66.27, exceeding SFT, DPO, and D-OPSD by 5.04, 4.34, and 5.31 percentage points, respectively. It also has the highest overall score for each training seed after averaging over the three generation seeds. On WISE, where you raised this concern, the mean scores are 52.32 for D-OPCD and 47.46 for D-OPSD—a difference of 4.86 points, with a positive difference for every training seed.
We also assess the overall score differences using two-sided paired t-tests, pairing results by training seed, and report the corresponding 95% confidence intervals:
All three confidence intervals lie above zero, and the corresponding p-values are below 0.05. These results are consistent with the overall average advantage observed across training seeds.
We will add the full experimental protocol, multi-seed results, and statistical analysis to the experimental section and Appendix D.4 of the revised manuscript.