- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
T2I-OPSD transfers reflection-conditioned supervision into a prompt-only text-to-image model. Starting from a reflection-capable AlphaGRPO checkpoint, the model generates an image and textual reflection, then constructs noisy query states by forward resampling from the generated endpoint. The prompt-only student matches a same-parameter teacher conditioned on the image and reflection. Interpolation with a frozen prompt-only reference stabilizes the targets. The resulting model requires no reflection or second generation round at inference. Evaluation covers compositional benchmarks, preference metrics, and a user study.
Critical strengths
-
Meaningful improvements without additional inference-time refinement. The method addresses the useful problem of internalizing an existing correction capability. Relative to the same-backbone AlphaGRPO model, GenEval improves from 77.17 to 85.04 and DPG-Bench from 86.05 to 88.77. These gains support practical value beyond simply spending an additional generation round at deployment.
-
Complementary evaluation supports the overall training recipe. Improvements extend to TIIF-Bench and preference metrics, and the method receives 52% of choices in a randomized anonymous four-way user study. Reference-weight, teacher-update, and SFT comparisons also expose unstable alternatives. Together, this is stronger evidence than a single benchmark improvement or selected qualitative examples.
Critical weaknesses
-
The independent contribution of textual reflection is not isolated. The teacher receives both the generated image and reflection, and its image CFG scale differs from rollout generation. Without an image-only teacher and removed or shuffled reflections under matched CFG and reference mixing, the gains cannot be attributed specifically to meaningful reflection. This is consequential because reflection is the central distinction in the paper’s framing, whereas image-conditioned privileged distillation could explain part of the improvement.
-
The proposed co-drift mechanism is not conclusively established. Eq. 9 decomposes target refresh but does not establish instability. A declining teacher-versus-student win rate can reflect a stronger student rather than decreasing absolute correction quality. Similar online, EMA, and frozen-teacher curves help rule out a purely parameter-refresh explanation, but do not distinguish degraded reflections from reduced relative headroom. The training-stability observations are valuable; their proposed causal interpretation requires more direct evidence.
Questions for authors
- Under matched CFG and reference mixing, how does an image-only teacher compare with image-plus-reflection conditioning? What changes when reflections are removed or shuffled?
- On a fixed image/prompt evaluation set, does absolute teacher correction quality decline during training, or does only its relative advantage over the improving student decline?
- For the user study, please clarify participant consent and ethics-review approval or exemption status. The stated anonymity and randomization do not specify this status.
2 comments
W2 / Q2. Evidence for co-drift and absolute correction quality. Eq. 9 decomposes target refresh but does not prove instability. On a fixed prompt–image evaluation set, does teacher correction quality actually decline, or does its advantage diminish because the student improves? Similar online, EMA, and frozen-teacher curves do not by themselves resolve this distinction.
Thank you for pointing out the limits of our original co-drift analysis. To distinguish changes in teacher correction quality from a loss of corrective benefit as the student improves, we evaluate different teacher checkpoints on two fixed sets of prompt–image pairs generated by the initial and step-200 students. Within each set, all teacher checkpoints use the same input images, correction settings, and correction seeds. The table below summarizes the mean ImageReward gains over the original inputs, while Sec. 4.5 and Fig. 7(b) provide a more complete visualization and analysis, including absolute corrected-image scores and 95% confidence intervals for the gains. We have also revised the co-drift discussion in light of these results.
Table: Mean ImageReward gain over the fixed student inputs.
| Fixed student inputs / Teacher step | 0 | 20 | 40 | 60 | 80 | 100 | 120 | 140 | 160 | 180 | 200 |
|---|---|---|---|---|---|---|---|---|---|---|---|
| Student 0 | +0.1372 | +0.1637 | +0.1222 | +0.1088 | +0.1406 | +0.2043 | +0.1923 | +0.2072 | +0.1053 | +0.1227 | +0.1448 |
| Student 200 | −0.1323 | −0.1858 | −0.2328 | −0.2292 | −0.1847 | −0.1473 | −0.1257 | −0.0851 | −0.1356 | −0.1417 | −0.1929 |
Columns indicate teacher checkpoint steps. Student 0 and Student 200 denote fixed input images generated by the initial and step-200 students, with mean input ImageReward scores of approximately 1.078 and 1.523, respectively. Gains are paired differences between corrected and original input-image scores.
On either fixed set, the corrected-image score does not decline consistently across teacher checkpoints. However, the correction benefit differs between the two input sets. All 11 teacher checkpoints yield positive mean gains on initial student inputs, while their mean gains on step-200 student inputs are consistently negative. Thus, the teacher remains useful for the initial student's outputs while, on average, degrading the stronger student's outputs according to ImageReward. The fixed-input evaluation therefore shows that the teacher’s corrections remain beneficial for the initial student but reduce ImageReward on the stronger student’s outputs, providing direct evidence that corrective supervision can become detrimental as the student improves.
Eq. 9 characterizes the cross-step coupling between the student and its supervision: student updates change the generated images and teacher-conditioning context, which in turn change the targets used for subsequent student updates. Our fixed-input evaluation shows that teacher corrections can become detrimental to the stronger student. Under this coupling, continued distillation from harmful corrective targets can degrade the student, whose altered outputs then affect subsequent supervision and potentially reinforce the degradation. This is the co-drift risk we aim to explain, and the collapse under weak reference anchoring in Fig. 6(a) is consistent with this interpretation. We have clarified this connection in the introduction, method discussion, and conclusion.
In the revised manuscript:
- Sec. 3.3, pp. 5–6: Revised the co-drift discussion following the target-refresh decomposition.
- Sec. 4.5 and Fig. 7(b), p. 10: Added fixed-input absolute correction scores and paired gains.
- Sec. 4.4 and Fig. 6(a), p. 9: Reported the reference-anchoring training dynamics.
- Appendix E.2, pp. 18–19: Clarified the relative nature of teacher win rates and linked to the fixed-input analysis.
- Introduction, p. 2, and Conclusion, p. 10: Reframed the claim as analysis of co-drift risk.
Q3. Consent and ethics-review status. Please clarify whether participants provided informed consent and whether the user study received ethics approval or a formal exemption. Anonymization and randomization alone do not specify these conditions.
Thank you for requesting this clarification. All participants provided informed consent before the evaluation, and the study has received ethics approval. We have added this statement explicitly to the user-study paragraph in Sec. 4.2.
In the revised manuscript:
- Sec. 4.2, p. 8, lines 420–426: Explicitly stated participant informed consent and ethics approval.
Thank you for recognizing the practical gains without additional inference-time refinement and the breadth of the evaluation. We have added teacher-conditioning ablations and fixed-input correction analysis to address your concerns about textual reflection and co-drift. We have also corrected the image-CFG description and explicitly documented participant consent and ethics approval for the user study.
Thank you for this suggestion. We apologize for the confusion caused by the original CFG description. In the original manuscript, we mistakenly reported the code’s default image CFG value of 1.0 as a rollout setting. This setting has no effect on student rollouts because they use only the prompt with text CFG 4.0 and receive neither an input image nor textual reflection. The image CFG scale of 2.0 belongs to the additional SRR round used to evaluate AlphaGRPO+SRR. During distillation, teacher velocity targets are computed without image CFG, so the reported 1.0-to-2.0 difference does not represent stronger image guidance for the teacher. We have clarified these settings in Sec. 4.1 and Table 2.
We also agree that controlled ablations are needed to distinguish the contributions of textual reflection and visual context. To address this, we compare the full method with four variants: image only, shuffled textual reflection, textual reflection only, and shuffled images. All variants share the same initialization, reference-mixing coefficient, teacher update strategy, and other training settings. Only the teacher-conditioning inputs change.
The table below summarizes student ImageReward at selected training steps. Fig. 7(a) shows the full training curves.
Step 0 reports the shared initialization score before training. “—” denotes training collapse. ImageReward is used only for monitoring and does not enter the training objective.
The full method achieves the highest final student ImageReward. Keeping the image input but shuffling textual reflection reduces the step-200 score from 1.513 to 1.391, showing that sample-specific textual feedback contributes beyond image conditioning. Removing or shuffling the image also lowers performance, supporting the additional value of correctly paired visual context. Training remains stable with textual reflection alone, whereas the image-only variant collapses after an initial improvement and exhibits substantially larger gradient norms. This suggests that removing textual reflection shifts the teacher’s conditioning distribution and may impair its ability to provide stable corrective supervision. Together, these results support the complementary roles of visual context and textual reflection in providing effective teacher supervision.
In the revised manuscript: