- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper proposes GenOPSD, a score-only variant of on-policy self-distillation for flow models. For each fresh ODE trajectory, it also completes a projected self-SDE branch (as in Self-OPD) and scores both decoded images with a black-box evaluator. Rewards are normalized within each prompt group into clipped signed preferences α. Each final latent is forward-noised at nine noise levels, and a detached target is built. The model is then fitted with one AdamW step of clean-prediction MSE, and the sampler is refreshed. Experiments on SD3.5-M (single- and multi-reward) and Z-Image-Turbo compare GenOPSD with FlowGRPO, DiffusionNFT, Self-OPD and the reward-gradient method DiffusionOPSD at matched trajectory budgets, with additional Qwen-Image-Bench evaluation and training-reward ablations.
Critical strengths
- Practical and competitive with score-only feedback. On single-reward SD3.5-M, GenOPSD stays within 2.9–5.6% of the reward-gradient method DiffusionOPSD on PickScore, CLIPScore and HPSv2.1 and matches it on ImageReward (1.35 vs. 1.33). At the same trajectory budget, it also outperforms DiffusionNFT on these metrics and on GenEval and OCR. Because many useful evaluators (GenEval, OCR, API judges) are non-differentiable, this matters in practice.
- Careful accounting and useful ablations. Trajectories, scored images, query forwards and optimizer steps are reported separately. The ablations show clearly that negative preferences and sampler refresh are both necessary.
Critical weaknesses
-
The novelty is overstated. GenOPSD takes a single step from the point where the target is built (θ_old = θ_i, Prop. A.2). At that point, the gradient of Eq. (10) equals the gradient of a signed, reward-weighted flow-matching loss Σ(ησ²α/2ω)‖y_θ − x‖². In other words, rewards do act as per-sample loss weights, and the "uniform positive MSE" is a reparameterization. Appendix A.3 builds the target from DiffusionNFT's components: the implicit positive and negative policies p± (Eq. 18), the optimality weight q = (1 + α/κ)/2 (Eq. 17), and ω, which corresponds to DiffusionNFT's self-normalized x₀-error weighting. The result is DiffusionNFT's gradient in its fully on-policy limit (η = 0) with one step per batch. What is new is the extra self-SDE branch sample and the nine-level query coverage, and the paper never tests these against DiffusionNFT directly. Advantage-weighted flow-matching methods such as AWM (Xue et al., 2025), which make the same equivalence explicit, are not discussed. The paper presents itself as a "generalization of OPSD," but by its own admission, the link is only "operational."
-
Some headline gains rest on weak references, and the evidence is thin. The Qwen-Image-Bench gains (+34.2% overall, +62.5% on Quality) are measured against the SD3.5-M base sampled without CFG. That model is weak: its GenEval macro score is 0.297, versus 0.667 with CFG. Against the base with CFG 4.5, Fig. 7c shows a GenEval gain of only 8.8%. Part of the improvement may therefore come from distilling guidance into the model rather than from reward optimization.
-
In addition, on Z-Image-Turbo, GenOPSD trails DiffusionOPSD by 11.7% on HPSv2.1 and 29.5% on ImageReward. Neither the abstract nor the text mentions this; the Z-Image-Turbo results are compared only against the base model. Every result is a single run with no seeds or confidence intervals, even though several margins are about one point (e.g., Z-Image-Turbo GenEval +1.8 and OCR +1.0). The ablations report relative changes in training reward after 20 updates, not held-out metrics.
- Xue. et al. Advantage Weighted Matching: Aligning RL with Pretraining in Diffusion Models. 2025
Questions for authors
-
At your one-step synchronized setting, does the GenOPSD gradient match that of DiffusionNFT with η = 0 (up to the weighting scale)? If so, please run a direct ablation of "DiffusionNFT + self-SDE branch sample + nine noise levels" against GenOPSD.
-
Please report Qwen-Image-Bench and GenEval results for the SD3.5-M base with CFG 4.5. Please also state which training CFG each matched baseline used.
-
Please report mean ± std over at least three seeds for the main SD3.5-M and Z-Image-Turbo results, and explain why GenOPSD trails DiffusionOPSD on Z-Image-Turbo.
-
GenOPSD scores twice as many images per trajectory as DiffusionNFT and adds SDE-suffix sampling. Please report wall-clock time or GPU-hours for all methods.