- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
RISE-DMD optimizes a deterministic few-step diffusion generator using black-box scalar rewards. It trains an auxiliary Reward Score model on student-generated samples with a positive-negative Flow Matching objective adapted from DiffusionNFT and a penalty toward a fixed reference/teacher model. The student is then updated through DMD, using the Reward Score EMA to supply the target velocity field and a separate Fake Score model to estimate the current student's marginal velocity field. Fresh student samples drive the next round of target learning. The authors characterize the population optimum and its target densities at each noise level, and report improvements on SD3.5 Medium, Sana, DrawBench, and an OCR task.
Critical strengths
S1. The method addresses a clear limitation of reward-weighted DMD. When the student and reference fields match, weighting the original DMD gradient by reward still gives a zero update. Learning a different target field provides a way to incorporate black-box feedback while retaining four-step generation.
S2. RISE-DMD obtains the best values on all six SD3.5 Medium metrics in Table 1, including comparisons with DMDR and TDM-R1. Results on Sana and DrawBench provide evidence beyond the main backbone and prompt set, and the gains extend to metrics excluded from the training reward.
S3. The population analysis helps explain the training objective. With an ideal current-student center, the analysis characterizes how reward changes the target density at each noise level. Its KL-regularized interpretation also makes the roles of the current student and the fixed reference easy to understand.
Critical weaknesses
W1. Some important training details remain unclear. Appendix A.6 provides reward-processing and batch settings, but does not give the values of and , the EMA schedules, or several optimizer and LoRA settings. Appendix A.3 also mentions a "practical branch normalization"; it is unclear whether this means the reward normalization documented in Appendix A.5 or an additional operation. A complete configuration table and focused sensitivity ablations for , , and the EMA schedule would help readers assess the effect of these choices. Also, The validation and checkpoint-selection protocol could be more explicit. Appendix A.5 states that the normalization configuration was selected considering all six metrics, including those excluded from the training reward. It is unclear whether this selection used a separate validation set or the final evaluation prompts. The checkpoint-selection procedure is also unspecified. Clarifying the selection data and criteria would help readers interpret the reported evaluation results.
W2. Some gains over strong baselines are modest; for example, Aesthetic improves from 6.422 to 6.521 over TDM-R1 in Table 1. The table reports only point estimates. 95% confidence intervals, computed from the existing evaluation outputs or additional repeated experiments, would help quantify uncertainty in the mean differences over the evaluation prompts.
W3. Under the stated ideal-field assumption, Eq. (16) characterizes a valid target density at each noise level. However, these targets need not be the noisy marginals of one clean distribution. Consider the unregularized case with a fixed reward , let denote forward noising operator. For a clean tilt , the actual noisy marginal and the target in Eq. (16) are
\qquad q_t\propto p_t e^{(2/\beta)\mathbb E[r\mid x_t]}.$$ These expressions generally differ, so the per-time construction does not by itself establish a compatible clean-space target. If the targets are incompatible, one clean student cannot match them all exactly. Appendix A.9 explicitly assumes cross-time compatibility and limits its convergence result to that idealized setting. A brief discussion of how DMD fits the targets when compatibility does not hold, and how that fit relates to the intended clean target, would strengthen the theoretical interpretation.Questions for authors
Q1. Could you provide a complete configuration table, including the implemented loss normalization and optimizer step counts or update frequencies for each component, and critical sensitivity ablations for , , and the EMA schedule?
Q2. Could you report prompt-level 95% confidence intervals for the main differences against the strongest baselines, using the existing evaluation outputs?
Q3. Which prompts and criteria were used to select normalization settings and evaluation checkpoints? Were these prompts separate from the final evaluation sets?
Q4. How should the DMD update be interpreted when the per-time targets are not compatible with one clean distribution, and what can be said about the resulting clean distribution?
5 comments
Author Response (2/4)
Sensitivity. For the EMA schedule, Fig. 9 compares Type-1 and Type-2: Type-1 improves faster, while the slower Type-2 is more stable over long training. For λ_ref, β and the number of field supervision times, we ran one-factor-at-a-time ablations on Pick-a-Pic. Each run starts from the same DMD checkpoint, changes a single hyperparameter from the default (λ_ref = 1e-2, β = 1, two supervision times), and is evaluated at a fixed endpoint after 200 updates. Each sweep is run on one machine together with its own default run. LPIPS measures per-prompt diversity.
Reference weight λ_ref.
| λ_ref | PickScore | CLIPScore | HPSv2.1 | Aesthetic | ImageReward | UnifiedReward | LPIPS |
|---|---|---|---|---|---|---|---|
| 0 | 22.452 ± 0.003 | 0.2815 ± 0.0003 | 0.3484 ± 0.0002 | 6.646 ± 0.004 | 1.384 ± 0.004 | 0.7090 ± 0.0004 | 0.563 |
| 1e-4 | 22.544 ± 0.004 | 0.2838 ± 0.0001 | 0.3483 ± 0.0001 | 6.637 ± 0.001 | 1.439 ± 0.002 | 0.7130 ± 0.0009 | 0.538 |
| 1e-2 (default) | 22.672 ± 0.004 | 0.2906 ± 0.0003 | 0.3432 ± 0.0001 | 6.529 ± 0.004 | 1.401 ± 0.005 | 0.7097 ± 0.0013 | 0.594 |
| 0.1 | 21.837 ± 0.010 | 0.2839 ± 0.0005 | 0.3009 ± 0.0001 | 6.201 ± 0.002 | 1.044 ± 0.003 | 0.6850 ± 0.0010 | 0.654 |
λ_ref controls the trade-off between reward and diversity. Removing or weakening the reference term (λ_ref ≤ 1e-4) raises HPSv2.1 and Aesthetic, but lowers PickScore and CLIPScore and clearly reduces diversity (ΔLPIPS −0.031 and −0.056; both 95% CIs exclude zero), a sign of reward over-optimization. A too strong reference term (λ_ref = 0.1) slows reward learning and lowers all six metrics; its higher LPIPS reflects artifacts rather than useful diversity. The default gives the best PickScore and CLIPScore, with the highest diversity among the well-performing settings.
Policy coefficient β.
| β | PickScore | CLIPScore | HPSv2.1 | Aesthetic | ImageReward | UnifiedReward |
|---|---|---|---|---|---|---|
| 0.1 | 22.599 ± 0.001 | 0.2865 ± 0.0001 | 0.3363 ± 0.0001 | 6.531 ± 0.004 | 1.414 ± 0.003 | 0.7163 ± 0.0026 |
| 1 (default) | 22.423 ± 0.005 | 0.2882 ± 0.0004 | 0.3285 ± 0.0001 | 6.372 ± 0.005 | 1.334 ± 0.006 | 0.7094 ± 0.0012 |
| 10 | 22.607 ± 0.003 | 0.2855 ± 0.0003 | 0.3465 ± 0.0002 | 6.553 ± 0.007 | 1.430 ± 0.008 | 0.7010 ± 0.0024 |
RISE-DMD is not sensitive to β. Over two orders of magnitude, the metrics change by at most 0.18 in PickScore and Aesthetic, without a monotonic trend. β = 0.1 and β = 10 are even slightly better than the default on most metrics, so the main results were not obtained by tuning β.
Number of field supervision times.
| Times | PickScore | CLIPScore | HPSv2.1 | Aesthetic | ImageReward | UnifiedReward | LPIPS | Train GPU-h |
|---|---|---|---|---|---|---|---|---|
| 2 (default) | 22.672 ± 0.004 | 0.2906 ± 0.0003 | 0.3432 ± 0.0001 | 6.529 ± 0.004 | 1.401 ± 0.005 | 0.7097 ± 0.0013 | 0.594 | 47.2 |
| 4 | 22.743 ± 0.005 | 0.2911 ± 0.0004 | 0.3426 ± 0.0002 | 6.502 ± 0.007 | 1.432 ± 0.004 | 0.7182 ± 0.0018 | 0.566 | 68.0 |
| 8 | 22.520 ± 0.007 | 0.2832 ± 0.0005 | 0.3477 ± 0.0002 | 6.648 ± 0.001 | 1.445 ± 0.006 | 0.7107 ± 0.0017 | 0.569 | 109.6 |
More supervision times give modest and mixed changes at a substantially higher cost: four times improve PickScore, CLIPScore, ImageReward and UnifiedReward, and eight times improve HPSv2.1, Aesthetic and ImageReward, while training compute grows by 1.4× and 2.3×. Two supervision times are therefore a cost-effective default.
Author Response (3/4)
W2 & Q2: Confidence intervals
We thank the reviewer for this suggestion. We re-generated all main-table checkpoints with three generation seeds and computed paired prompt-level 95% confidence intervals: for each of the 2,048 prompts, the paired differences are averaged over seeds, and prompts are resampled with replacement (10,000 replicates).
| Metric | RISE-DMD − TDM-R1 | 95% CI |
|---|---|---|
| PickScore | +0.583 | [0.561, 0.606] |
| CLIPScore | +0.005 | [0.004, 0.006] |
| HPSv2.1 | +0.015 | [0.014, 0.015] |
| Aesthetic | +0.100 | [0.089, 0.111] |
| ImageReward | +0.076 | [0.064, 0.089] |
| UnifiedReward | +0.035 | [0.032, 0.038] |
The Aesthetic gain highlighted by the reviewer is modest but clearly nonzero. Against all five baselines, 29 of 30 intervals are entirely positive; the exception is CLIPScore against DMDR ([−0.0009, +0.0009]), which we report as a tie. The full three-seed main table is:
| Method | PickScore | CLIPScore | HPSv2.1 | Aesthetic | ImageReward | UnifiedReward |
|---|---|---|---|---|---|---|
| DMD | 21.930 ± 0.008 | 0.2832 ± 0.0005 | 0.3230 ± 0.0003 | 6.091 ± 0.003 | 1.305 ± 0.007 | 0.6901 ± 0.0017 |
| DMD2 | 21.970 ± 0.010 | 0.2854 ± 0.0001 | 0.3262 ± 0.0002 | 6.154 ± 0.002 | 1.315 ± 0.003 | 0.6918 ± 0.0012 |
| DMDR | 22.818 ± 0.002 | 0.2921 ± 0.0002 | 0.3516 ± 0.0002 | 6.362 ± 0.002 | 1.458 ± 0.006 | 0.7199 ± 0.0015 |
| RT-DM | 21.781 ± 0.006 | 0.2842 ± 0.0003 | 0.3175 ± 0.0003 | 5.986 ± 0.007 | 1.281 ± 0.002 | 0.6845 ± 0.0008 |
| TDM-R1 | 22.401 ± 0.003 | 0.2870 ± 0.0003 | 0.3383 ± 0.0002 | 6.422 ± 0.004 | 1.443 ± 0.003 | 0.6894 ± 0.0013 |
| RISE-DMD | 22.984 ± 0.011 | 0.2921 ± 0.0003 | 0.3532 ± 0.0002 | 6.522 ± 0.004 | 1.519 ± 0.004 | 0.7247 ± 0.0011 |
Author Response (1/4)
We sincerely thank the reviewer for the careful and constructive review, and in particular for the precise theoretical question on cross-time compatibility and the concrete suggestions on configuration, uncertainty and selection protocol. They helped us make both the analysis and the experimental protocol clearer and more rigorous. We address each point below. Unless stated otherwise, all numbers are means ± std over three generation seeds.
W1 & Q1: Configuration and sensitivity
Branch normalization. We apologize for the ambiguous wording: this is a separate operation from reward normalization. Reward normalization acts on advantages (group std, clip 1.0). Branch normalization is the adaptive loss weighting of DiffusionNFT (Sec. 3.3, "self-normalized x₀ regression"), applied separately to the positive and negative branches: L±=∥x0±−x0∥2/sg(mean∣x0±−x0∣), with x0±=xt−σv±.
Hyperparameters and compute (to be added to the appendix):