- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
The paper proposes Displacement Matching (DM), a post-training method for few-step text-to-image generators, covering both few-step distillation from a multi-step teacher and reward alignment with black-box rewards.
The core contribution is to take a forward-noising based RL method (DiffusionNFT), and naturally combine it with an already forward-noising based few-step distillation method (DMD).
Validation is performed on 4-step distillation, and several rewards (HPSv2.1, CLIPScore, PickScore, GenEval). Baselines consist of representative prior methods on (i) few-step distillation, (ii) reward alignment, (iii) joint few-step distillation and reward alignment. Methods are compared at a matched number of reward queries. The proposed method outperforms all baselines across all rewards.
Critical strengths
- Strong empirical results. Across two backbones, DM improves across all trained and held-out rewards over recent few-step RL and joint distillation+RL baselines. It is abundant in breadth, so if the comparison is fair, it would strongly support acceptance.
Critical weaknesses
- Clarity. The manuscript is very difficult to follow despite careful reading, to the extent that it is unreadable. To name a few,
- “Few-step generators lack a learning interface that is both faithful to deployment and practical” (L54–55) conveys nothing beyond the following sentence.
- I cannot comprehend what the authors are trying to communicate in the first three sentences of Section 2.2 (L124-126).
- “Clean law” / “clean (probability) law” (L136, L140) is never defined.
The poor ordering of content exacerbates this problem. To name a few,
- There is no mention of diffusion / flow-matching models in Section 1; the paper dives straight into their few-step distillations.
- The paper does not introduce its notation for diffusion / flow-matching before Section 2.1.
- Section 2.2 begins by referencing Section 2.4.
-
Unclear novelty. As a direct consequence of the absence of a 'Preliminaries' section, well-established results and the paper's contributions are not distinguished, and instead intertwined throughout Section 2. Notably, it does not cite previous work for well-established works (e.g., Section 2.1). Since this paper does not use the exact same notation as prior work it compares against, it is unclear which contributions are novel.
-
Unclear connection between theory and practial implementation. Several assumptions have not been qualified well enough. For instance, the root-mean-square gap of (P2*) is , which is not small for 4-step disillation. The GAN-distilled initialization does not guarantee the flow-map assumption (Section 3.3 L320), but performs as well as the flow-distilled version (Table 2).
Questions for authors
-
Novelty (affects assessment): Which parts of the formulation are novel contributions, and which parts can be deduced directly from prior work? To properly address this question, authors shall properly cite previous works across both reward alignment [1-2] and joint distillation & reward alignment [3-7] literature.
-
Fairness of evaluation (affects assessment): DiffusionNFT, AdvantageFlow, and Flow-GRPO were designed for multi-step models. Therefore, a more detailed description is granted for the evaluation protocol used to obtain results in Table 2 for these methods.
Results were compared at a matched number of reward queries, but a fair comparison grants a matched amount of wall-clock time. There is only an indirect measurement of wall-clock time on a subset of methods (Table 6). Does DM outperform existing methods when the total compute budget is controlled?
- Connection between theory and practical implementation (affects assessment): Given [Weakness 3], which empricial results of DM merit Theorem 1 and the flow-map assumption?
[1] https://openreview.net/forum?id=3mJ9eTU319
[2] https://openreview.net/forum?id=jqsxOdmX88
[3] TDM-R1
[4] GDMD
[5] MeanFlowNFT
[6] DMDR
[7] RTDMD
4 comments
Response to Reviewer toU3 (2/2)
Q2. Four-step adaptation and compute
FlowGRPO, DiffusionNFT, and AdvantageFlow are sequential baselines: starting from the same flow-distilled checkpoint as our sequential flow-base DM variant, we apply each method's own reward-training objective with four-step sampling for 1,000 iterations at 512 px. They retain their respective optimization formulations. Table 2 evaluates all methods with the deployed four-step CFG-free sampler. Training settings can differ from evaluation settings. For example, the timed FlowGRPO configuration below uses CFG. The joint baselines instead keep distillation active during reward optimization. We will clarify this distinction in a future revision.
Measured training costs. We measure iteration times at 512 px on eight dedicated H200 GPUs with 1,152 samples per iteration. The following are actual iteration times, without normalizing methods to the same number of update timesteps:
| Configuration | Seconds / iteration |
|---|---|
| DM, joint | 79.0 |
| DiffusionNFT, four-step configuration | 39.7 |
| FlowGRPO, four steps + CFG, | 57.1 |
| RTDMD, reward backpropagation on | 67.0 |
| TDM-R1 | 74.6 |
| DMDR | 87.1 |
| GDMD | 104.3 |
| MeanFlowNFT | 109.0 |
| AdvantageFlow | 209.3 |
The component measurements also give a similar cost comparison to Table 6. When normalized to four update timesteps, the corresponding measurements are 81.3 / 84.1 / 161.5 s for DM / RTDMD / TDM-R1. This normalized comparison is reported separately from actual elapsed training cost.
Joint training at common time budgets. We compare DM with the joint RL-and-distillation baselines on the same held-out validation prompt set, using the same reward metric, MNR = HPS + CLIP + PickScore/26. The time column gives wall-clock hours on eight H200 GPUs. This validation set differs from the evaluation set used for Table 2.
| Wall-clock time (h) | DM | RTDMD | TDM-R1 | MeanFlowNFT | GDMD | DMDR |
|---|---|---|---|---|---|---|
| 1 h | 1.466 | 1.451 | 1.443 | 1.389 | 1.411 | 1.423 |
| 4 h | 1.534 | 1.496 | 1.478 | 1.421 | 1.455 | 1.452 |
| 8 h | 1.569 | 1.510 | 1.482 | 1.438 | 1.468 | 1.466 |
| 12 h | 1.560 | 1.520 | 1.474 | 1.453 | 1.473 | 1.465 |
| 16 h | 1.570 | 1.529 | 1.484 | 1.465 | 1.498 | 1.475 |
DM achieves the highest validation MNR among these joint-training methods at every reported common time budget, including 1.534 at 4 hours and 1.570 at 16 hours. Thus its advantage in this comparison persists when methods receive the same estimated elapsed training time, despite DM's higher per-iteration cost than RTDMD and TDM-R1.
W3 / Q3. Connection to four-step sampling and the GAN initialization
Theorem 1 explains when fitting the proposed update changes the final output distribution in the intended direction. Its assumptions allow us to follow that change through the sampling steps. The remaining approximation error can still matter at four steps, so the theorem does not guarantee improvement for every practical model.
The posterior used by DM is defined from the generator's own outputs. We can estimate it whether the starting generator was trained by flow distillation or GAN distillation. The GAN initialization therefore does not invalidate the construction of the reward target. It also does not establish the theorem's flow-map assumption.
The experiments answer the practical questions separately. The controlled comparisons show that the posterior correction improves training. The local response measurements show positive reward changes on the actual four-step sampler. Table 2 shows that DM works from both initializations: the flow-base and GAN-base variants reach HPS 0.358 / 0.360, ImageReward 1.52 / 1.51, and DPG 85.77 / 85.53. Together, these results support the usefulness of the method beyond the idealized assumptions. We do not use the GAN results as evidence that those assumptions hold.
We also examine whether the marginal-distribution agreement required by (P1) holds approximately in this setting. We compare visited sampler states with independently re-noised outputs for both the frozen anchor and the trained student. Using 16,384 prompts per model with separate training and test prompts, a small CNN and a logistic regression classifier achieve 50.1/50.5% test accuracy. These near-chance results provide empirical support for this agreement at the tested checkpoint.
Reference
[1] Choi et al. (2026), likelihood estimation for diffusion RL.
[2] Lee et al. (2026), Reward Score Matching (RSM).
[3] Dong et al. (2026), GDMD
I thank the authors for a detailed response. While reviewing the updated manuscript, I found that the authors have revealed their names and affiliations. As such, I am unable to continue reviewing under the double-blind policy. I have notified the Area Chair and Program Chairs but have not yet received a response. I will update my assessment after receiving a response.
We apologize for the issue. We immediately removed the problematic version, including its revision history, to ensure that it does not influence others’ assessments.
Response to Reviewer toU3 (1/2)
We thank the reviewer for recognizing the strong empirical results and for identifying the questions about novelty, evaluation fairness, and the role of the theory.
W1. Presentation
We agree that these passages need clearer wording. The intended meanings are:
We have revised Sections 2.1 and 2.2 to clarify these concepts and added an overview figure (Fig. 2).
W2 / Q1. Why DM is more than a combination of reward learning and DMD
The contribution is a correction that adding DMD to an existing reward loss does not provide. We agree that regression, EMA, and distribution matching build on prior work. DM addresses a specific error in using them for few-step reward learning. The model's prediction is trained to make a finite sampling step. The reward update instead needs the average clean output associated with a re-noised input, namely the posterior mean. These quantities need not agree. Eq. (47) identifies the bias caused by using the prediction as the posterior.
DM estimates the posterior from the generator's own samples and uses it in the reward residual $ A(X_0-\hat\mu) $. The same estimate is reused for the teacher update. With an exact posterior, the reward residual has the intended first-order conditional mean. It can be learned from scalar rewards without reward gradients or backpropagation through the sampling process. This is the specific correction and integration that our experiments test.
The cited methods address related goals, but a shared training objective alone does not resolve this issue. The likelihood-estimation view of Choi et al. [1] and the common score-regression view of Lee et al. [2] do not by themselves establish that a finite-step prediction equals its output posterior. TDM-R1, GDMD, MeanFlowNFT, DMDR, and RTDMD already combine distillation and reward optimization. DM's contribution is more specific than performing these tasks together. The matched forward-process control retains the head-substitution bias. Direct reward backpropagation takes another route, but requires differentiable rewards. DM uses a separately estimated posterior to correct the reward residual while retaining support for black-box rewards. Tables 2 and 3 also show that DM improves both trained and held-out rewards over the compared joint methods.
The controlled comparisons show why this distinction matters. With the same EMA and reward processing, changing the reward reference from the head to the posterior improves MNR from 1.441 to 1.568 in Table 4. In our independently rerun joint experiment, both methods also receive the same teacher supervision and posterior-estimator training. The correction improves MNR from 1.482 to 1.572, DPG from 84.22 to 85.69, and ImageReward from 1.4528 to 1.5105. Thus the gains persist when the control already combines reward learning, distillation, and EMA.
The pure-distillation comparison provides additional evidence. DMD2-style + EMA and DM both obtain 0.66 GenEval, while DM improves DPG from 84.21 to 84.90. Our main evidence for the posterior correction is the controlled reward comparison above.