acceptodds
Under review as a conference paper at ICLR 2027

One Stage Few Step Distillation with On Policy Trajectory Anchoring

Abstract

Preference-aligned diffusion models achieve strong generation performance but typically require many sampling steps. Developing few-step diffusion models that retain strong alignment with human preferences remains an important research goal. On-policy distillation (OPD) transfers teacher-aligned capabilities through local supervision, but its performance degrades sharply with fewer sampling steps. Distribution Matching Distillation (DMD) enables few-step generation but can introduce distributional drift and weaken aligned capabilities. Separating the transfer of teacher-aligned capabilities and few-step distillation into two stages can cause the second stage to undermine the aligned capabilities or few-step generation ability learned in the first. To address these limitations, we propose On-Policy Distribution Matching Distillation (OPDMD), a single-stage framework that jointly enforces teacher anchoring and distribution matching along student-generated trajectories. OPDMD anchors the student velocity to the teacher at each selected student state and applies a distribution-matching correction at the corresponding successor state. We unify these two supervision signals into a transition-aware MSE objective that scales the correction by the transition span. Experiments on SD-3.5-Medium with GRPO-aligned expert teachers show that OPDMD better preserves aligned capabilities and distributional fidelity than OPD, DMD, and their sequential combinations. Additional analyses show that transition-aware correction is especially beneficial under sparse sampling and that balancing the two supervision signals is important for overall performance. These results establish OPDMD as an effective approach to preference-preserving few-step generation.

Primary area:
generative models
Keywords:
Few-step generation, On-policy distillation, Vision Generation

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_3JunOpenReview ↗
Rating
3(+1)
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper investigates how to combine OPD and distillation into a single method that improves the generation quality of the student model while reducing the number of generation steps. The paper conducts experiments on SD 3.5, demonstrating the superiority of the proposed method over training with OPD and distillation sequentially.

Critical strengths
  • Combining OPD and DMD into a single distillation method indeed has considerable research value, as it can prevent the gains from the preceding stage from deteriorating when OPD and DMD training are performed sequentially.
Critical weaknesses
  • The paper provides relatively weak theoretical justification for the effectiveness of the method. According to the OPDMD loss in Eq. 20, the proposed method can essentially be viewed as a linear combination of the DMD loss and the OPD loss. Moreover, the OPDMD loss is applied only at a single point rather than along the entire ODE sampling trajectory. The paper does not provide a theoretical justification for the effectiveness of this design, but merely describes how the loss is constructed. I would like the authors to theoretically establish the convergence of the OPDMD loss, or at least explain why this design was chosen over other alternatives.
Questions for authors

See “Critical Weaknesses.”

4 comments

Thank you for recognizing the research value of combining OPD and DMD into a single distillation framework. Below, we explain (1) why our design was chosen over naive linear combination alternatives, (2) the design rationale and idealized balance condition of the objective, and (3) how single-node sampling relates to full-trajectory supervision and actual gradient computation.

1. Why This Design Over Alternatives While the target velocity v~k=sg⁡[vT,k+λhk2gk]\tilde{v}_k = \operatorname{sg}[v_{T,k} + \frac{\lambda h_k}{2} g_k] in Eq. 20 combines two signals, OPDMD differs from a naive linear combination of standard OPD and DMD losses (LOPD+λLDMD\mathcal{L}_{\text{OPD}} + \lambda \mathcal{L}_{\text{DMD}}) in two ways:

  • Transition-Coupled vs. Decoupled States: In a naive linear combination (Joint OPD + DMD), the OPD term supervises an intermediate rollout state xkx_k at noise level σk\sigma_k, whereas the standard DMD term independently re-noises a terminal prediction x^0\hat{x}_0 to a newly sampled noise level σ\sigma (Eq. 11). In OPDMD, both signals are coupled within the same student transition (xk→zk)(x_k \to z_k): the teacher anchors the velocity at xkx_k, while the distribution-matching correction is evaluated at the one-step Euler successor zk=sg⁡[xk]−hkvkz_k = \operatorname{sg}[x_k] - h_k v_k.
  • Jacobian Span Scaling (hkh_k): Because zkz_k depends on vkv_k through ∂zk∂vk=−hkI\frac{\partial z_k}{\partial v_k} = -h_k I (Eq. 18), mapping the successor-state surrogate gradient back to vkv_k introduces the transition span hk=σk−σk+1h_k = \sigma_k - \sigma_{k+1}, scaling the correction according to each transition's sensitivity to velocity changes under our nonuniform schedule.

We empirically compare OPDMD against two-stage sequential pipelines, the one-stage Joint OPD + DMD baseline, and Ours (const. weight) (which replaces hkh_k with the mean span hˉ=1/K\bar{h} = 1/K) at 4 NFE:

MethodNFEFD ↓OCR ↑GenEval ↑PickScore ↑
OPD → DMD (Two-stage)4143.550.84130.831322.60
DMD → OPD (Two-stage)4471.300.57940.682021.06
Joint OPD + DMD (One-stage)4282.740.68270.794821.86
Ours (const. weight, α=λhˉ/2\alpha = \lambda\bar{h}/2)4180.100.83580.874622.32
OPDMD (Transition-aware, Eq. 20)4139.970.88350.884922.68

OPDMD outperforms both Joint OPD + DMD (+0.2008 on OCR, +0.0901 on GenEval) and the constant-weight variant (reducing FD from 180.10 to 139.97 and improving OCR from 0.8358 to 0.8835). Together with the λ\lambda ablation in Fig. 5, these results support the benefit of our transition-coupled design under the evaluated settings.

2. Design Rationale and Idealized Pointwise Balance Condition Our design is motivated by the complementary failure modes of OPD and DMD under coarse sampling: matching only the teacher's instantaneous velocity vT,kv_{T,k} over coarse steps hkh_k accumulates discretization error and produces blurry outputs (Fig. 3), whereas optimizing distribution matching alone does not constrain the local transport path and causes style/contrast drift (Fig. 3).

  • Span-Scaled Correction: In Eq. 20, vT,kv_{T,k} provides a local teacher anchor at xkx_k, while λhk2gk\frac{\lambda h_k}{2} g_k adjusts the target velocity using the correction evaluated at zkz_k, scaled by the transition Jacobian ∂zk∂vk=−hkI\frac{\partial z_k}{\partial v_k} = -h_k I.

  • Idealized Pointwise Balance Condition: Treating vkv_k as an unconstrained vector at a given state, the local surrogate gradient in Eq. 19 vanishes when

    vk∗=vT,k+λhk2gk.v_k^* = v_{T,k} + \frac{\lambda h_k}{2} g_k.

    In the idealized limit where vϕ=vTv_\phi = v_T, the correction vanishes (gk=0g_k = 0) and the target reduces to the local teacher velocity v~k=vT,k\tilde{v}_k = v_{T,k}. When the successor correction is nonzero (gk≠0g_k \neq 0), this pointwise balance condition shifts the target velocity away from vT,kv_{T,k} by λhk2gk\frac{\lambda h_k}{2} g_k. This pointwise balance condition clarifies how the objective combines local teacher anchoring and successor-state correction at a fixed state.

3. Single-Point Formulation vs. Full-Trajectory Supervision and Gradient Path Although Eq. 21 is written for a single step kk, supervision covers all discrete rollout steps in expectation. At each training iteration, we uniformly sample k∼U0,…,K−1k \sim \mathcal{U}\\{0, \dots, K-1\\} from the rollout nodes (Algorithm 1, Line 13), optimizing the expected objective across the trajectory:

LOPDMD=Ek∼U0,…,K−1[∣vθ(xk,σk,c)−v~k∣22].\mathcal{L}_{\text{OPDMD}} = \mathbb{E}_{k \sim \mathcal{U}\\{0, \dots, K-1\\}} \left[ \\| v_\theta(x_k, \sigma_k, c) - \tilde{v}_k \\|_2^2 \right]. Furthermore, because the single-transition Jacobian ∂zk∂vk=−hkI\frac{\partial z_k}{\partial v_k} = -h_k I is analytically absorbed into the detached target v~k\tilde{v}_k (Algorithm 1, Lines 26 and 30), the detached MSE implements a gradient equivalent to the single-transition surrogate while actual backpropagation proceeds only through the single student prediction vk=vθ(xk,σk,c)v_k = v_\theta(x_k, \sigma_k, c). Compared with unrolling gradients through all KK rollout steps (full-trajectory BPTT), this design avoids O(K)\mathcal{O}(K) multi-step activation memory overhead and avoids backpropagating through long chains of multi-step Jacobians.

Thank you again for your thoughtful review. We hope our theoretical clarifications and empirical comparisons address your concerns, and we welcome any additional questions or suggestions you may have.

Thank the authors for their reply. My concerns have largely been addressed, so I decide to raise my score to wa.

Thank you very much for taking the time to read our response and for raising your score! We sincerely appreciate your constructive feedback and insightful questions, which have greatly helped us strengthen both the theoretical clarity and empirical evaluation of our work. We will make sure to incorporate all the new comparisons, ablations, and clarifications discussed above into the revised manuscript.

Reviewer_C9BTOpenReview ↗
Rating
3(+1)
Confidence
2(−1)
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes On-Policy Distribution Matching Distillation (OPDMD), which is a single-stage diffusion distillation approach that enables distilling multiple specialized diffusion teacher models into a single few-step student model. At each selected student rollout state, it combines a teacher velocity anchor with a distribution-matching velocity correction (scaled by the sampling transition span) evaluated at the successor state. These signals form a detached transition-aware MSE target as the training objective. Empirical evaluation demonstrates OPDMD's capability preservation performance and teacher-distribution similarity for 4-step and 8-step student models. These results are compared with relevant baselines such as OPD, DMD and their sequential combinations.

Critical strengths
  1. OPDMD addresses a practically valuable problem: preserving preference-aligned teacher capabilities under few-step sampling for diffusion models. The motivation is reasonable and the presentation is generally clear.
  2. Experiments on SD-3.5-Medium demonstrate OPDMD's significant performance gain over two-stage sequential baselines for the 4-step sampling setting.
Critical weaknesses
  1. The training distribution for the fake-score model does not match the distribution being corrected: the fake model is trained on forward-noised terminal student samples, but then it is queried on intermediate student rollout successors. However, there are discrepancies between these distributions for a finite-step sampler, so even a perfectly trained fake model will not correctly estimate the score of the successor-state distribution.
  2. The paper does not compare OPDMD with closely related trajectory distribution matching methods such as TDM.
  3. The noise grid hkh_k is central to the transition-aware loss, but the paper does not specify what kind of noise grid is actually used and lacks an ablation study that compares this transition-aware weighting with independently tuned constant weighting.
Questions for authors
  1. How do you evaluate the final-step correction at the "zero-noise" successor state as stated in Appendix A? I understand that σ\sigma must be positive in these models, otherwise Eq. 3 and Eq. 7 are not well-defined.
  2. Could the authors empirically compare OPDMD with existing trajectory distribution matching methods such as TDM using the same teachers, and discuss why the local velocity anchor in OPDMD is a better design choice?
  3. What noise grid hkh_k do you use and how sensitive is the model performance to the choice of hkh_k? Fig. 5 shows that the performance is very sensitive to λ\lambda, so I suppose that hkh_k is an important hyperparameter as it also affects the weighting in the loss, but it's not specified in the paper.
  4. Could the authors conduct an ablation study to compare the empirical performance of this transition-aware weighting with a simple baseline that uses independently tuned constant weighting?
  5. Could the authors specify the exact formula for the normalization Nσ(⋅)\mathcal{N}_\sigma(\cdot)?
  6. Could the authors explain why OPDMD has larger performance gain for 4-step than 8-step? How do OPDMD and other baselines perform in the 1-step setting?
7 comments

Thank you for your careful assessment of the method and for recognizing its practical motivation and 4-step results. Below, we address the score-distribution mismatch, zero-noise endpoint evaluation, comparison with TDM, noise grid and transition weighting, constant-weight ablations, normalization formula, and step-count behavior.

1. Fake-Model Distribution Mismatch & Zero-Noise Endpoint Evaluation (Weakness 1 & Q1)

  • Fake-Model Training vs. Successor-State Querying: We agree that for a finite-step sampler, the forward-noised terminal distribution used to train vϕv_\phi serves as a tractable proxy rather than the exact marginal of the intermediate successor zkz_k. Training vϕv_\phi on re-noised terminal predictions x^0\hat{x}_0 provides well-defined flow-matching targets (ϵ−x^0\epsilon - \hat{x}_0) across all noise levels (Algorithm 1, Lines 6–9), whereas we formulate gkg_k in Eq. 16 as a velocity-space correction evaluated at zkz_k (with Eqs. 19–21 establishing exact algebraic gradient equivalence for the unified MSE target).

    OPDMD combines this correction with a direct teacher-velocity anchor at xkx_k. In Joint OPD + DMD, the DMD queries are evaluated on re-noised terminal predictions, matching the fake model's training construction. In the evaluated four-step setting, OPDMD achieves lower FD (139.97 vs. 282.74) and higher OCR accuracy (0.8835 vs. 0.6827). These results support the practical effectiveness of our transition-coupled formulation.

  • Zero-Noise Endpoint (σK=0\sigma_K = 0): As shown in Algorithm 1 and the normalization formula in Q5 below, our training implementation operates directly in velocity space. The score identities in Eqs. 3 and 7 are explicitly restricted to σ>0\sigma > 0 to provide background context in the Preliminaries and are not used in the training algorithm. At the final transition (k=K−1k = K-1, where σK=0\sigma_K = 0), we directly evaluate the velocity networks vT(zK−1,0,c)v_T(z_{K-1}, 0, c) and vϕ(zK−1,0,c)v_\phi(z_{K-1}, 0, c) and normalize their difference without dividing by σK\sigma_K.

2. Empirical Comparison and Design Discussion vs. TDM (Weakness 2 & Q2) We adapted TDM to SD-3.5-Medium using the same three GRPO-aligned preference teachers, prompt router, LoRA configuration (r=32,α=64r=32, \alpha=64), CFG scales (teacher 4.5, student 1.0), and 2,000-step student update budget at 512×512512 \times 512 resolution. TDM uses one fake-model update per student update, while OPDMD uses five:

MethodNFEFD ↓OCR ↑GenEval ↑PickScore ↑
TDM4335.360.32190.631921.88
OPDMD (Ours)4139.970.88350.884922.68
TDM8225.390.41810.745622.08
OPDMD (Ours)897.900.86390.903722.76

OPDMD achieves better scores across all metrics at both 4 and 8 NFE, with particularly strong gains on OCR (0.3219 →\to 0.8835 at 4 NFE; 0.4181 →\to 0.8639 at 8 NFE). Distribution matching alone does not explicitly constrain the student's local velocity to follow the routed teacher at student-visited states, which can deviate under coarse sampling. Our source-state velocity anchor provides direct supervision from the routed teacher at visited states, while the successor-state correction helps mitigate distributional errors introduced by few-step sampling.

3. Noise Grid and Transition Weighting (Weakness 3 & Q3) We use fixed, nonuniform noise grids obtained by subsampling the SD-3.5 16-step schedule with shift = 3, followed by a zero-noise endpoint σK=0\sigma_K = 0:

  • 4-step grid (σ0,…,σ4\sigma_0, \dots, \sigma_4): [1.00,0.72,0.43,0.18,0.00][1.00, 0.72, 0.43, 0.18, 0.00], with transition spans hk=[0.28,0.29,0.25,0.18]h_k = [0.28, 0.29, 0.25, 0.18].
  • 8-step grid (σ0,…,σ8\sigma_0, \dots, \sigma_8): [1.00,0.89,0.77,0.66,0.52,0.43,0.32,0.18,0.00][1.00, 0.89, 0.77, 0.66, 0.52, 0.43, 0.32, 0.18, 0.00], with transition spans hk=[0.11,0.12,0.11,0.14,0.09,0.11,0.14,0.18]h_k = [0.11, 0.12, 0.11, 0.14, 0.09, 0.11, 0.14, 0.18].

Since OPDMD relies on teacher anchoring, we retain the teacher's native shift = 3 schedule so that noise-level queries remain consistent with the teacher's parameterization, rather than tuning the grid as a separate hyperparameter. In Eq. 20, hk=σk−σk+1h_k = \sigma_k - \sigma_{k+1} is determined by the fixed grid and scales each step's correction according to the transition Jacobian ∂zk/∂vk=−hkI\partial z_k / \partial v_k = -h_k I. Figure 5 evaluates sensitivity to the correction strength λ\lambda on the 4-step grid, while the constant-weight comparisons below assess the contribution of transition-dependent weighting.

4. Ablation Against Independently Tuned Constant Weighting (Weakness 3 & Q4) In Table 1, Ours (const. weight) uses α=λhˉ/2=7.5\alpha = \lambda \bar{h} / 2 = 7.5 at 4 NFE (since λ=60\lambda = 60 and hˉ=1/4\bar{h} = 1/4), whereas our transition-aware weighting uses λhk/2=[8.4,8.7,7.5,5.4]\lambda h_k / 2 = [8.4, 8.7, 7.5, 5.4] across the four steps. To address your request, we conducted two additional 4-step constant-weight experiments with independently selected coefficients α∈5.0,10.0\alpha \in \\{5.0, 10.0\\} in v~k=sg⁡[vT,k+αgk]\tilde{v}_k = \operatorname{sg}[v_{T,k} + \alpha g_k] under the same training setup:

Weighting Scheme (4 NFE)FD ↓OCR ↑GenEval ↑PickScore ↑
Constant weight (α=5.0\alpha = 5.0)267.750.82380.878121.95
Constant weight (α=7.5\alpha = 7.5, mean span)180.100.83580.874622.32
Constant weight (α=10.0\alpha = 10.0)439.790.88000.827220.90
Transition-aware (λhk/2\lambda h_k / 2, Ours)139.970.88350.884922.68

Increasing the constant weight from α=5.0\alpha = 5.0 to 10.010.0 improves OCR but degrades FD, GenEval, and PickScore, mirroring the trade-off observed for large λ\lambda in Fig. 5. Neither additional constant weight—nor the mean-matched constant α=7.5\alpha = 7.5—matches the transition-aware result across all metrics. These results support the benefit of transition-aware weighting under the evaluated settings.

5. Exact Formula for the Normalization Nσ(⋅)\mathcal{N}_{\sigma}(\cdot) (Q5) For each sample, let vT+=vT(zk,σk+1,c)v_T^+ = v_T(z_k, \sigma_{k+1}, c), vϕ+=vϕ(zk,σk+1,c)v_\phi^+ = v_\phi(z_k, \sigma_{k+1}, c), and x^0,k=xk−σkvk\hat{x}_{0,k} = x_k - \sigma_k v_k. The per-sample normalization is computed in FP32 over latent channels and spatial dimensions (C,H,W)(C, H, W) as:

ak=mean⁡C,H,W∣vT+−(ϵ′−x^0,k)∣,ϵ′∼N(0,I),a_k = \operatorname{mean}_{C,H,W} \left| v_T^+ - (\epsilon' - \hat{x}_{0,k}) \right|, \qquad \epsilon' \sim \mathcal{N}(0, I),

Nσk+1(vT+−vϕ+)=vT+−vϕ+max⁡(ak,10−6),gk=sg⁡![Nσk+1(vT+−vϕ+)].\mathcal{N}_{\sigma_{k+1}}(v_T^+ - v_\phi^+) = \frac{v_T^+ - v_\phi^+}{\max(a_k, 10^{-6})}, \qquad g_k = \operatorname{sg}\\!\left[ \mathcal{N}_{\sigma_{k+1}}(v_T^+ - v_\phi^+) \right].

Here, ϵ′∼N(0,I)\epsilon' \sim \mathcal{N}(0, I) is freshly sampled Gaussian noise independent of zkz_k, used only to compute the normalization denominator aka_k (it does not re-noise zkz_k). The denominator is clamped at 10−610^{-6} for numerical stability. We will add this formula to Section 3.3.

6. Performance Gain at 4 vs. 8 Steps and Behavior in the 1-Step Setting (Q6)

  • 4 Steps vs. 8 Steps: The larger gain at 4 steps is consistent with the greater difficulty of preserving teacher behavior under coarser sampling. Each transition covers a larger average noise interval (hˉ=0.25\bar{h} = 0.25 vs. 0.1250.125), and for a fixed source state, a velocity error Δvk\Delta v_k produces a successor-state error Δzk=−hkΔvk\Delta z_k = -h_k \Delta v_k. As shown in Table 1, the 8-step OPD baseline already preserves much of the teacher's capabilities (0.8591 OCR, 0.8768 GenEval), leaving a smaller margin for improvement (+0.0048 OCR, +0.0269 GenEval), whereas 4-step OPD degrades sharply and benefits substantially from the successor-state correction (+0.3472 OCR, +0.2261 GenEval).
  • 1-Step Setting (K=1K=1): Under our current experimental settings, one-step OPDMD does not produce visually acceptable samples, and we therefore focus our quantitative comparisons and claims on 4- and 8-step generation. Intuitively, in a 1-step rollout (σ0=1→σ1=0\sigma_0 = 1 \to \sigma_1 = 0), the only source state is pure Gaussian noise x0∼N(0,I)x_0 \sim \mathcal{N}(0, I), so the student never visits intermediate low-noise states (0<σk<10 < \sigma_k < 1) where fine-grained structural teacher guidance is most informative during multi-step rollouts.

Thank you again for your detailed and insightful comments. Please let us know if our responses and additional experiments resolve your concerns, or if you have any further questions we can address.

Thank you for the detailed responses, which address several of my concerns. However, the following points remain:

  1. Fake-score distribution mismatch. The new comparison with joint OPD+DMD supports the method’s practical effectiveness, but still does not establish that the proposed correction is a valid distribution-matching gradient for the successor-state distribution. This concern could be addressed by consistently presenting the correction as a heuristic approximation based on a proxy distribution and qualifying the associated theoretical claims throughout the manuscript.

  2. Sensitivity to the noise grid. Thanks for specifying the noise grids. However, it remains unclear how sensitive performance is to these choices. Could the authors provide a sensitivity analysis varying the grid or shift parameter?

  3. 1-step sampling. The explanation for the larger gains at 4 steps compared with 8 steps is helpful. For the 1-step setting, could the authors also provide quantitative results for OPDMD and the other baselines?

Reviewer_fjSJOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes OPDMD, a single-stage method for distilling preference-aligned diffusion models into few-step (4/8 NFE) students while preserving the teacher’s aligned behavior. During the student’s rollout, the teacher’s velocity guides the student at its current state, and a DMD term corrects its distribution at the next state. Both signals are combined into one training target. The authors show that sequential pipelines (OPD→DMD and DMD→OPD) cause distribution drift or loss of the experts’ aligned capabilities. In experiments, OPDMD distills GRPO experts into one SD3.5-Medium student and outperforms OPD, DMD and both sequential pipelines.

Critical strengths

The paper addresses the practical problem of preserving RL-aligned capabilities under step distillation. Furthermore, its multi-teacher setting distills some experts into a few-step student which is harder than single-task distillation. The method is also simple but seems yet effective. It uses a single detached target and an MSE loss, with no extra loss terms beyond a standard DMD fake model.

Critical weaknesses

-The novelty is incremental. Matching distributions at intermediate states of the student’s trajectory, with gradients passed through one sampling step, is already established by TDM [1]. OPDMD’s distribution-matching term closely follows this design and adds an OPD anchor. Combining a teacher-based loss with a DMD loss under a balancing weight is also not new. rCM [2] does this, and the original DMD paired a regression loss with distribution matching.

-The evaluation is narrow. It uses one backbone at 512² resolution. The main comparison in Table 1 includes only OPD, DMD and their sequential combinations. DMD2 and consistency distillation appear only on T2I-CompBench++ and are trained from Flow-OPD checkpoints rather than the same teachers. Closely related few-step methods such as TDM [1], π-Flow [3], rCM [2] and Decoupled DMD [4] are not compared. Moreover, the simplest one-stage baseline is missing. The paper never compares against the OPD loss plus a standard DMD loss trained jointly, so the benefit of the successor-state design cannot be separated from the benefit of joint training. Also apart from T2I-CompBench++ and FD, the metrics are the same rewards the teachers were trained on. Out-of-domain quality metrics such as Aesthetic Score or ImageReward are missing.

[1] Luo Y, Hu T, Sun J, Cai Y, Tang J. Learning few-step diffusion models by trajectory distribution matching. In2025 IEEE/CVF International Conference on Computer Vision (ICCV) 2025 Oct 19 (pp. 17719-17728). IEEE.

[2] Zheng K, Wang Y, Ma Q, Chen H, Zhang J, Balaji Y, Chen J, Liu MY, Zhu J, Zhang Q. Large scale diffusion distillation via score-regularized continuous-time consistency. InInternational Conference on Learning Representations 2026 Apr 20 (Vol. 2026, pp. 2582-2603).

[3] Chen H, Zhang K, Tan H, Guibas L, Wetzstein G, Bi S. pi-flow: Policy-based few-step generation via imitation distillation. InInternational Conference on Learning Representations 2026 Apr 20 (Vol. 2026, pp. 151521-151547).

[4] Liu D, Peng G, Liu D, DU D, Li Z, Wu Q, Jin X, Cao S, Zhang S, HOI S, Li H. Decoupled dmd: Cfg augmentation as the spear, distribution matching as the shield. InInternational Conference on Learning Representations 2026 Apr 20 (Vol. 2026, pp. 140643-140666).

Questions for authors

-Could the authors explain how OPDMD differs from TDM, which also matches distributions at intermediate states of the student’s trajectory with gradients passed through one sampling step? How does OPDMD compare to TDM under identical teachers, steps and compute?

-What is the training cost relative to the baselines, given that each student update includes 5 fake-model updates with a full student rollout each?

5 comments

Thank you for your constructive review and for recognizing the practical value of preserving RL-aligned capabilities in multi-teacher few-step distillation, as well as the simplicity and effectiveness of our unified objective. Below, we clarify our methodological distinctions from prior work, present new comparisons using the same teachers (including TDM and Joint OPD + DMD), report additional evaluation metrics, and analyze the training cost.

1. Distinction and Empirical Comparison with TDM (Weakness 1, Weakness 2 & Q1) While both TDM [1] and OPDMD evaluate distribution matching at intermediate rollout states, OPDMD adds explicit source-velocity anchoring and integrates it with the successor-state correction as follows:

  • On-Policy Source Velocity Anchoring: TDM relies on distributional score differences to align intermediate marginals without explicitly constraining the student's local velocity at visited states. In contrast, OPDMD anchors the student's velocity at the source state xkx_k to the routed preference teacher (vT,kv_{T,k}), providing direct local supervision along student rollouts.
  • Transition-Coupled Target with Span Scaling: OPDMD couples the source-state velocity anchor at xkx_k with a distribution-matching correction evaluated at the one-step Euler successor zk=sg⁡[xk]−hkvkz_k = \operatorname{sg}[x_k] - h_k v_k. Mapping this successor correction back to the source velocity vkv_k via ∂zk∂vk=−hkI\frac{\partial z_k}{\partial v_k} = -h_k I scales the correction by the transition span hk=σk−σk+1h_k = \sigma_k - \sigma_{k+1} within a single detached MSE target (Eq. 20).

We adapted TDM to SD-3.5-Medium using the same three GRPO-aligned preference teachers, prompt router, resolution (512×512512 \times 512), LoRA configuration (r=32,α=64r=32, \alpha=64), CFG scales (teacher 4.5, student 1.0; we will also clarify this phrasing in Appendix A), and 2,000-step student update budget. This comparison matches the teachers and student-update budget, though not total training compute:

MethodNFEFD ↓OCR ↑GenEval ↑PickScore ↑
TDM [1]4335.360.32190.631921.88
OPDMD (Ours)4139.970.88350.884922.68
TDM [1]8225.390.41810.745622.08
OPDMD (Ours)897.900.86390.903722.76

OPDMD outperforms routed TDM across all metrics at both 4 and 8 NFE. The improvement is particularly pronounced for OCR accuracy, which increases from 0.3219 to 0.8835 at 4 NFE and from 0.4181 to 0.8639 at 8 NFE.

2. Training Cost Relative to Baselines (Q2)

  • Comparison with DMD and Two-Stage Baselines: Following the two-time-scale update rule in DMD2, our DMD only baseline and the DMD stage of the sequential baselines (OPD → DMD and DMD → OPD) in Table 1 also perform M=5M=5 fake-model updates per student update (whereas the OPD stage does not train a fake model). Compared with a DMD update, each student update in OPDMD adds one local teacher forward pass at the source state xkx_k. While the sequential baselines train for two 2,000-step stages (4,000 student updates in total, with 16-step rollouts used in the OPD stage of OPD → DMD), OPDMD is trained for a single 2,000-step stage (taking approx. 12 hours on 8×A100 GPUs in the 4-step setting).
  • Comparison with TDM: Compared with our TDM implementation (M=1M=1), OPDMD performs four additional fake-model updates (each with a detached student rollout) and one additional source-state teacher query per student update. All additional computations occur only during training; at inference time, OPDMD introduces no extra overhead at the same backbone, resolution, and NFE.

3. Relationship to Prior Hybrid Distillation Methods [2, 3, 4] (Weakness 1 & Weakness 2)

  • Comparison with [2, 3, 4]: Original DMD pairs terminal distribution matching with an offline regression loss on pre-generated teacher pairs, and rCM [2] combines consistency training with score regularization. Meanwhile, π\pi-Flow [3] studies policy-based imitation distillation, and Decoupled DMD [4] investigates decoupling CFG augmentation from distribution matching. In contrast, our work focuses on preserving multiple routed preference-teacher capabilities in a single few-step student by combining a local teacher anchor at the student's current state with a span-scaled distribution correction at its one-step successor within a detached MSE objective. We are currently adapting and running empirical comparisons with [2, 3, 4] under our multi-teacher setup and will incorporate these results.
  • Clarification on Table 2 Setup: In Table 2 (T2I-CompBench++), DMD2 and CD were trained from the publicly released Flow-OPD checkpoint to follow Flow-OPD's evaluation protocol. To compare methods trained directly from the same GRPO expert teachers, we provide the routed TDM experiments above (Point 1) and the one-stage Joint OPD + DMD experiments below (Point 4).

4. Comparison with Joint OPD + DMD & Additional Quality Metrics (Weakness 2)

  • Comparison with One-Stage Joint OPD + DMD: We implemented a one-stage Joint OPD + DMD baseline that jointly optimizes the on-policy teacher velocity loss Lanchor(k)\mathcal{L}_{\text{anchor}}^{(k)} at xkx_k (Eq. 10) and the standard DMD loss LDM\mathcal{L}_{\text{DM}} evaluated on independently re-noised terminal clean predictions x~σ=(1−σ)x^0+σϵ\tilde{x}_\sigma = (1-\sigma)\hat{x}_0 + \sigma\epsilon (Eqs. 11–13) under the 4-step setting:

    MethodNFEFD ↓OCR ↑GenEval ↑PickScore ↑
    Joint OPD + DMD (One-stage)4282.740.68270.794821.86
    Ours (const. weight, α=7.5\alpha = 7.5)4180.100.83580.874622.32
    OPDMD (Transition-aware, Ours)4139.970.88350.884922.68

    OPDMD outperforms both Joint OPD + DMD (+0.2008 on OCR, +0.0901 on GenEval, and lower FD) and the constant-weight baseline under the evaluated settings. These results support the benefit of evaluating the correction at the one-step successor state and scaling it by the transition span hkh_k.

  • Evaluation Beyond Teacher-Training Rewards: To assess performance on preference metrics not used to train the GRPO teachers, we evaluated HPSv3 and ImageReward:

    MethodNFEHPSv3 ↑ImageReward ↑
    Joint OPD + DMD44.77640.7122
    TDM [1]48.35921.1247
    OPDMD (Ours)48.44211.2174
    TDM [1]89.11541.1974
    OPDMD (Ours)89.62691.3022

    OPDMD outperforms Joint OPD + DMD at 4 NFE and exceeds TDM at both 4 and 8 NFE on both metrics, showing that its improvements extend to preference evaluators beyond the teacher-training rewards.

Thank you again for your valuable feedback. We hope these clarifications and new experiments address your concerns, and we would be glad to answer any further questions during the discussion period.

Thank you for the clarification and additional experiments. My concern about novelty still remains. The “span scaling” is not a new design choice; it is the factor that automatically appears in the gradient when the matching loss is backpropagated through one sampling step, which TDM already does. To me, it seems that the main addition is therefore the on-policy teacher anchor combined with a TDM-style loss, considering TDM already matches distributions at intermediate states of the student trajectory. Moreover, the TDM baseline seems under-tuned. Since TDM extends DMD to intermediate trajectory states, it should perform at least close to DMD. Yet it falls well below your own DMD-only baseline in paper’s Table 1 on all metrics. My concerns about the evaluation also remain. The comparisons with π-Flow, rCM and Decoupled DMD are not yet available, and the use of only a single backbone at 512² resolution is not resolved yet. Therefore, I will maintain my original score.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.