- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 3
Read review
Paper summary
TRD combines ArcFlow's momentum-mixture velocity policy with distribution matching distillation. One student network does closed-form one-step generation, self-estimates the marginal velocity on re-noised samples and regresses teacher velocities on short trajectories, so no separate fake network is needed. An EMA endpoint term adds stability. Experiments on SANA-1.6B and SD3.5-Medium at 1024×1024 show clear one-step gains over retrained baselines at some cost in diversity.
Critical strengths
- Shared self-estimation is an identifiable contribution. Each part has precedents, but using one policy network for generation, velocity estimation and trajectory supervision removes the fake network, and the ablations show these choices matter. I don't think it is fair to call it a renamed method.
- I think the results is the most convincing part. One-step FD-DINOv2 is about 70 versus 97 for the next best baseline on SANA, and about 87 versus 118 on SD3.5-Medium, across many metrics and baselines. This is the main reason I see the paper as close to borderline.
- The notation table, algorithm box and configurations are complete, Proposition 1 shows the four losses share an ideal target, and the authors are upfront about limits such as the diversity cost, single runs and how CFG affects the theory.
Critical weaknesses
1 The gain over well-tuned distribution-matching baselines is not isolated. The main table matches inference NFE, but Fig. 1 shows different training lengths. All baselines were retrained by the authors with batch size 8, and full budgets and convergence are not reported. Removing L_fm and L_dm together cannot separate them, and removing L_tr or changing the fake estimator also shifts quality. So it is unclear how much of the advantage comes from the new components rather than budget, tuning and the EMA term. My concern is that the increment is not isolated, not that quality comes only from the distribution term. 2 The efficiency claims cannot be checked, and the trade-offs are not quantified. Table 2 gives no common FD target or timing rule, and its "Trainable params" do not match the description of a frozen backbone with LoRA. Every result is a single run, so the small two-step leads are not significant, and DINO Diversity is the lowest in three of four settings. This weakens the claim that quality and efficiency improve together. 3 The momentum-mixture parameterization, closed-form integration and teacher-velocity regression come from ArcFlow, as the paper acknowledges. Mixing trajectory and distribution objectives has precedents in DMD, rCM and SANA-Sprint, and rCM is not cited. Removing the fake network is also explored by concurrent work such as FSF-DMD and TwinFlow. The new part is shared self-estimation within this combination. I think this is a reasonable contribution only.
Questions for authors
Maybe more refer to major weakness. And what are the common FD target, the first-hit rule and the time units in Table 2, and does the 1.9B trainable count include the frozen backbone? What are the full training budgets and convergence curves of each method, and could you compare TRD with the main baselines and MeanFlow at a common time budget? Do the conclusions hold when L_fm and L_dm are removed separately, and when the baselines get the same EMA term? How is the separate fake network in Table 7 configured? Do you have repeated runs to quantify the uncertainty of the one-step and two-step results?
3 comments
Q1: What are the common FD target, first-hit rule, and time units in Table 2?
The FD target refers to convergence rather than a fixed numerical FD-DINOv2 threshold shared by all methods. We use a standard plateau criterion: FD-DINOv2 is considered converged at the earliest checkpoint after which the best value improves by less than 1% over three consecutive evaluation intervals. The reported time is the wall-clock time to this first converged checkpoint, normalized relative to DMD: the duration of one DMD training update is defined as one time unit.
Q2: Does the 1.9B count include the frozen backbone?
Thank you for pointing this out. The 1.9B value includes the frozen 1.6B backbone and the added policy parameters, so the column should be labeled "Total params" rather than "Trainable params". We will correct it.
Q3: Do the conclusions hold when and are removed separately?
We also tried removing and separately, and either removal caused training to collapse. These two losses form one coupled distribution-matching mechanism and are jointly required by the mathematical training objective: identifies the student's marginal velocity, which then uses together with the teacher velocity to update the generated distribution toward the teacher distribution.
Q4: What happens when a baseline receives the same EMA term?
We conducted controlled experiments that add to DMD and remove it from the separate-estimator TRD variant. Please refer to our response to reviewer 95e6, W2, for the complete results and analysis.
Q5: How is the separate fake network in Table 7 configured?
It is a conventional full-size DMD fake-velocity network initialized from the teacher. Only the student marginal-velocity estimate used by distribution matching is replaced by the separately trained network.
Q6: Do you have repeated runs?
We did not repeat each experiment or compute run-level confidence intervals. Each full-scale training run at is computationally expensive, and repeated training with confidence intervals is uncommon in large-scale diffusion distillation; DMD, DMD2, SANA-Sprint, and ArcFlow likewise report point estimates without run-level confidence intervals.
Thank you again for the detailed and constructive review. Your questions identified several places where the comparison protocol and the scope of our claims should be stated more precisely. Please let us know if you have any further questions and we are happy to discuss them.
I appreciate the authors’ efforts. The new ablations clarify the role of trajectory supervision and show that the EMA term alone does not explain TRD’s gain over DMD. After considering the other reviewers’ comments, my main concern remains whether TRD retains its reported advantage over well-tuned baselines at the same training budget. I will therefore maintain my original score.
Thank you for recognizing the empirical strength of TRD and the clarity of our analysis and presentation. We address the concerns about comparison fairness, efficiency, novelty, and the requested experimental details below.
W1: Fairness and isolation of the gains
Baseline training. We trained every reimplemented baseline until its performance had converged and tuned its method-specific hyperparameters to obtain its best result. All methods use the same evaluation protocol. The right panel of Figure 1 reports the resulting one-step FD-DINOv2 curves on a common normalized wall-clock axis. Its horizontal axis is logarithmic, so equal horizontal distances represent multiplicative rather than additive increases in training time and the late portions are visually compressed. The reported endpoints correspond to the converged checkpoints. The training budgets and per-iteration costs are reported in Table 2.
Isolation of the components. We agree that the original ablations do not fully separate all effects. We therefore performed two additional controlled studies. First, we conducted a full 2×2 ablation that independently varies the estimator and trajectory supervision, demonstrating that their contributions are distinct and complementary. Please refer to our response to reviewer YinY, Q2, for the complete results.
Second, we separately evaluated the EMA-based distribution-preservation loss by adding Ldp to DMD and removing it from the separate-estimator TRD variant. The results show that Ldp alone does not explain TRD's gain over DMD and instead specifically stabilizes the coupled shared-estimator training. Please refer to our response to reviewer 95e6, W2, for the complete results.
W2: Scope of the quality and efficiency claims
Section 5.3, Table 2, and the right panel of Figure 1 report per-iteration time, peak memory, full training time, and time-quality curves. These results support a specific efficiency claim relative to fake-model distribution matching: TRD removes the auxiliary fake model, reduces peak memory from 75/80 GB for DMD/DMD2 to 45 GB, and reaches its converged result in 4,500 normalized time units rather than 10,000/12,000.
For quality, the main SANA results in Table 1 and the SD3.5-Medium results in Table 5 show that TRD achieves the strongest one-step results on the primary fidelity, preference, aesthetic, and text-alignment metrics. The two-step results remain strongest on most preference and quality metrics. We also explicitly report the diversity and coverage trade-off: TRD is not best on DINO Diversity, and some two-step baselines have higher Recall-DINOv2.
W3: Contribution relative to ArcFlow and related methods
We agree that TRD inherits ArcFlow's momentum-mixture parameterization and that hybrid supervision and fake-score-free estimation have precedents. TRD's contribution is the way the ArcFlow curve-valued output integrates these signals. The same predicted curve serves as the endpoint generator, supplies its start-time value as the student's marginal-velocity estimator, and supplies its values over the teacher grid for trajectory supervision. Distribution and trajectory supervision therefore act on different evaluations of one policy, with no separately optimized fake model.
Our response to reviewer 95e6 W1 gives a detailed comparison with FSF-DMD, TwinFlow, Self-Evaluation, SANA-Sprint, rCM, and TDM. We will also add the missing rCM citation.