- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
This paper proposes MetaDMD, a few-step posterior sampler for test-time steering. By extending distribution matching distillation to conditional posterior sampling, MetaDMD avoids explicit trajectory matching and provides a more efficient training framework for posterior samplers. The method achieves improvements over existing baselines on both ImageNet and text-to-image generation tasks.
Critical strengths
- By extending distribution matching distillation to conditional posterior sampling, this paper explores a well-motivated and theoretically grounded direction for efficient test-time steering.
- The evaluation on large-scale text-to-image models demonstrates that the proposed method is not limited to small-scale settings and provides convincing evidence of its practical effectiveness.
Critical weaknesses
The claim that avoiding trajectory matching is the main reason for faster convergence may be overstated. While MetaDMD avoids explicit trajectory-level losses, DMD still performs distribution matching along the probability path. Therefore, the improvement may come from the optimization behavior of the DMD framework itself, which is not sufficiently disentangled in the experiments.
Questions for authors
Why is DMD used instead of DMD2? Given that adversarial training has shown benefits for one-step and few-step generative distillation, could a DMD2-style objective further improve the quality and stability of MetaDMD?
5 comments
I am still not fully convinced by the explanation. Although independently re-noising the student output removes the explicit sample-wise correspondence with the teacher's ODE trajectory, isn't the gradient loss itself still exploiting the teacher's denoising-field information along the probability path through the difference between the real and fake denoisers? In this sense, teacher-specific denoising information is still incorporated into the supervision, even though explicit trajectory matching is removed. I would appreciate a more explicit clarification of this point.
We thank the reviewer for this follow-up question. We agree with the reviewer that MetaDMD also exploits the teacher's denoising-field information. However, the key difference lies in how this information is used. DMD uses the teacher's denoising-field information to compute the score difference between the teacher and student distributions, which provides the gradient for minimizing the KL divergence between two distributions in the noised space. In contrast, flow maps use the same denoising-field information in consistency objectives to approximate the exact integration of the teacher's ODE, i.e., trajectory. This strong constraint leads to unstable, costly optimization behavior. MetaDMD avoids such trajectory-level consistency objectives, which we believe contributes to its more efficient training.
In summary, both DMD and flow maps use the teacher's denoising-field information but impose different constraints: DMD matches the student and teacher distributions, whereas flow maps aim to reproduce the exact individual trajectories induced by the teacher's ODE.
We thank the reviewer for the valuable feedback, mainly asking about the source of the faster convergence of MetaDMD and our choice to omit the adversarial loss of DMD2. All revisions are shown in skyblue in the revised PDF, and those addressing your comments are marked with cdL2 in orange.
W1. Source of faster convergence.
We agree that DMD matches distributions at multiple noise levels along the probability path, and we would like to clarify why this does not act as a trajectory constraint. In MetaDMD, the student output is noised with fresh Gaussian noise that is independent of the inner noise z0 used to generate it. The noised distribution at every noise level is therefore determined only by the student’s output distribution p1∣tθ(⋅∣xt), so matching it across noise levels is equivalent to matching this endpoint distribution. The trajectory distillation of MFM instead requires each z0 to pass through the intermediate states zs along the inner ODE path, which MFM enforces with consistency objectives that are known to be unstable to train [1, 2], and we believe that removing this requirement contributes to the efficient training of the sampler (Table (a)). We have clarified this point in the revised manuscript (Sec. 3.1).
Table (a) GenEval score on SD3.5-M, N=8, gradient-free estimator. (Fig. 3, Sec. 4.2; Tab. 11, App. F)
Q1. DMD versus DMD2.
We thank the reviewer for pointing out the possibility of applying adversarial loss to MetaDMD. Distribution matching remains a core component of DMD2, and our objective is to extend it to conditional posterior sampling. We largely follow DMD2’s implementation, including the two-time-scale update rule, but omit the adversarial loss to isolate the contribution of the distribution-matching objective for training the stochastic posterior sampler. We have clarified these implementation choices and their relationship to DMD2 in the revised manuscript (App. D), and leave the integration of adversarial loss to MetaDMD as a future work.
[1] Hu et al. CMT: Mid-training for Efficient Learning of Consistency, Mean Flow, and Flow Map Models. In ICLR, 2026.
[2] Geng et al. Improved Mean Flows: On the Challenges of Fastforward Generative Models. In CVPR, 2026.