acceptodds
Under review as a conference paper at ICLR 2027

PDMD: Projected Distribution Matching Distillation for Video Diffusion Models

16 views

Abstract

Modern video diffusion models require tens of denoising evaluations over long spatiotemporal token sequences. Distribution Matching Distillation (DMD) reduces the number of function evaluations (NFE) to just a few. However, DMD samples can degrade during training, exhibiting progressive oversaturation and artifacts. We trace this instability to critic errors, which enter successive student updates and accumulate over time. We introduce Projected Distribution Matching Distillation (PDMD) to filter critic errors. PDMD projects out the component of the DMD update parallel to the student–critic endpoint residual. At a fixed noisy query, we prove that this residual is an unbiased estimate of the critic's endpoint error. Under high-dimensional assumptions, this projection removes a constant fraction of critic error while discarding only a vanishing fraction of ideal DMD signal. Empirically, the projection stabilizes training and improves sample quality where DMD degrades and develops unnatural textures. PDMD requires only a one-line code change to DMD, with no extra loss, network, data, model pass, or multi-stage training. With Wan2.1, PDMD achieves a VBench total score of 83.73 at 4 NFE, surpassing matched DMD by 1.03 points. On MiniMax-H3 joint video–audio generation, PDMD achieves a VideoGen-Eval visual total score of 83.17, 0.41 points above the strongest distilled baseline. PDMD also achieves the best performance on all six audio metrics among the compared 4-NFE models. Qualitative comparisons and user studies favor PDMD over the distilled baselines in visual quality, motion, and audio quality. Code and models will be publicly available.

Primary area:
generative models
Keywords:
diffusion distillation, distribution matching distillation, few-step generation, video generation, training stability

Reviews

Among:
Shape:
1234this paper, Top 4%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.33 over 3 reviews, above 96% of the others.
Reviewer_jkw2OpenReview ↗
Rating
4(+1)
Confidence
3
Informativeness
3
Presentation
3
Soundness
3(+1)
Read review
Paper summary

This paper proposes PDMD to reduce the training instability of DMD, a widely used distillation method for diffusion models. The fix is simple but effective: the DMD update is projected onto the orthogonal complement of the student-critic endpoint residual. The authors show that this residual is a conditionally unbiased estimate of critic endpoint error and analyze error removal and signal retention under additional assumptions. Experiments demonstrate improved training stability and video quality.

Critical strengths
  1. The proposed method is simple, intuitive yet effective.

  2. The claim that projecting the DMD update improves training stability is well supported by both the theoretical analysis and the synthetic experiments. The theory shows that a constant fraction of the critic error (under mild high-dimensional assumptions) necessarily lies along the student-critic endpoint residual direction and is therefore removed by the projection, while the controlled experiments in Figs. 4 and 5 are consistent with these predictions. Together they make PDMD's motivation and solution very convincing.

  3. The experimental design is good. Comprehensive experiments show PDMD's improved video quality, clear advantages in human preference, and the improvement in training stability (Fig. 10).

Critical weaknesses

Overall, I consider this a solid work and have no major concerns. My minor concerns are:

  1. The paper evaluates only benchmark scores and does not assess how the student's diversity differs from the teacher's. A supplementary analysis on diversity would strengthen the evaluation.
  2. In most cases, small metric advantages in automatic evaluation cannot be valided in human-evaluation win rates. PDMD exhibits the opposite pattern: on Wan2.1/VBench its metric margins over AnyFlow/DMD2 are marginal, while its advantages in the user study are obvious. I would like to hear the authors' thoughts on this discrepancy.

After reading the rebuttal, I think the authors have addressed my concerns, so I will raise my rating to 4.

Questions for authors

See "Critical Weaknesses" 2.

1 comment
AuthorsOpenReview ↗

Response to Reviewer jkw2

Thank you for your thoughtful review and for describing our method as “simple, intuitive yet effective.” We appreciate your assessment that the theoretical analysis and controlled experiments make PDMD's motivation and solution “very convincing,” and your overall judgment: “Overall, I consider this a solid work and have no major concerns.” Thank you also for rating both the paper's informativeness and presentation 3/3.

We warmly welcome further discussion and hope our responses encourage you to raise your score to 4.

We address your concerns below.

W1. Student diversity relative to the teacher

Four-step PDMD achieves 95.5% of the full-step teacher's LPIPS diversity score on Wan and 94.2% on H3. We evaluate all 944 Wan/VBench prompts and all 387 H3/VideoGen-Eval prompts, with five generation seeds per prompt. We compute LPIPS diversity among the five videos for each prompt and average over all prompts. Higher values indicate greater perceptual diversity.

MethodWan LPIPS ↑H3 LPIPS ↑
PDMD0.52340.4757
DMD0.51180.4670
DMD20.56390.4448
rCM0.49850.4802
AnyFlow0.45320.4272
Full-step teacher0.54820.5049
H3 4-step teacher–0.4329
H3 + Turbo LoRA–0.4469

PDMD ranks among the top two four-step methods in LPIPS diversity on both benchmarks, without a discriminator or multi-stage training. It outperforms its fair baseline DMD on both backbones, as well as the four-step teacher and Turbo LoRA on H3.

PDMD closely preserves the full-step teacher’s perceptual diversity on both backbones, reaching 95.5% of its LPIPS diversity score on Wan and 94.2% on H3. This consistent result shows that four-step distillation retains rich variation across seeds for the same prompt, alongside PDMD’s improvements over DMD in generation quality and diversity.

W2/Q1. Why the VBench and human-preference margins differ

We appreciate the reviewer’s thoughtful comments on video-quality evaluation. Reliable video-quality evaluation remains a challenging and rapidly evolving research area. Our interpretation is that aggregate metrics may respond less strongly to more natural colors and textures and better motion than human viewers do.

We therefore see automatic evaluation and the user studies as helpful complementary evidence. Both automatic evaluation and the user study consistently favor our method by different margins.

We would qualify the observation about small automatic-metric gains: clear human preferences can coexist with close benchmark scores. Recent peer-reviewed video-distillation studies report this pattern:

  • Self Forcing (NeurIPS 2025, Table 1 and Fig. 4): VBench totals of 84.31 versus 84.26 for Wan2.1 under their prompts and seeds, a 0.05-point gain, accompany a 62.7% overall human preference rate.
  • RAVEN (NeurIPS 2026, Table 1 and Fig. 6): VBench totals of 85.15 versus 84.96 for Causal Forcing under their prompts and seeds, a 0.19-point gain, accompany a 76% overall human preference rate.
  • APT (ICML 2025, Tables 4 and 7) further illustrates the complementary role of human evaluation. Its two-step model scores 81.85 versus 82.15 for the diffusion baseline on VBench under their prompts and seeds, yet achieves a +32.3-percentage-point net preference in visual fidelity.

Our human preferences are consistent with the more natural colors and textures in Figs. 6 and E5 and the motion comparison in Fig. E2. We warmly invite you to inspect the full videos under Benchmarks → Check qualitative comparisons side by side in the submitted supplementary website.

Thank you again for your thoughtful feedback. We welcome further discussion and sincerely hope you will consider raising your score to 4.

Reviewer_yFiyOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes Projected Distribution Matching Distillation (PDMD), a simple modification to DMD that removes the update component parallel to the student–critic endpoint residual. The authors show that this residual is an less biased estimate of critic endpoint error at a fixed noisy query, and derive error-removal and signal-retention results under additional assumptions. Experiments on 2D distributions, Wan2.1, and MiniMax-H3 show improved training stability and video quality, with additional gains in joint audio generation.

Critical strengths

This paper shows a simple, intuitive method with clear practical value. PDMD uses quantities already available in DMD and requires one line change in implementation. This makes it an appealing modification for existing distillation pipelines.

There are convincing evidence of improved training stability. I appreciate the discussions on training dynamics and related ablations.

Critical weaknesses

The paper does not directly evaluate video diversity across seeds and prompts, so it remains unclear whether the quality and stability improvements preserve sample diversity.

The paper compares selected frames, but the supp webpage did not side-by-side compare video results, making the motion-quality claims difficult to assess.

Questions for authors

NA

2 comments
AuthorsOpenReview ↗

Response to Reviewer yFiy

Thank you for your thoughtful review and for describing PDMD as a “simple, intuitive method with clear practical value” and “an appealing modification for existing distillation pipelines.” We also appreciate your recognition of the “convincing evidence of improved training stability” and your positive feedback on our training-dynamics analysis and ablations. Thank you also for rating both the paper's informativeness and presentation 3/3.

We warmly welcome further discussion and hope our responses encourage you to raise your score to 4.

We address your concerns below.

W1. Video diversity analysis

PDMD preserves strong sample diversity while improving video quality and training stability. We evaluate all 944 Wan/VBench prompts and all 387 H3/VideoGen-Eval prompts, with five generation seeds per prompt. We compute LPIPS diversity among the five videos for each prompt and average over all prompts. Higher values indicate greater perceptual diversity.

MethodWan LPIPS ↑H3 LPIPS ↑
PDMD0.52340.4757
DMD0.51180.4670
DMD20.56390.4448
rCM0.49850.4802
AnyFlow0.45320.4272
Full-step teacher0.54820.5049
H3 4-step teacher–0.4329
H3 + Turbo LoRA–0.4469

PDMD ranks among the top two four-step methods on both benchmarks without a discriminator or multi-stage training. It is worth highlighting that PDMD exceeds its fair baseline DMD on both backbones: 0.5234 vs. 0.5118 on Wan and 0.4757 vs. 0.4670 on H3. PDMD also achieves 95.5% and 94.2% of the full-step teacher's LPIPS diversity scores, respectively.

Measuring diversity within each prompt captures the variety of visual realizations a model can produce for the same request. PDMD’s gains over DMD on both backbones show that video quality, training stability, and sample diversity improve together.

W2. Where to find the side-by-side videos

The submitted supplementary website includes the required full-video side-by-side comparisons with synchronized playback. To see them, open pdmd-website-supp/index.html. Click Benchmarks in the top navigation bar, select H3 · VideoGen-Eval or Wan2.1 · VBench, and then click Check qualitative comparisons side by side above the results table. The resulting Qualitative Comparisons page also lets you switch between the two backbones. We will make this entry point more prominent so the full-video comparisons are easier to find.

Thank you again for your thoughtful feedback. We welcome further discussion and sincerely hope you will consider raising your score to 4.

Thank you again for your feedback! Is there anything else we could clarify? If your concerns have been addressed, would you consider raising your score?

Reviewer_Hpq5OpenReview ↗
Rating
3
Confidence
4
Informativeness
3
Presentation
2
Soundness
3
Read review
Paper summary

Projected Distribution Matching Distillation (PDMD) removes the DMD update component parallel to the residual between the critic's predicted clean endpoint and the student's generated endpoint. A conditional-mean identity connects this residual to critic endpoint error. Under additional alignment and concentration assumptions, the analysis establishes error removal with retention of the ideal update in high dimensions. Synthetic experiments and distillation of Wan2.1 and MiniMax-H3 test stability and generation quality. PDMD reuses existing predictions without adding a network or model evaluation.

Critical strengths
  • Projection along the endpoint residual gives an inexpensive intervention with a mathematical rationale. Appendix A separates the error-removal claim from the assumptions needed for signal retention and derives an exact decomposition of the projected update error. Section 2 also distinguishes this operation from projections used for guidance.
  • Controlled comparisons in Section 6 and Appendix D examine alternative directions, restoration of half the removed component, a larger student learning rate, and different critic update ratios. These controls make simple update shrinkage an unlikely sole explanation. Changing the direction matters in the reported experiments.
  • Across 20 seeds, the two-mode experiment reduces collapses from 14 to 3 at one learning rate and from 19 to 12 at the other. Appendix B can also measure critic-error and ideal-signal removal in a tractable setting. Together, these tests connect the proposed mechanism to observable training failures.
  • Video experiments extend the evidence to two backbones, with aggregate Wan2.1 scores of 83.73 for PDMD and 82.70 for DMD. MiniMax-H3 adds audio and synchronization measurements. Human comparisons favor PDMD over the compared four-step baselines on visual and motion quality, so the evidence is not confined to one automatic score.
Critical weaknesses
  • Checkpoint tracking and final evaluation share all 387 VideoGen-Eval prompts in Appendix C.2. Learning-rate and checkpoint selection on this benchmark makes the final advantage harder to interpret as performance on unseen evaluation cases. For MiniMax-H3, the aggregate gain over the best DMD result is 0.41 points. Selection may matter at this scale, although the manuscript does not establish the size of that effect. Clarifying the selection criterion is possible from existing records, and a revision should evaluate the selected checkpoints on prompts and seeds excluded from selection.
  • Preference comparisons in Appendix C use 20 annotators without specifying judgments per pair or reporting uncertainty. Large visual-preference margins and smaller audio-preference differences cannot be assessed with the same confidence from averages alone. I would like the annotation allocation and intervals based on shared prompts and repeated judgments, with variation across generated clips distinguished from variation across training runs.
  • Connecting the high-dimensional analysis to video training requires assumptions that the reported diagnostics do not verify. Proposition A4 requires weak alignment with critic error and posterior covariance, and Corollary A6 adds a signal-to-error scaling condition. Retained update norm and reduced decoded speckle do not establish those conditions, and the generator Jacobian and optimizer further transform the endpoint update. Appendix F already limits the theory's scope. Carry that qualification into the discussion of video stability by identifying which conclusions rest on empirical controls and which depend on the signal-retention assumptions.
Questions for authors

Which criterion selected each reported checkpoint, and were the same prompts and generation seeds used during selection and final scoring? Does the human evaluation also reuse the prompts involved in selection?

5 comments
AuthorsOpenReview ↗

Response to Reviewer Hpq5

Thank you for the careful review and for recognizing the projection's practical value, theoretical motivation, and controlled stability evidence. We appreciate your observation that the projection “gives an inexpensive intervention with a mathematical rationale.” We also appreciate that all three reviewers rated the paper's informativeness 3/3.

We warmly welcome further discussion and hope our responses encourage you to raise your score to 4.

Regarding weaknesses and questions, we use three separate comment messages as below.

AuthorsOpenReview ↗

W1/Q1. Checkpoint selection and evaluation

For each reimplemented method, we save and evaluate checkpoints every 500 iterations and select the one with the highest benchmark total. Wan uses all 944 VBench prompts averaged over seeds 0–4. H3 uses all 387 VideoGen-Eval prompts, identical prompt rewrites, and seed 42 alone. The original results in Tables 1 and 2 use the same prompts and seed settings as checkpoint selection.

Comparing checkpoints during training and reporting the best score is common practice in diffusion generation and distillation. DMD2 (NeurIPS 2024, Appendix F.2) resumes training from the checkpoint with the best FID. Diffusion-DRF (NeurIPS 2026, Section 4.1, Fig. 6) reports each model's best checkpoint alongside full training curves. AnyFlow (ECCV 2026) evaluates checkpoints on VBench during training. Reusing prompts and generation seeds for checkpoint comparison and final evaluation is also common practice. DMD2 (NeurIPS 2024) evaluates successive checkpoints on fixed COCO prompts and seeds. AnyFlow (ECCV 2026) uses the same VBench prompts and seeds for training-time and final evaluation.

We selected the best H3 checkpoints for each baseline for fair comparisons. They are listed below (Appendix C.2, Table C2).

MethodReported iterationTotal score
PDMD3,50083.17
DMD5,50082.76
DMD250082.27
rCM2,000 in stage 281.18
AnyFlow3,000 in stage 281.97

Training trajectories show stable gains across checkpoints (Fig. D3).

  • Every evaluated PDMD checkpoint after 2,000 iterations outperforms the best reported scores of prior methods.

  • The reported 3,500-iteration checkpoint maximizes the selection score, but the advantage is not confined to this checkpoint.

The H3 checkpoint selected using seed 42 consistently outperforms state-of-the-art methods on other seeds.

We select the H3 checkpoint using seed 42 alone. Here we further evaluate the same checkpoint on four additional seeds (0–3), with the same 387 prompts and prompt rewrites. PDMD ranks first in Total, Quality, and Dynamic on every seed.

Four additional seeds (0–3): mean video scores. Bold marks the highest score in each column.

MethodNFETotal ↑Quality ↑Dynamic ↑Semantic ↑
H3 teacher5082.1882.2668.6781.83
H3 (undistilled)479.7479.9348.0679.01
H3 + Turbo LoRA481.5181.3456.3382.22
rCM480.9280.9060.4081.01
AnyFlow481.7681.7164.7381.93
DMD482.3782.5763.1881.56
DMD2482.0082.2860.0180.88
PDMD482.8683.1570.8781.70

Across the four additional seeds, PDMD significantly outperforms all seven comparison models in total score (Holm-adjusted p ≤ 0.0286), with a +0.50-point gain over DMD (95% CI [+0.10, +0.89]).

Averaging all five seeds confirms this result: PDMD scores 82.92 versus DMD’s 82.45 and significantly outperforms all seven comparison models (Holm-adjusted p ≤ 0.0082).

Across the four additional seeds, PDMD also leads all four-step baselines in PQ, CE, CU, and IS on every seed and achieves the best mean DeSync.

Overall, our results show that PDMD's gains are robust to checkpoint selection. First, our checkpoint selection follows common practice in diffusion generation and distillation. Second, Fig. D3 shows that every evaluated PDMD checkpoint after 2,000 iterations outperforms the best reported baseline in total score. Third, the H3 checkpoint selected using seed 42 alone significantly outperforms all seven comparison models in total score across four additional seeds.

AuthorsOpenReview ↗

W2/Q2. Regarding human evaluation.

Our H3 user study covers 2,709 distinct video pairs across seven baselines. We evaluate all 387 VideoGen-Eval benchmark prompts, using the complete set of generated videos in Table 2 without selecting examples (Appendix C.2).

We assign each pair to one of 20 annotators without overlap. Their combined judgments reduce the influence of individual preferences. We match prompts and seeds within each pair and randomize left-right order.

Pairwise human evaluation is standard practice in peer-reviewed work, including Self Forcing (NeurIPS 2025), RAVEN (NeurIPS 2026), ADV (CVPR 2026), APT (ICML 2025), and DMD2 (NeurIPS 2024) (for images). Self Forcing and RAVEN also use one evaluator per video pair. Their protocols imply approximately 540–4,012 total pair evaluations per study. Our 2,709 evaluations are fully comparable in scale.

We added confidence intervals and significance tests using the win/tie/loss counts recovered from Table C4. Net preference is (wins minus losses) divided by 387, including ties.

We treat the 387 shared prompts as independent sampling units and compute a marginal 95% percentile-bootstrap interval for each baseline and question. We apply two-sided exact sign tests to non-tied judgments and Holm correction across all 35 comparisons. Holm correction allows dependence between tests that share prompts.

PDMD significantly outperforms all six four-step baselines in visual, motion, and audio preference after Holm correction (adjusted p ≤ 0.000154). The table reports net preference [95% interval] in percentage points.

BaselineVisual qualityMotion qualityAudio quality
DMD+25.1 [+18.9, +31.3]+12.1 [+7.0, +17.3]+10.9 [+6.5, +15.2]
DMD2+28.7 [+22.2, +35.1]+19.4 [+13.7, +25.1]+22.2 [+17.1, +27.4]
H3 Turbo LoRA+38.0 [+31.0, +44.7]+19.4 [+14.0, +24.8]+9.3 [+5.2, +13.4]
rCM+51.2 [+44.2, +57.9]+31.5 [+25.3, +37.7]+17.1 [+11.1, +23.0]
AnyFlow+66.4 [+60.5, +72.1]+27.9 [+22.2, +33.6]+25.8 [+20.7, +31.0]
Undistilled H3, 4 steps+72.6 [+66.9, +78.0]+44.7 [+38.8, +50.6]+47.5 [+42.1, +53.0]
H3 teacher, 50 steps-35.4 [-41.6, -29.2]-11.9 [-17.1, -6.7]-3.9 [-8.3, +0.5]

PDMD significantly outperforms DMD in both visual and audio quality. Net preference margins are +25.1 and +10.9 percentage points, respectively (Holm-adjusted p = 9.69 × 10⁻¹³ and 2.17 × 10⁻⁵).

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.