acceptodds
Withdrawn from ICLR 2027

TSD: Few-Step Video Diffusion with Temporally Decoupled Distribution Matching Distillation

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Reliable video generation is central to interactive applications and digital content creation. Although distribution matching distillation (DMD) enables few-step video generation, balancing visual fidelity with coherent motion remains challenging during prolonged training, and temporal variation may even collapse. We present Temporal Subspace Distillation (TSD), a general supervision framework designed to make better use of teacher knowledge during continued training. Our key idea is to decompose video latents into a temporal mean replicated across frames and zero-mean deviations. These components lie in orthogonal subspaces, enabling separate supervision of persistent visual content and temporal variation. Building on this decomposition, we introduce an identity anchor to transfer time-constant content from the teacher and one-sided deviation floors to penalize insufficient temporal variation in the student. We further extract motion saliency from the same teacher predictions to guide where variation is protected without prescribing a specific motion trajectory. Experiments show that TSD improves video quality over baseline methods, producing videos with high visual fidelity and coherent, plausible motion. TSD retains the student architecture and inference procedure and adds no inference cost.

Primary area:
generative models
Keywords:
video generation, distribution matching distillation, few-step video diffusion

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_K9SpOpenReview ↗
Rating
2
Confidence
3
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes Temporal Subspace Distillation (TSD) to reduce motion degradation in few-step video distillation. It separates generated videos into temporal means and deviations, adding appearance and motion constraints to DMD. Experiments on Wan2.1 show improvements in overall video quality and motion preservation over DMD and adapted AVD baselines.

Critical strengths
  1. Practical motivation. The paper addresses an important trade-off in video distillation: visual details can improve while motion deteriorates.
  2. Structured design with empirical support. Temporal decomposition enables separate appearance and motion constraints. Experiments across multiple seeds and initialization settings support quality improvements without additional inference steps.
Critical weaknesses
  1. Missing comparisons with relevant recent methods. The experiments mainly compare against DMD and adapted AVD baselines, without clearly establishing comparisons to Causal-rCM[1] or SGMD[2]. Causal-rCM offers a closely related causal few-step generation setting, while SGMD provides an alternative approach to video distillation optimization. Their absence limits evidence that TSD’s temporal decomposition and auxiliary losses offer advantages over recent methods. The relationship between CF++ and Causal-rCM also needs clarification.
  2. Insufficient justification of the auxiliary components. The ablations show mixed benefits: adding VF to IA+MF lowers both benchmark means in Table 2, and adding MSW to TSD-B does the same in Table 5. Although the paper highlights gains from joint use, it provides limited analysis of these interactions and trade-offs. The need for the full design over simpler variants therefore remains unclear.
Questions for authors
  1. Does CF++ incorporate Causal-rCM, and if so, which components? Could you provide comparisons with Causal-rCM and SGMD using matched backbones, inference steps, and comparable training budgets, or explain the constraints preventing a fair comparison?
  2. What explains the lower scores when adding VF to IA+MF or MSW to TSD-B? Could paired per-seed results and motion-specific metrics clarify whether these changes reflect training variability or trade-offs between appearance and motion?
  3. What supports choosing the full configurations over simpler alternatives? IA+MF scores higher than TSD-B on VBench-1, while TSD-B scores higher than TSD-D on VBench-2. Could you clarify the selection criteria and sensitivity to the auxiliary loss weights?

[1] Zheng K, He G, Zhao M, et al. Causal-rCM: A Unified Teacher-Forcing and Self-Forcing Open Recipe for Autoregressive Diffusion Distillation in Streaming Video Generation and Interactive World Models[J]. arXiv preprint arXiv:2606.25473, 2026.

[2] Wu Z, Gong R, Yong Y, et al. SGMD: Score Gradient Matching Distillation for Few-Step Video Diffusion Distillation[J]. arXiv preprint arXiv:2605.30116, 2026.

Reviewer_KbxuOpenReview ↗
Rating
2
Confidence
4
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

The paper Temporal Subspace Distillation (TSD) addresses motion degradation during continued distribution matching distillation of video generators. It splits a latent clip into a replicated temporal mean and zero-mean deviations. A teacher anchor supervises the mean, while one-sided variance and temporal-change penalties protect variation and a smoothing term discourages rapid fluctuations. TSD-D adds teacher-derived spatial saliency and shared reference statistics across training ranks. The distillation and autoregressive recipe build on DMD2, Self Forcing, and Causal Forcing. Experiments distill a Wan2.1 (14B) teacher into a (1.3B) causal student generating 81-frame, 480p videos in four steps. The paper reports three training seeds, two VBench versions, ablations, alternative initialization, and two-step generation.

Critical strengths

TSD addresses a meaningful failure mode. Its temporal-mean and zero-mean projections define an apt regularization strategy.

The mathematical scope is careful. The auxiliary latent-output gradient identities are correct, and the method retains joint video-level DMD. It does not incorrectly claim that matching independent appearance and motion marginals is sufficient to match the joint video distribution.

The late-training comparison is substantial. Every reported run of either TSD variant exceeds every 3K DMD run. The individual-seed table reproduces the reported means and sample deviations. Retaining the student architecture and inference procedure seems an additional practical advantage.

Critical weaknesses

The benefit of continued training is not established against an earlier selected checkpoint. Tab 1 shows that TSD mitigates deterioration of the particular DMD baseline at 3K iterations. Nevertheless, DMD already scores 86.43±0.6186.43 \pm 0.61 on VBench-1 at 1K, compared with 86.86±0.4886.86 \pm 0.48 for TSD-D and 84.97±1.1384.97 \pm 1.13 for TSD-B at 3K. TSD variants themselves score higher at 1K. Their dynamic degree also falls. In the separate-cluster experiment, TSD-B’s short-prompt dynamic degree falls from 1.00 to 0.41 and median flow from 11.2 to 4.8 pixels (D.2, Tab 9).

Reduced displacement does not establish worse motion coherence…it could accompany better action quality. That is the distinction the paper needs to measure directly.

VBench-2 supplies relevant dimensions, however TSD-D’s 3K aggregate gain over DMD is minor against standard deviations, and its multi-view consistency is 27.7 versus 38.0 for DMD (Tab 8). TSD-B has stronger gains in some physical-quality dimensions, so the evidence indicates a tradeoff. Selected still frames cannot settle it. In my view, comparing complete clips with identical prompts and sampling seeds against a validation-selected early DMD checkpoint may show whether continued training improves the joint visual-quality and motion result claimed in Sec. 4.2.

The particular benefit of the temporal-mean projection is not isolated. The new question is whether projecting teacher regression onto the temporal mean improves the tradeoff compared with regression to the same full-clip teacher prediction. That matched control is missing.

The proven orthogonality concerns output-space gradients. It does not imply semantic appearance-motion separation, i.e., the temporal mean depends on object occupancy and camera motion, and the generator’s shared parameters can couple the two subspaces…. A matched full-clip regression control would establish the value of the proposed projection without requiring a claim of superiority over the complete AVD system.

Questions for authors

Under a held-out checkpoint-selection rule and identical prompts and sampling seeds, does TSD outperform an earlier selected DMD checkpoint on visual quality and coherent, prompt-specified motion? Please report paired complete-clip judgments or validated motion/consistency measurements, together with training GPU-hours.

What changes when the identity anchor is replaced by full-clip regression to the same detached teacher prediction, with all other losses and the tuning budget held fixed? For any full-AVD comparison, please identify its additional video-data access.

Was the optional pixel-flow floor active in any reported experiment? Eq.11 includes its weight, but the configurations do not specify it or its sampling parameters. Please also identify the VBench-2 evaluator or model versions and the main-benchmark sampling counts.

Reviewer_stmzOpenReview ↗
Rating
2
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

TSD augments distribution matching distillation with supervision on two temporal components. It anchors the student's temporal mean to a teacher prediction and applies variance and velocity floors to the deviations, together with temporal smoothing. TSD-D adds teacher motion saliency and a shared velocity reference. These objectives affect training while leaving student inference unchanged.

Critical strengths
  1. The degradation under continued DMD training is supported by repeated runs. DMD's VBench-1 score falls from 86.43 at 1K iterations to 80.82 at 3K. Under the same teacher, initialization, and prompts, every reported TSD run at 3K exceeds every DMD run at 3K on this metric. The three-seed results support a real benefit in the tested setting.
  2. The supervision is inexpensive to deploy. Mean anchoring reuses an existing teacher prediction, and the auxiliary losses add no inference cost. The distinction between mean and deviation gradients makes the design interpretable. The paper also acknowledges that latent variation alone does not establish plausible motion.
  3. The experiments test more than one training configuration. Component ablations examine the auxiliary losses, and Table 3 reports improvements over DMD with CF++ initialization at both two and four sampling steps. These results strengthen the case for TSD as a useful training recipe within the tested model family.
Critical weaknesses
  1. The central choice of mean anchoring lacks a direct control. Table 2 tests adding mean anchoring, but does not compare it with regressing the complete teacher latent prediction under the same auxiliary losses and a comparable loss scale. Both AVD adaptations omit real-video regression, as Appendix A explains. They therefore do not resolve whether restricting teacher supervision to the mean is responsible for the improvement.
  2. The benefit of continuing training remains unclear. TSD already improves over DMD at 1K, which is a positive result. However, TSD-B's VBench-1 score falls from 87.00 to 84.97 by 3K while VBench-2 rises, and TSD-D does not improve either aggregate score over its own 1K checkpoint. Appendix E argues that later checkpoints look better. A blinded comparison of the existing checkpoints would provide stronger evidence for that claim.
  3. Protecting variation does not yet establish better motion with preserved content. The variance threshold is shared across prompts, and static teacher predictions do not disable the floor. Behavior on prompts requiring little movement is therefore important. Table 8 also shows TSD-B below DMD on Human Identity and Instance Preservation. These tradeoffs warrant a more specific assessment of action coherence and unwanted changes, rather than relying on aggregate scores.
Questions for authors
  1. How does mean anchoring compare with full latent anchoring when the teacher target, remaining losses, training budget, and regression scale are controlled?
  2. Do viewers prefer the 3K outputs to the corresponding 1K outputs for both appearance and action coherence? Please include prompts requiring little movement.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.