- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper builds FReD on top of the distribution-matching distillation framework of DFD (Data-Forcing Distillation, Chen et al. 2026). Using a pair of non-learnable complementary Gaussian masks, it splits the student's full latent prediction — together with the predictions of the frozen teacher and the fake-score model — into a low-frequency and a high-frequency path in the spatial frequency domain, and constructs a separate residual loss for each. A shared exponential weight, driven by the degree of overlap between the two masks, then down-weights the loss contribution of the transition band. The method only changes how the loss is computed at student-update time: the sampler, the number of denoising steps, and inference cost are entirely unchanged.
Validation covers text-to-video, autoregressive video generation, and image-to-video, all with 1.3B-scale Wan models, compared against DMD2, DP-DMD, and DFD. The paper reports six VBench quality metrics, visual and camera-pose diversity, training and inference time, and three ablations — overlap weighting, band composition, and mask bandwidth.
Critical strengths
S1. The paper is well organized with clear demonstration of the motivaiton. The gap between reference and generated frames is first quantified in the frequency domain, showing that preserving magnitude and energy is not sufficient to preserve decoded appearance. Every design choice at the decomposition end has a corresponding ablation. The validation end covers three tasks and three ablation groups.
S2. The proposed method yielded modest improvements without much extra overhead. Under the same checkpoint and the same 40-step budget, continued DFD training does not move at all (86.46 → 86.44) while this paper reaches 87.23, with Imaging Quality +2.42 while all three baselines sit near 68 (Table 5). Inference time is unchanged, and only 5 lines of code replace DFD's 2. The appendix also volunteers two restrictive derivations.
Critical weaknesses
W1. Novelty is insufficient: band splitting and frequency reweighting are both precedented, and a near-concurrent, more complete version exists Designing separate losses per frequency band is routine — SDKD (arXiv 2507.02939, 2025) decouples teacher latents into high/low bands and distills them separately; an AAAI 2026 cross-modal distillation work decomposes features with FFT high/low-pass masks, using MSE on low frequencies and LogMSE on high. Frequency reweighting in diffusion distillation also has a 2023 precedent that this paper misses: CVPR 2023 Diffusion Probabilistic Model Made Slim (arXiv 2211.17106) applies a DFT to teacher and student predictions and weights by a negative power of spectral amplitude to amplify the high-frequency loss, motivated by exactly the same "high frequencies are missing" argument. More importantly, the paper almost fully overlaps with SAP-DMD (arXiv 2609.29116, 2026-09-24): same quantity acted upon, same per-frequency reweighting, same zero inference overhead — two instances of one idea, differing only in that SAP-DMD is adaptive while this paper is fixed. This paper's PDF is dated two days later, so I do not count it as a missed citation, but this paper is the weaker of the two, and their conclusions conflict: SAP-DMD suppresses low-frequency dominance, whereas this paper leaves low frequencies entirely unsuppressed.
W2. Soft-Overlap is a heuristic trick with no theoretical grounding By the paper's own two appendix identities, its gradient equals the DFD gradient times a coefficient that is strictly positive and never exceeds 1 — no new information, no change of direction. Under this premise, SO merely picks the shape of one scaling function. Its justification is only three design constraints (equals 1 when there is no overlap, decays monotonically with overlap, invariant to swapping the two masks), which admit infinitely many solutions; no derivation is given for this particular form, and the decay exponent is tested only at 0 and 1. No objective function that SO optimizes is stated anywhere, nor any optimality argument. Worse, it points the wrong way: the overlap region is in fact under-counted (about 0.5×), so restoring it would require the reciprocal (about 2), but SO takes 0.5 and pushes the coefficient down to 0.25.
Questions for authors
Q1 (method stability assessment) The three gains (+0.77 / +0.47 / +0.43) currently have no variance estimate at all — all ablations share one training seed, and the appendix states that generation seeds do not constitute an independent run. Please report mean ± std over at least 3 independent training runs, with significance tests for the text-to-video average score and for Imaging Quality (+2.42); and explain why metrics degrade monotonically with training steps (Imaging 70.78@40 → 68.98@100), plus the basis for reporting the 40-step point. If the favorable window is as narrow as 8–20 student updates, how is stability guaranteed?
Q2 (positioning against baselines) Please add at least one non-DMD-family 2026 baseline to position this work relative to the current SOTA. The related work already lists the following candidates, yet none of them is compared anywhere: rCM (Zheng et al., ICLR 2026, pp. 2582–2603), TMD (Nie et al., CVPR 2026, pp. 4645–4655), Causal-rCM (Zheng et al., arXiv:2606.25473), and One-Forcing (Feng et al., arXiv:2605.23458). Note that rCM and TMD are already formally published at ICLR and CVPR 2026 respectively, and neither appears in any comparison in this paper, so a reader has no way to judge where the work actually stands.
2 comments
Thank you again for your detailed assessment and constructive suggestions. Our response clarifies the relationship to prior work and the role of soft-overlap weighting, and provides results from independent training runs together with additional baseline comparisons. We would be grateful for your feedback on these explanations and results, particularly whether any of your original concerns require further clarification. We would be happy to address any remaining questions.
We thank the reviewer for the careful assessment and constructive suggestions.
1. Contribution and related work (W1)
Our motivation was DFD-generated videos with hazy backgrounds, poor color rendition and unstable fine details. Section 3 diagnoses these failures through spatial-spectral analysis and latent interventions. Building on frequency-aware supervision in SDKD, FD-CMKD and Diffusion Probabilistic Model Made Slim, our contribution is a diagnosis of DFD-based video distillation and explicit control of its guidance allocation, evaluated across T2V, AR and I2V.
SAP-DMD addresses dominant low-frequency amplitudes in the DMD directional field using sample- and channel-adaptive modulation. We examine preservation across both frequency bands relative to reference videos. FReD uses a fixed frequency profile defined by complementary masks, band residual losses and overlap weighting. Both updates have a spectral-preconditioning interpretation; FReD focuses on guidance allocation across both frequency bands in DFD-based video distillation.
2. Soft-overlap weighting (W2)
SO attenuates the transition region shared by both loss paths. Its student-output gradient is
∇gLSO=N−1F−1[KF(d)],K=W(ML2+MH2).
Equation (10) defines the weighted residual objective:
LSO=21∑b∈{L,H}meanB,C,T,ω[W∣F(rb)∣2].
We use the overlap-dependent attenuation profile W=2−γO, O=4MLMH, with γ=1, evaluated empirically in Table 4. Positive frequency-dependent weights preserve individual Fourier phases but can change the complete gradient's direction, redistributing existing guidance. At the crossover, SO reduces the coefficient from 1/2 to 1/4. Recovering DFD instead requires W=1/(ML2+MH2), which cancels the reweighting.
At 40 continuation steps, SO reduces DD from 67.61% (768/1136) to 66.20% (752/1136), a 1.41-point decrease, while TF/MS remain comparable. Table 4 reports the accompanying quality gains.
3. Independent training runs and training length (Q1)
FReD is reported as mean ± sample SD over three independent training runs. DFD uses its officially released checkpoint as a fixed reference. Avg. is the mean of SC, BC, AQ, IQ, TF and MS; DD is separate.
For T2V Avg., the mean gain over fixed DFD is 0.783: a two-sided one-sample t-test gives t(2)=11.208, p=0.00787, and a 95% CI of [0.482, 1.084] for the gain. FReD's IQ runs are 70.78, 69.63 and 69.82 (main-run gain: 2.42), giving 70.077±0.616 versus fixed DFD 68.36: mean gain 1.717, t(2)=4.823, p=0.0404, CI [0.185, 3.248]. These tests condition on fixed DFD scores, assume approximately Gaussian FReD run scores, and use unadjusted p-values (n=3).
The updated DD results are below; parentheses give dynamic/total videos. The T2V FReD entry is the 40-step main result.
FReD improves DD over DFD by 3.87/1.76/1.58 percentage points, computed from unrounded counts. This full-method comparison is distinct from the SO ablation.
We chose 40 continuation steps empirically. With SO, IQ at 40/60/80/100 steps is 70.775/70.797/69.803/68.982 (Table 4). We hypothesize that the inherited DFD query mismatch, where fake scores use noised student predictions and teacher scores use noised real data, contributes to less reliable guidance during continued refinement.
4. Additional baselines (Q2)
The four added baselines use 500 training iterations. FReD rows are main-experiment references; Avg. excludes DD.
FReD has the highest T2V Avg. in this comparison. In AR, Causal-rCM has a slightly higher Avg. (84.66 vs. 84.51), while FReD has higher AQ and DD, showing metric-dependent trade-offs.