- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes NoiseFlow, a video generation framework that learns diffusion-timestep-dependent temporal noise correlations from a frozen pretrained denoiser. A warp network is trained using teacher-derived sensitivity signals, and a stochastic transport process constructs correlated noise while preserving standard-Gaussian frame marginals. A student denoiser is then fine-tuned on these noise priors and used to guide the base model during inference, without requiring a reference video or external optical flow. Experiments based on CogVideoX-2B demonstrate improvements in FVD and several temporal metrics, alongside trade-offs in dynamic content and instance preservation.
Critical strengths
- An interesting approach to learning temporal noise structure. Learning noise-space transport from the denoiser provides a meaningful alternative to prescribing correspondences from pixel-space optical flow. The formulation directly connects the temporal prior to the generative model and supports generation without reference videos at inference.
- A statistically motivated construction with measurable empirical benefits. The transport process is supported by an analysis of Gaussian marginal preservation, addressing an important issue when warping noise. The full method also improves FVD over both the original backbone and the no-warp self-guidance baseline, suggesting practical value beyond ordinary fine-tuning or the simplest self-guidance configuration.
Critical weaknesses
- The evaluation is limited to CogVideoX-2B. While the authors acknowledge the model-specific nature of the learned prior, further evidence would be useful to establish whether the framework provides consistent benefits when trained on other backbones.
Questions for authors
- Is student adaptation necessary? Could the authors evaluate retaining the learned warp while using the frozen teacher, rather than the trained student, for the second denoising prediction under the same guidance rule? The existing self-guidance ablation uses i.i.d. noise and therefore does not test this configuration. This comparison would clarify whether student training is essential to benefiting from the learned noise prior.
- What motivates the warp supervision design? Could the authors explain the intuition behind the magnitude target in Eq. (15)? I am also curious whether directly regressing a complete warp field, using a calibrated target derived from the same teacher signals, would perform similarly to separately supervising shape and magnitude.