- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper presents an intriguing empirical observation that the denoising dynamics typically distributed across multiple sampling steps may instead emerge across network depth in one-step generative models. The authors introduce the concept of depth as time, showing that progressive denoising can be observed across network layers by decoding intermediate representations with the model's output head. They further demonstrate that this behavior depends on the transport objective, with MeanFlow exhibiting denoising followed by renoising. Motivated by these observations, they propose a time-conditioned weight-sharing architecture that achieves substantial parameter compression while maintaining competitive generation quality.
Critical strengths
-
Novel and insightful empirical observation regarding the internal computation of one-step generative models. The central observation that denoising-like computation progressively emerges across network depth is interesting and potentially important.
-
Empirical analysis across different generative objectives. The comparison among time-indexed flow-map models and Drifting Models is valuable because it suggests that depthwise denoising may be associated with the training objective rather than simply being an inherent property of deep neural networks.
-
The work connects an interpretability finding to a concrete practical application. The reported 16.6× parameter compression of MeanFlow SiT-L/2, from approximately 459M to 27.6M parameters, is substantial.
Critical weaknesses
-
The authors attribute the observed denoise-then-renoise phenomenon to the interval-conditioned transport task learned by MeanFlow. However, the paper does not establish whether a conventional multi-step flow-matching model exhibits similar non-monotonic depthwise behavior, which is an important baseline.
-
The evidence does not conclusively establish that depthwise representations follow an actual denoising trajectory.
-
The compression results do not establish a clear computational advantage over conventional architectures. The experimental evaluation should report wall-clock latency, peak memory usage, FLOPs, and quality under matched inference compute budgets.
Questions for authors
Q1: How can the authors rule out generic hierarchical refinement as an alternative explanation?
Q2: If the depthwise denoising pattern is deliberately disrupted or strengthened during training, how does this affect the compressibility of the resulting model?