- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper proposes Layerwise Closed-Form initialization (LCF) for diffusion denoisers, using linear probes, residual-augmented targets, and layerwise least-squares updates. The authors evaluate initialization quality and training efficiency across multiple datasets and architectures. LCF consistently improves initial validation loss, while positive net compute savings are mainly observed for small convolutional denoisers.
Critical strengths
1.The paper addresses a relevant problem and evaluates initialization cost rather than relying solely on initial loss. The FLOP-adjusted steps-to-target metric is a useful aspect of the evaluation.
2.The authors include competitive initialization baselines, a zero-initialized output-head control, and longer convergence experiments. The negative results on other architectures help clarify the method's limitations.
Critical weaknesses
1.Firstly, the source of the improvement is not clear. In Table 4, when the residual strength α is set to zero, the earlier calculation results are exactly the same. This makes me question the necessity of the proposed residual detection mechanism. Moreover, the paper fails to clearly distinguish the contributions from the output head fitting, the update of the hidden layer, and the scaling of the weight.
2.Then, the claim about efficiency lacks sufficient support. LCF and the main baseline use different initial weight distributions, making the comparison difficult to explain. The positive computational results mainly focus on small-scale diffusion CNNs, while the results of U-Net are negative. The reported savings effect is based on the estimated number of floating-point operations (FLOPs), rather than the actual measured training time. It is still unclear whether this method has practical advantages under controlled comparison conditions.
Questions for authors
1.Can the authors report results using identical initial weight tensors for LCF and its baselines, together with separate ablations of the residual probe, hidden-layer updates, and output-head fitting?
2.Does LCF reduce actual end-to-end GPU training time, including initialization overhead? Can the authors provide this comparison for at least one standard diffusion U-Net?