- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper develops CAFFEINATE, a closed-form correction of LayerNorm’s affine parameters, which is applied once to a pruned net work to restore the original per-channel output moments from only forward passes, without gradients or labels. It seems that this is a derivative work of REFLOW[1] in LN CNN.
Critical strengths
- This paper is well-written.
- The idea makes sense.
- The performance is very good, as shown in Table 1.
- I do not see any other similar work in the field of CNN pruning, nor does my agent. I would like to vote for acceptance because of the simplicity and effectiveness.
Critical weaknesses
- There are similar ideas in the field of quantization[1,2]. Scaling the activations is part of these works. The authors could add some discussion of these works.
[1] SmoothQuant: Xiao et al., "SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Models", ICML 2023.
[2] AWQ: Lin et al., "AWQ: Activation-aware Weight Quantization for LLM Compression and Acceleration", MLSys 2024.
Questions for authors
Please check the Critical Weaknesses.