- Rating
- 1
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
This paper proposes SSR to selectively restore full FFN computation at important denoising steps in compressed diffusion language models, aiming to recover model performance with limited additional computation. Overall, the motivation is interesting, but the experimental evaluation is too limited to convincingly demonstrate the generality and robustness of SSR.
Critical strengths
- Interesting motivation. The motivation of this paper is interesting. It investigates whether selectively disabling compression at a small number of denoising steps and restoring the original full forward computation can recover most of the performance of the uncompressed model while using substantially less computation overall. This provides an interesting perspective on how computation should be allocated across different denoising steps in diffusion language models.
Critical weaknesses
The main issue is that the experiments are not sufficiently comprehensive to demonstrate the effectiveness and robustness of the proposed method.
-
Limited practicality of the compression method. Existing DLM compression methods such as Quant-dLLM have already been published, while the experiments in this paper are mainly based on a self-designed FFN compression method that is not compared with existing methods. Since SSR relies on an underlying compression method, the paper lacks evidence that SSR can generalize to mainstream DLM compression methods.
-
The experimental setting is relatively limited and simple. Mainstream DLM works often evaluate on reasoning tasks such as mathematics and coding with 256 or more denoising steps, while this paper only uses 32 steps on relatively simple commonsense tasks.
There are also some presentation issues, such as the confusing first column of Table 1 and overlapping numbers in Figure 3.
Overall, the motivation is interesting, but the current experiments are insufficient to convincingly demonstrate the generality and robustness of SSR.
Questions for authors
as in weaknesses