- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper switches between quantized and full-precision inference at different generation steps, using prediction uncertainty and changes in entropy. The aim is to recover generation quality while retaining much of the benefit of quantization. Experiments mainly use LLaDA-8B, with a supplementary Dream result and comparisons against static precision schedules.
Critical strengths
The method recovers much of the quality lost through quantization. On GSM8K, accuracy increases from 69.8 to 74.7, close to the FP16 result of 74.8. The comparison with several static precision schedules supports studying when full-precision computation is needed.
Critical weaknesses
-
The reported 1.66× speedup is estimated rather than measured end to end. The paper does not establish the actual cost of keeping both models in memory, switching precision, or performing additional steps. It also lacks results on a competitive modern dLLM, so the practical advantage remains unproven.
-
The ablations do not match the amount of full-precision computation or the number of additional steps. The quality recovery could therefore come from spending more computation rather than from the proposed selection signals. This prevents a clear assessment of the value of tracking changes in entropy.
Questions for authors
-
With actual low-precision kernels on SDAR or other competitive dLLMs, what are the measured latency, throughput, and peak memory at matched quality, including switching and additional steps?
-
Does the proposed method outperform the best static schedule when the full-precision budget and total computation are matched? Does tracking entropy changes still help under this comparison?
-
Is the entropy window measured in generation steps or committed tokens, and which definition was used for the reported experiments?