- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
ScaleSweep searches neighboring FP8 E4M3 scales for NVFP4 blocks using MSE or importance-weighted MSE. It derives a block-size-16 MSE window of [-3, 7] and uses an empirical WMSE window of [-8, 7]. Experiments cover four Llama/Qwen models with RTN and GPTQ, from weight/activation quantization through additional KV-cache and query quantization, plus native Blackwell kernel and throughput measurements.
Critical strengths
-
The MSE window has a concrete basis: a grid-containment upper bound and a lower bound backed by finite convex certificates (Section 4.2.2; Appendix B). This explains why a short search can replace full FP8 enumeration.
-
The accuracy gains come with usable throughput. For Llama-3.1-8B with GPTQ, WAKVQ average accuracy rises from 66.09 with 4/6 to 67.35 with ScaleSweep (Table 3). Restricted WMSE and MSE retain about 93.9% and 99.5% of default throughput, well above full search (Table 4).
Critical weaknesses
-
The closest methods are neither adequately distinguished nor compared. Section 2, lines 124-125, incorrectly limits ScaleSearch to attention inputs: ScaleSearch already searches neighboring FP8 bit patterns for weight/activation PTQ and selects scales by block MSE. H-Scale, Sections 3.2-3.3, uses the same diagonal activation-energy objective and a 16-candidate weight-scale search. Comparisons only with AbsMax and 4/6 leave the value of ScaleSweep's window and broader WMSE use unclear.
-
The optimality claim is broader than the analysis supports. The abstract and conclusion claim preservation of the optimal candidate under both objectives, but Section 4.2.2 acknowledges that the optimal WMSE scale can be arbitrarily small. Thus [-8, 7] can exclude it. Figure 3's downstream comparison on one model does not quantify how often this happens or the objective gap. Since WMSE is the default, this distinction matters. The Q/K/V importance weights also need a clearer connection to attention-output error.
Questions for authors
-
How does ScaleSweep compare with ScaleSearch and H-Scale under matched global scales, calibration, PTQ backend, and candidate budgets? Which gains come from the proved MSE window or from extending WMSE beyond weights?
-
Is optimal-candidate preservation intended only for MSE? For WMSE, how often does [-8, 7] exclude the optimum, and what is the objective gap for weights, activations, and Q/K/V? What attention-output approximation justifies the Q/K/V weights, and is Q/K importance calibrated or computed online?