- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes DMPoT, a weight quantization method that adjusts the allocation of logarithmic levels under hardware constraints. Its deployed 4-bit grid allows a shared 3x computation at the array input, followed by selection, shifts, and accumulation in each processing element. The method uses learned rotation and smoothing with GPTQ calibration, while activations and the KV cache use uniform INT. Experiments on seven models report accuracy close to uniform INT and lower perplexity than FPoT. The hardware evaluation reports smaller synthesized MAC-tile area.
Critical strengths
The grid supports sharing computation across output channels. For the 4-bit design, 3x is computed once per input lane and reused across the tile. This provides a concrete way to reduce repeated arithmetic (Sec. 5; App. E).
The reported accuracy is close to uniform INT. The main results cover seven models and two quantization configurations, with three calibration seeds. DMPoT has lower perplexity than FPoT in all fourteen comparisons and similar perplexity to APoT (Table 1).
The ablations help explain the design choices. The paper examines preprocessing, rounding, grid approximation, and calibration settings. It also evaluates different combinations of preprocessing and deployed grids, which helps distinguish their effects (App. H).
Critical weaknesses
The theoretical and format novelty is limited. The point-density analysis follows existing quantization theory, the deployed 4-bit values match scaled E2M1, and calibration and grid alignment are inherited from FPoT. The main contribution is the grid selection under hardware constraints and its hardware implementation.
Table 1 should include a full-precision baseline and comparisons with other INT quantization methods. BF16 accuracy is reported in the appendix, but the main table should include the full-precision accuracy and perplexity to make the absolute quantization loss clear. Most comparisons use the same preprocessing and GPTQ pipeline, so the comparison with other INT quantization approaches remains limited.
Synthesis area alone is insufficient to establish the implemented area savings. The paper reports Yosys mapped-cell area without timing constraints, placement and routing, or clock-tree synthesis. The area comparison should use the die area obtained after completing the full physical design flow. The current results do not establish whether the savings in Figures 3(a) and 3(c) will hold after physical implementation.
The proposed ASIC design lacks implementation-level validation. The hardware evaluation stops at tile synthesis. There are no post-layout simulation results, silicon measurements, or FPGA results. Practical power, performance, and area (PPA), end-to-end numerical accuracy, and inference throughput are therefore not validated in an implemented system.
The area savings are sensitive to the INT baseline. With the Booth INT baseline reported in the appendix, the W4A4/W4A8 savings decrease from 29.8%/38.2% to approximately 17.2%/25.4%. This dependence should be made clearer in the main results.
Questions for authors
Please address weaknesses above.
Also, were the accuracy and perplexity results in Table 1 obtained entirely through PyTorch inference, or was inference evaluated using the synthesized netlist? Please clarify how the hardware's numerical behavior was checked against the software evaluation.
Thanks for the detailed follow-up. The Q2 experiment — running all 19 b=4 schedules end-to-end under GPTQ + KL — is exactly the evidence I asked for, and the Spearman 0.99 agreement between the proxy and deployed ordering resolves my central concern that the theory might be lack of predictive power. The post-route data in Q1 also addresses my earlier point that the hardware evidence was gate-level only.
The practical case nevertheless remains narrower than the framing: the theory predicts a schedule that a cheap search would find anyway, and that schedule ties APoT on accuracy; on ASAP7, which is the relative more advanced node, the throughput advantage is 0.99×, showing that the advantage does not hold; and the near-lossless claim remains for the standard attention transformer models, which may not be applicable for future hybrid attention models (at least the author did not explain). Therefore I will maintain my score.