- Rating
- 3(+1)
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper studies quantization at the level of these reused transition operators, including how transition perturbation, blockwise integer transition rules, and recursion depth and reuse modulate quantizability. Experiments on Sudoku and Maze show the effectiveness of the proposed method.
Critical strengths
Quantization is useful for neural networks, and this paper studies quantization at multiple levels.
Critical weaknesses
- The effectiveness of the proposed method is not sufficiently verified. For example, how about the performance with different backbones?
- Quantization has been studied for several years. Due to the lack of comparisons with the SOTA methods, the advantages of the proposed method are not well demonstrated.
- As discussed in the limitations. Quantization is general, and it can be applied to many tasks, such as LLMs and vision tasks. Is the proposed method effective on those tasks?
Questions for authors
Due to the lack of a deep discussion of recent methods, the advantages of the proposed method are not well demonstrated. See weaknesses.
3 comments
Thanks for the feedback. After reading the response and other comments. I think my main concerns about the backbones and comparisons are addressed. I would like to raise the rating.
Thank you for your review
Dear reviewer Sk9n, thank you for reading the response and the discussion, and for the updated rating.
Dear reviewer, thank you for your review. The revised PDF is uploaded with these replies.
W1, backbones. Four recursive architectures now, with HRM new on its public checkpoints: 55.1% in full precision, 0.0% with per-tensor 4-bit, 53.8% with MXINT4. On HRM Maze: 74.2%, 0.0 and 74.3.
W2 / Q1, state of the art. The contribution is the comparison across four recursive architectures: which standard fixes recover exact accuracy, and what accuracy is traded for a shared-exponent integer format.
Float scales of the same granularity are 1.0 to 4.2 points better than MXINT4. That is the measured price of the power-of-two rule, which buys integer elements with one shared exponent per block. We claim no accuracy advantage for MXINT4. The other models and methods are in the general response, part 1.
GPTQ and AWQ act on the weights, which are the small term here: 4-bit weights with per-channel scales cost 1.0 point on Sudoku (one seed), while the activation scale alone moves accuracy between 1% and 82%. We therefore compared against methods that act on the activations and did not run those two. LoopQ is concurrent work on looped language models that adds loop-aware adapters and trajectory-aware calibration. We study post-training quantization without adapters on exact-solution tasks, and Sec. 2 states the difference.
The format family runs on a shipping accelerator. On a Tenstorrent Blackhole card, 4-bit weights are lossless and 3.55x smaller, with 4.9% higher device throughput than bf16 (general response, part 2).
W3, scope. Our claim is for models that reuse one small block many times per forward pass (7M parameters for TRM, 27M for HRM), which is what makes them candidates for memory-constrained inference. We make no claim for feed-forward LLMs or vision models. For those, 4-bit microscaling and rotation are covered by the work cited in Sec. 2. The language-model case with our structure is the looped transformer, which LoopQ studies.