- Rating
- 1
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper proposes RECAP, the integer-based W4A4KV4 post-training quantization (PTQ) framework for reasoning models. This paper observes that the existing PTQ methods fail to capture the historical KV cache’s information while calibrating the later tokens in long-reasoning trajectories. To address this, RECAP proposes dynamic prefix calibration, which regenerates long KV caches with the current quantizer while reconstructing only a short recent blocks. Also, RECAP employs Prefix Read Geometry Loss which weights historical key and value errors differently. Experiments on DeepSeek-R1-Distill-Qwen 1.5 and 7B, Qwen3-4B, the proposed method shows clear improvement over the FlatQuant.
Critical strengths
- Beyond the Microscaled FP4 (e.g., NVFP4), developing integer-quantized W4A4KV4 reasoning model is an important direction for the edge devices. I encourage authors to include this key motivation why exploring integer-based W4A4KV4 is important.
- The proposed method shows clear improvement over FlatQuant, the main PTQ baseline.
- The paper is easy to follow, and the figure and diagrams are intuitive.
- Ablation studies including the impact of the Read Geometry Penalty helps to understand of the individual role of each calibration objective terms.
- While the proposed method using calibration with NuminaMath, it improves the other domain performances such as GPQA or LCB.
Critical weaknesses
- The critical weakness is the fairness of the comparison with FlatQuant. This paper simply adopt the accuracy from other paper. The calibration dataset, hyperparameters, and other settings are whole different. To isolate the gain from the proposed calibration objective, we highly recommend to re-run the baseline and fill in the step-by-step gap. If the main advantages are come from the calibration dataset (e.g., simple Wikitext-2 vs. FP model’s reasoning trajectory), the novelty will largely disappears.
- I think that the proposed observation and method are not limited to long-decoding-based reasoning models. There can be similar issue in long-context task such as long document QA or summarization. Adding results on long-context benchmark such as LongBench would be beneficial.
- Calibration overhead over the PTQ baseline should be discussed. It seems that the proposed method requires longer tensor during training, so it would be helpful to report the peak memory and total calibration time, compared to FlatQuant.
- For generalization, adding the recent reasoning models (e.g., Qwen3.5) and larger models would be beneficial.
- The proposed calibration objective can be extended to other general PTQ methods. Is there any specific reason to employ this object to FlatQuant. If not, can you add the results on the other method, such as GPTQ in weight-only quantization?
- It would be helpful to add ablation study on W4A4KV16 and W16A16KV4.
Questions for authors
Please see weaknesses.