- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper studies INT8 rollout generation for reinforcement learning post-training on Ascend hardware. It presents a framework that combines quantization-aware training, jointly learned smoothing factors and weight quantization scales, reproducible hash-based rounding, and selective mixed-precision fallback to mitigate training–rollout numerical mismatch and quantization-induced performance degradation. The framework is evaluated on openPangu-7B and Qwen3-30B MoE using GSM8K and DAPO-Math-17K, with component ablations examining the proposed techniques. The results show reward recovery relative to the INT8 baselines. The paper also measures rollout latency, reporting a 19.7% reduction in per-token decoding latency on openPangu-7B.
Critical strengths
The paper clearly explains the challenges of INT8 rollouts in RL, particularly activation outliers and training–rollout numerical mismatch, and presents a practical combination of techniques to address them.
Critical weaknesses
-
Adapting established quantization techniques to RL on Ascend is of practical interest. However, the additional methodological or system-level contributions beyond this platform adaptation are not sufficiently demonstrated in the current paper.
-
Identically labeled configurations have inconsistent results across tables: the openPangu-7B SR-only reward is 0.543 in Table 1 but 0.550 in Table A2. This unresolved difference equals both the WS gain and the combined method’s advantage over BF16 on GSM8K. Without reconciling these results and providing repeated runs with uncertainty estimates, the reliability of these small gains remains unclear.
-
QAT and learnable quantization parameters introduce training overhead, yet the efficiency evaluation mainly measures rollout latency. The step-time experiment disables training-side fake quantization, so it does not establish whether rollout savings outweigh the complete method’s additional training cost.
Questions for authors
-
What causes the discrepancies between identically labeled configurations in the main tables and appendix? Could you reconcile these results and provide multi-seed means and standard deviations to assess whether the small reported gains are reliable?
-
Could you provide a training-time breakdown showing the overhead introduced by each proposed component, particularly fake quantization, learnable smoothing and weight scales, and hash-based rounding, both individually and in combination?