- Rating
- 2
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper proposes dynamic sensitivity-selective quantization (DYNSENS). It periodically changes which modules remain in BF16 during low-precision LLM RL training. The main contribution is adaptive precision allocation. This paper claims that the quantization sensitivity o modules changes during RL training, making a protection set is not enough. Experimental results show that it improves the training stability, compared with the baseline. The authors test the method with BF16 training with fake-QAT. Experiments uses Qwen3-8B, DAPO-17K on 8 H100 GPUs.
Critical strengths
-
The problem of sensitivity changing across modules is a clear motivation.
-
The method is concrete and can complement weight adaptation through QAT. Ranking benefit per protected parameter accounts for different module sizes.
-
The paper reports results of several seeds, initialization situations, late instability, and implementation details. These are very helpful to understand the method and performance in details.
Critical weaknesses
-
There is no final performance on benchmarks comparisons. The authors should at least finish training and testing a 7-8B size LLM and report the results with comparison to full-precision results and other quantized RL methods, like QeRL.
-
There is no end-to-end efficiency benefit reported. Both rollout time, and GPU memory should be benchmarked in the paper.
-
The sensitivity proxy is indirectly related to RL training. There is no evidence show that better sensitivity proxy leads to good final performance on benchmarks or less exploration during RL training.
Questions for authors
It would be much helpful if the authors could conducted more extensive experiments and comparisons with other quantized RL methods.