- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper studies the spectral properties of RLVR gradients and observes that their effective rank decreases during training. Based on this observation, the authors propose Ares, which estimates the effective rank from the gradient spectrum and dynamically adjusts the rank of reconstructed gradients before applying AdamW. Experiments on Qwen3 models of different scales show improvements over several optimizers on mathematical reasoning benchmarks.
Critical strengths
-
The observation that RLVR gradients become increasingly spectrally concentrated is interesting, and the proposed optimizer is well motivated by this finding.
-
The experimental results are promising, showing consistent improvements over AdamW across different model scales and RLVR algorithms.
Critical weaknesses
-
The benefit of adaptive rank is not sufficiently demonstrated. In Table 4, removing dynamic rank only reduces average accuracy from 27.2 to 26.6 on Qwen3-1.7B and from 38.5 to 37.6 on Qwen3-4B. This suggests that much of the improvement may come from low-rank gradient filtering rather than rank adaptation itself. Comparisons with fixed-rank variants under the same optimizer would better support the main contribution.
-
The spectral interpretation needs further justification. Although Ares reconstructs low-rank gradients, the subsequent AdamW momentum and elementwise preconditioning do not necessarily preserve their rank. It is therefore unclear whether the actual parameter updates remain in the claimed low-rank subspace. Also, no multiple-seed results are reported, making the relatively small gains from adaptive rank difficult to assess.
Questions for authors
-
How does Ares compare with fixed-rank variants using different ranks but otherwise identical settings?
-
Does the low-rank structure persist in the actual parameter updates after AdamW preconditioning?
-
Are the improvements from dynamic rank consistent across multiple training seeds?
I appreciate the authors' patient responses, which addressed most of my questions; however, I remain somewhat uncertain about the value of low-energy gradients in RLVR and SFT. Due to the additional experiments and explanations provided by the authors, I will consider raising my score.