- Rating
- 2
- Confidence
- 2
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper proposes Stage-Aware Weighting (SAW), a lightweight method that adjusts objective weights using coefficients of variation computed from the current rollout batch. SAW computes weights using the coefficient of variation of each reward in the current rollout batch and applies them to reward or advantage aggregation. It can be integrated with both GRPO and GDPO without additional gradient computations. The paper further evaluates its effectiveness through ablation studies comparing alternative weighting statistics and fixed reward-weight ratios.
Critical strengths
The paper introduces a stateless weighting rule based on current-batch reward dispersion, integrated at the reward level in GRPO and the advantage level in GDPO.
Critical weaknesses
- The writing needs improvement. The paper should be organized in a more logical order, and the motivation should be made clearer.
- The effectiveness of SAW remains insufficiently explored. The experiments are limited to tool calling and summarization, without investigating how the proposed weighting mechanism performs as task difficulty increases.
Questions for authors
- More details about existed work are necessary and show me the clear motivation and advantage of the proposed approaches.
- SAW derives adaptive weights solely from the current rollout batch which is sensitive to batch size. Small batches may yield noisy estimates, causing weight fluctuations that could destabilize training without historical statistics or moving averages. I want to know if we can see varying batch size to assess SAW’s robustness.