- Rating
- 1
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes TrendGRPO, a sample-wise, history-conditioned advantage shaping method for GRPO in mathematical reasoning. The motivation is that standard GRPO builds advantages only from the current rollout group. Two prompts with identical current reward patterns therefore receive identical updates, even if one is improving across repeated visits and the other is deteriorating. TrendGRPO keeps a persistent correctness history for each training sample and computes the deviation between current group accuracy and historical accuracy. After GRPO normalization, it uses this deviation to rescale positive and negative advantages asymmetrically, together with a warmup coefficient, a difficulty weight and an auxiliary failure-streak term. The method also adds a numerical-proximity reward for incorrect answers. The authors train Qwen3-0.6B, Qwen3-8B and OLMo-3-7B-Instruct on a 2,048-sample subset of DAPO-Math-17K. They report average gains of 0.7–1.9 points over GRPO across seven math benchmarks, supported by history-correspondence ablations, component ablations and training diagnostics.
Critical strengths
-
The research motivation is clear, and TrendGRPO can be integrated into existing GRPO training pipelines with only minor implementation changes.
-
The paper clearly distinguishes between maintaining correctness statistics and replaying historical trajectories, providing a useful basis for understanding the positioning of the proposed method.
-
The controlled experiments cover three models, including 7B and 8B models from two model families, and demonstrate improvements on most of the evaluated benchmarks.
Critical weaknesses
-
Insufficient comparison with the most closely related methods; missing baselines. Section 2 discusses GRPO-LEAD and HA-DW, but the controlled comparisons are essentially GRPO versus the paper's own internal variants. There is no comparison, under the same reward and training budget, against representative difficulty-aware or history-based advantage reweighting baselines, so it is hard to judge what the proposed design actually gains over existing conditioning strategies.
-
The training-cost evidence is useful but incomplete. The relative costs of 1.05, 0.971 and 0.966 suggest that the additional end-to-end overhead may be limited. However, the paper does not report the GPU model, the actual wall-clock units, the run-to-run timing variation, or the number of generated tokens.
-
Validation is concentrated in a training regime tailored to repeated sample visits. All experiments train for six epochs on the same 2,048-sample subset, and the key ablations are conducted only on Qwen3-0.6B.
-
The existing ablations do not adequately separate temporal information from sample difficulty. Shuffling the historical state across samples breaks not only the sample–history correspondence but also the link between a sample and its historical difficulty. The resulting performance drop therefore supports using statistics tied to the current sample, but it does not by itself establish that the signed current-to-historical deviation is the decisive factor. Adding controls that use only historical difficulty and only current group accuracy would make this much clearer.
-
Some presentation choices weaken the clarity of the core evidence. Table 1 places externally reported results alongside the controlled experiments, and some of the averages are computed over different subsets of available benchmarks.
Questions for authors
-
Under matched training conditions, how does TrendGRPO perform relative to a purely difficulty-based reweighting baseline, and to at least one of the related methods discussed in Section 2?
-
What are the exact decoding, answer-verification, validation-set and checkpoint-selection settings used for Table 1?
-
Could you provide multi-seed results for the main comparisons?
-
Under a fixed generation budget, how do the gains of the method change when the number of distinct training problems is increased and the number of training epochs is reduced?