- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
In this paper, the authors propose LP-GRPO, a method that decides which prompts get the next GRPO rollout groups. It keeps two scalars per prompt: a slow EMA of the pass rate (“mastery”) and a fast EMA of the signed gap between the new pass rate and that mastery (“learning progress”, LP). The motivation is that difficulty-based schedulers give two prompts with the same pass rate the same priority, even when one is improving and the other is regressing. On 14,795 revisits from uniform GRPO, the authors report that the next visit raises the pass rate by 22.4 points when stored LP is positive and by 1.1 points when it is negative (Section 1).
LP-GRPO has two parts. LP-Sampling samples prompts in proportion to , multiplied by so that only positive progress gets a boost (Eq. 8). LP-Reweighting multiplies each prompt’s GRPO advantages by a positive weight built from the offline pass rate and the stored LP (Eqs. 11 to 14). Both are meant to read only state stored before the current group is generated.
With 128 retained groups per step and 360 steps, LP-GRPO reaches 69.75 on eight multimodal benchmarks with Qwen3-VL-8B-Instruct. GRPO reaches 66.50 and CDAS 68.13 (Table 1). LP-Sampling alone reaches 69.18 (Table 2). Sampling with positive LP, no LP, and negative LP gives 69.18, 68.29, and 66.89 (Figure 5). The paper also reports 58.20 on Qwen2.5-VL-7B (Table 1) and 44.86 on a text-only Qwen2.5-7B protocol (Table 5).
Critical strengths
- The method is cheap and easy to adopt. It stores a few scalars per prompt and adds no rollouts, no selector model, and no value model (Section 3.3). Sampling reads only state written through step , so a group’s own reward cannot raise its own chance of being picked. This matters to my assessment because even a reliable gain of one or two points at no extra cost would be useful to anyone running GRPO.
- The direction ablation is well controlled. Figure 5 compares positive LP, no LP (), and negative LP with the same eligibility rule, batch size, and optimizer. The order is the same on all eight benchmarks and at every checkpoint, ending at 69.18, 68.29, and 66.89. This is the most direct evidence in the paper that the sign of recent progress carries information beyond the current pass rate. It is the main reason I am not recommending a clear rejection.
Critical weaknesses
- The evidence does not separate signed progress from noise or from simpler explanations. Each method is trained once (Appendix A.2: “Each trained checkpoint is from one run”). Each benchmark is scored with a single greedy decode at temperature 0 (Section 4.1). So n = 1 for training runs and n = 1 sample per question. The gaps behind the main claim are small: 1.62 over CDAS (Table 1), 0.89 between positive LP and no LP (Figure 5), and 0.57 from adding reweighting to LP-Sampling (Table 2). Several test sets are small, including LogicVista and the testmini/MINI splits of MathVista and MathVerse. Headroom is also uneven. MathVista starts at 77.20 for the base model, and the best matched run reaches only 80.40 (Table 1). LP-GRPO is not the best matched method on MathVista or MMK12.
There is also a simpler explanation for much of the gain. The pool keeps 10,000 prompts that the base model solved 0 of 8 times, about a third of the 30,445 prompts (Appendix A.1). GRPO spends 29% of its groups on nearly unsolved prompts (Figure 1). Under LP-Sampling these prompts score about , while mid-difficulty prompts score near 1. Mastery-only sampling already moves the average from 66.50 to 68.29 (Figure 5). The paper does not compare against absolute LP, the usual choice in the learning-progress literature (Graves et al., 2017; Portelas et al., 2020; Kanitscheider et al., 2021). It also does not report the constant-scale, difficulty-only, and progress-only reweighting controls promised in Section 3.3. I do not expect a static filter to match LP-GRPO. My point is that without these controls I cannot tell how much of the 3.25-point gain comes from the sign of progress, and the sign of progress is the claim in the title.
What would fix this: 3 training seeds for GRPO, CDAS, LP-Sampling, and LP-GRPO with average pass@1 over at least 8 samples, plus GRPO on the 1/8 to 7/8 subset, absolute-LP sampling, and the reweighting split.
- The motivating statistic and the mechanism analysis may not show what the paper says they show, and the paper describes the method in two different ways. The 22.4 vs 1.1 result (Section 1) does not say what the gain is measured against. The 142 strata are not described anywhere. If the gain is the next pass rate minus mastery, I would expect a gap like this even with no learning: mastery is a lagged, noisy estimate, and positive LP means the recent pass rates were above it. Proposition 3 already gives a stationary noise model that could serve as the null. Table 3 has a similar issue. It follows only prompts that reach a fourth visit, and the sampler favors prompts whose pass rate went up, so the 21.9-point gain on hard prompts is partly survivorship. It also has no GRPO column.
Separately, Eq. (10) and Section 3.3 say that first-visit prompts get LP = 0 for reweighting, so that the method uses only state stored before generation. Appendix B.4 and step 5 of Algorithm 1 use on a first visit instead, which lets the current group set its own weight. At step 40 only 5 of 128 groups use stored LP (Table 4), so this rule sets the weight of most groups early in training.
What would fix this: the exact definition of the 22.4/1.1 statistic, the same numbers under a stationary null, a GRPO column for Table 3, and a statement of which reweighting rule produced the reported runs.
Questions for authors
- How were the method’s 13 hyperparameters chosen? Table 10 covers only MathVision, and no validation set is mentioned. Table 6 lists a 3-epoch schedule, but all methods are compared at step 360, about 1.5 epochs, and the curves in Figure 3 are still rising. * What is the learning-rate schedule? Does “matches GRPO’s final average in half the updates” refer only to step 360?
- PSPO is listed as a baseline (Section 4.1) and appears in Figure 3, but not in Table 1. Why?
- In Table 5, Random and Curriculum Sampling have identical scores on all six benchmarks, and the PSPO row is a published Qwen2.5-* Math-7B result. Were these baselines rerun? The AIME scores (e.g., 12.08) are not multiples of 1/30. Is the text-only evaluation averaged over several samples, unlike the multimodal one?
- Do the Qwen2.5-VL-7B baselines taken from Huang et al. (2026) use the same data, rollout count, and step budget as your LP-GRPO run?
- Minor: in Table 1, VPPO’s 80.40 on MathVista should be bold. Line 221 says “the ICLR formulation”. Figure 2 drops the from Eq. (8). Figure 1 is hard to read. Related work should discuss Foster and Foerster (2025) on learnability sampling, as well as GRESO, SEC, and Online Difficulty Filtering.