- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
DePO modifies GRPO by smoothing the per-token importance ratios within each response with a simple filter (moving average, exponential average or Savitzky–Golay), and uses the smoothed ratios as fixed weights in the update. The aim is to reduce noisy updates and slow entropy collapse. Experiments on math and code with Qwen3-1.7B, Qwen3-4B and LLaMA-3.2-3B show the best average scores and higher entropy than GRPO.
Critical strengths
- Simple, cheap method that is easy to add to GRPO.
- The authors are honest about what their analysis does not prove.
Critical weaknesses
- Key settings of the method are missing. The paper itself says several are "not specified in the available records," including whether the default window size of 5 is a radius or a full width (Sec. 4.1, App. B), the EWMA initialization, the seeds and the decoding settings. No code is provided.
- The tables are described as "existing records," not a rerun; there are no seeds, confidence intervals or significance tests.
- Gains over the best baseline are small (e.g., +0.7 on Qwen3-4B math) and may be noise.
- Some results look implausible: LLaMA-3.2-3B-Instruct scores 56–59% on AIME24 with every method.
- Baseline results are unusually regular, with the same ranking and nearly constant gaps on every benchmark.
- Limited novelty: smoothing the ratios with a standard filter is a small change to GRPO.
- Hard to read; most claims are followed by long caveats.
Questions for authors
Q1 Can you release code and specify the missing settings?
Q2 How many seeds were used in Table 1, and what is the variance?
Q3 How do you explain the LLaMA-3.2-3B AIME24 results compared with published numbers?
Q4 Were all baselines run by you with the same budget and tuning?
Dear Reviewers,
Thank you for your time and thoughtful reviews. Please read the authors’ response and share your follow-up comments promptly, indicating whether your concerns have been addressed. Please raise any additional questions early enough to give the authors sufficient time to respond.
Thank you.
Best,
AC