- Rating
- 3
- Confidence
- 3
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
DataFlex-RL provides a common GRPO evaluation platform for selection, reweighting, and mixture adaptation. Its primary Qwen2.5-7B-base experiment compares 13 configurations over 12 matched seeds and 12 math, logic, and science benchmarks. Uniform GRPO gains 7.76 Overall points over the untrained model, but no tested data policy shows a paired 95% confidence interval excluding zero relative to its control. A corrected Llama-3.1-8B-base extension and an evaluation-sensitivity analysis examine whether this result depends on model family or benchmark coverage.
Critical strengths
The finding that tested data policies provide no reproducible improvement over uniform controls supports uniform sampling as a strong baseline for RLVR data-policy evaluation. Uniform GRPO itself improves Overall by 7.76 points on Qwen2.5-7B-base, so this negative result is unlikely to be explained simply by failed training.
The primary experiment covers 13 configurations, 12 matched seeds, and 12 benchmarks, with additional model-family and model-scale checks. Appendix A.3 adds matched random, matched update tokens, and alternative strengths controls to examine targeting, update volume, and intervention strength. The Math-Heavy-6 versus DB-12 analysis also checks evaluation sensitivity.
The platform brings selection, reweighting, and mixture adaptation into a shared rollout, verification, GRPO optimization, and evaluation workflow. New data policies can therefore be compared under the same protocol without rebuilding the pipeline.
Critical weaknesses
Section 3.2 and Table 1 describe the signals but leave some intervention rules unclear, including difffilter's default solve-rate band and the weighting formulas for per and softmax. These details matter for reproducing the comparison.
The Llama gain interval needs clarification. Under the same seed-level interval calculation, subtracting the fixed untrained score of 10.87 from Table 6's [20.01, 22.23] gives [9.14, 11.36], whereas Table 2 reports [7.75, 12.33]. This appears to be a fixable reporting issue and does not overturn the primary Qwen result.
Questions for authors
What are difffilter's default solve-rate band and the weighting formulas and default hyperparameters for per and softmax?
Do Tables 2 and 6 use the same Llama seeds and interval estimator? If so, how should the gain interval be reconciled?