- Rating
- 2
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper introduces Pareto-GRPO, which incorporates prompt-local Pareto ranking into GRPO to balance answer correctness and search efficiency. Rather than relying on a fixed weighted reward, the method ranks trajectories within each prompt group according to both correctness and search cost. Experiments on seven search-augmented QA benchmarks with Qwen2.5-3B and Qwen2.5-7B show improvements in accuracy, search efficiency, and training stability over several GRPO-based baselines.
Critical strengths
- Using prompt-local Pareto ranking for credit assignment is intuitive and well motivated. It addresses the limitations of fixed reward weighting while requiring only a relatively minor modification to the standard GRPO framework.
- In addition to the empirical evaluation, the paper provides theoretical analysis and proofs for the proposed method.
Critical weaknesses
- The maximum search budget is set to B=4, which seems relatively small for evaluating search efficiency. Allowing only four searches may make it difficult to fully distinguish between efficient and inefficient search strategies, particularly on more complex tasks that require longer search trajectories.
- Although the evaluation covers seven benchmarks, most of them are relatively easy QA datasets. They may not be sufficiently challenging to demonstrate the effectiveness of the proposed method in more realistic search settings. Additional evaluation on harder benchmarks, such as BrowseComp-Plus and GAIA-Text, would make the conclusions more convincing.
Questions for authors
- Have the authors considered evaluating larger budgets, such as B=8 or B=12? It would be helpful to know whether the observed improvements remain consistent when longer search trajectories are allowed.
- Could the authors provide additional results on more challenging search benchmarks, such as BrowseComp-Plus or GAIA-Text? Such experiments would help clarify whether the method generalizes to more complex, real-world search tasks.