- Rating
- 2
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 3
Read review
Paper summary
This paper proposes an RL algorithm for training search agents that uses fine-grained credit assignment, process rewards, and process-level advantage assignment to improve their performance.
Critical strengths
Fine-grained credit assignment for retrieval actions is a timely topic. The proposed method is reasonable and feasible.
Critical weaknesses
-
Qwen3-8B is not a strong baseline for search agents, so the choice of backbone is not representative. The authors should evaluate the gains using backbones such as Qwen3.5 or Qwen3.6 that have undergone targeted agent training.
-
The naïve Qwen3-8B baseline achieves particularly low scores on some benchmarks, such as HotpotQA, which is inconsistent with my expectations based on previous results. I would like the authors to clarify the experimental setup.
-
Process rewards and process-level credit assignment are not particularly novel in RL research and have already been explored in previous work on training search agents. The novelty of this paper is therefore limited.
-
The ablation comparisons, such as those against outcome-based methods, do not control for total computational cost, including PRM training.
Questions for authors
See weaknesses.