- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
The paper proposes TPV, a turn-level process verifier for deep research agents. An offline privileged teacher with access to ground truth assigns A/B/C/D grades to search and visit turns. An unprivileged student verifier is then trained to predict these grades from interaction context alone. The frozen verifier serves two purposes: providing dense turn-level rewards during GRPO training and reranking sampled trajectories at test time. Experiments cover five deep research benchmarks and compare TPV against ORM, IGPO, and a uniform-reward ablation, with additional analyses of verifier scaling, cross-architecture transfer, and reranking aggregation.
Critical strengths
- The paper addresses a relevant and practically motivated problem in turn-level credit assignment. It identifies a real limitation in multi-turn deep research agents: terminal outcome rewards cannot easily distinguish productive exploration from unproductive or harmful intermediate actions, and all-fail rollout groups may provide limited learning signal. TPV’s turn-level process supervision offers a plausible direction for mitigating this issue.
- The proposed framework is reasonably clear, and the empirical results are somewhat encouraging. Separating privileged label construction from unprivileged verifier deployment is a sensible design choice, and reusing the same frozen verifier for both RL training and test-time reranking is a potentially useful contribution. The evaluation across five benchmarks, together with verifier scaling and reranking analyses, provides some initial evidence of practical potential.
Critical weaknesses
- The cross-architecture generalization claim is not supported at the policy level.
RQ4 asks whether the framework generalizes across policy architectures and verifier scales. However, Section 3.5 only replaces the student verifier's backbone from Qwen3-8B to Llama-3.1-8B and compares verifier macro-F1 (0.643 vs. 0.648). The policy remains Qwen3-4B-Thinking throughout the experiments.
Therefore, the current experiment demonstrates that turn-level discrimination can transfer across verifier model families, but it does not establish that TPV as an RL training method generalizes across policy architectures. The conclusion that performance stems from process supervision rather than biases tied to a particular backbone is consequently broader than the evidence supports. Different policies may respond differently to process rewards, and TPV's improvement over ORM could vary substantially across architectures.
This gap is particularly important because cross-policy generalization is explicitly included in the research questions, and Section 5 concludes that TPV "scales effectively across policy architectures." The claim would be much better supported by evaluating at least one additional policy architecture under comparable SFT, ORM, and TPV settings.
- An internal inconsistency in the reported GAIA results makes the magnitude of the main performance gain unclear.
In Table 1, the TPV row reports GAIA accuracies of 76.19%, 57.58%, and 31.58% for L1, L2, and L3, respectively. According to Table 4, these levels contain 42, 66, and 19 questions, for a total of 127 questions. The reported level-wise accuracies therefore imply 32, 38, and 6 correct answers, respectively, yielding an overall accuracy of 76/127 = 59.84%. However, Table 1 reports 53.54%, corresponding to 68/127 correct answers. Section 3.2 repeats the reported overall figure of 53.54%.
The discrepancy should be checked against the original evaluation results and the inconsistent entries corrected. While this does not by itself establish that the other benchmark results are incorrect, it leaves the reported GAIA performance and the magnitude of TPV's improvement over ORM uncertain.
- The necessity and practical benefits of teacher-to-student distillation are not sufficiently established.
Appendix D.2 reports a turn-level AUC of 0.820 for the teacher without ground-truth access, compared with 0.777 for the student verifier. This raises an important question about the motivation for distillation: what does the student provide beyond what could be obtained by deploying the teacher in its ground-truth-free configuration?
The comparison requires clarification. In particular, the paper should specify how the AUC scores are computed and whether the teacher and student use comparable grade-to-score mappings. A difference in offline AUC alone does not establish which model is more effective for the intended applications. Comparing the two models as downstream RL reward providers and test-time rerankers would be more informative. If deployment efficiency motivates distillation, the student's advantages in inference cost, memory, or throughput should also be quantified.
Questions for authors
- GAIA result consistency: Table 1 appears to be internally inconsistent. The TPV L1/L2/L3 accuracies (76.19% / 57.58% / 31.58% over 42 / 66 / 19 questions, respectively) imply 76/127 = 59.84%, whereas the reported overall accuracy is 53.54% (= 68/127). Could you check the original evaluation results and correct the inconsistent entries? Section 3.2 repeats the same overall figure.
- Teacher-to-student distillation: Appendix D.2 reports a higher turn-level AUC for the teacher without ground-truth access than for the student verifier (0.820 vs. 0.777). Could you clarify how these AUC scores are computed and whether both models use comparable grade-to-score mappings? More importantly, what motivates distillation in this setting? Could you compare the two models' downstream performance as RL reward providers and test-time rerankers, and quantify the student's efficiency advantage?
- Cross-policy generalization: Could you evaluate TPV with at least one additional policy architecture, reporting results for SFT, ORM, and TPV? Comparing TPV's improvement over ORM across policy architectures would directly test the cross-policy generalization claim in RQ4. Alternatively, the authors should narrow the corresponding claims to verifier-backbone transfer.
Thank you for suggesting stronger reranking baselines and for flagging the inconsistency in the GAIA results.
We compared all methods on the same 128 BrowseComp-Plus questions, with eight fixed candidates per question. No reranker had access to the reference answers.
We focus on the answer-only comparison because it directly tests whether TPV's gains in Figure 4 can be recovered by strengthening the original majority-vote baseline. The answer-only selectors use the same eight final answers as majority voting and replace vote counts with the model's judgment given the question. TPV improves over majority voting by 8.59 percentage points and over answer-only Qwen by 4.69 points, and comes within 2.34 points of answer-only DeepSeek. The whole-trajectory judges provide a complementary comparison by evaluating each candidate's final answer alongside the evidence from its full rollout.
TPV also outperforms the rules-only control by 10.16 percentage points. This ablation holds the candidate pool, tool turns, and mean pooling fixed and measures what the learned turn scores add beyond the novelty and format rules.
We ran multiple evaluations. The originally reported overall score of 53.54% (68/127) came from an earlier complete evaluation, and the higher per-level scores, which imply 59.84% overall, came from another evaluation. The inconsistency arose because we combined figures from different runs in the same row.
We will update Table 1 and Section 3.2 using all four figures from the latest complete TPV run: 73/127 (57.48%) overall, with 30/42 (71.43%) on L1, 37/66 (56.06%) on L2, and 6/19 (31.58%) on L3.