- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
HaPRL trains visual-search agents with a reward based on human search traces. A frozen judge compares each rollout with a human reference, and incorrect answers receive zero reward. Experiments compare this approach with outcome-only RL on four backbones and study subsequent outcome-only training.
Critical strengths
The human traces outperform teacher-generated paths by 4.4 points and it gives 5.87 points gain over Outcome-RL from a cold start close to the original backbone's accuracy. These address whether the gains come from an evidence reference alone or from recovering the accuracy lost during SFT. The author finds larger gains from subsequent outcome-only training after process supervision, across three seeds. The reported improvements across all four backbones support the value of the training method.
Critical weaknesses
-
Table 2 scores each method on the questions it answers correctly. The Hard score falling from 71.54 to 65.56 could therefore reflect a change in the questions being scored, rather than worse search on the same questions. This does not establish the reward hacking claimed in Section 4.2. The scoring inputs also need clarification. Sections 3.1 and B.5 describe human traces for training questions, while Table 2 covers held-out benchmarks. Appendix B.4 lists the original image and a textual rollout without stating whether the judge sees returned crops. These details matter for judging whether the agent saw the visual evidence needed to answer.
-
Two Hard accuracies do not fit the stated evaluation setup. Section 4 specifies one greedy rollout per question, Appendix B.1 uses binary correctness, and the released split has 106 questions [1]. The 20.09 and 27.92 scores for the two 8B HaPRL models cannot come from integer correct-answer counts over 106, rounded to two decimals. A different subset or run average could explain them, but the exact Hard gains remain uncertain until the calculation is specified.
[1] Mini-o3: Scaling Up Reasoning Patterns and Interaction Turns for Visual Search. https://arxiv.org/abs/2509.07969v1
Questions for authors
- For Table 2, where did the held-out human references come from, and did the judge receive the actual crops?
- How many questions were used for the two Hard scores, and were the results averaged over multiple runs?
3 comments
Response to Reviewer YFFR [2/2]
W2 & Q2: Exact Calculation of the Two Hard Scores and Unified Evaluation Protocol
Thank you for checking the integer denominators. The discrepancy arose because the manuscript omitted one detail of our unified evaluation aggregation rule, specifically that unparseable rollouts are excluded from that run's denominator:
- To reduce variance from multi-turn online serving and API judging, every cell in Table 1 reports the arithmetic mean accuracy across 2 independent greedy evaluation runs. Each run executes 1 greedy rollout per question (
temperature = 0) with binary semantic grading; if a rollout fails to produce a parseable answer within the 6-round budget (e.g., an unclosed<answer>tag or judge timeout, i.e., an invalid bad case), that instance is excluded from the valid denominator of that run. - Under this single rule across all 96 cells of Table 1, 90 cells have zero bad cases and identical outcomes across both runs, reducing to standard integer ratios .
- Only 6 cells in the entire table have a slightly smaller valid denominator than due to occasional unparseable rollouts, and the arithmetic mean of their two runs reproduces the reported numbers to two decimal places:
| Model & Benchmark | Run 1 (Correct / Valid) | Run 2 (Correct / Valid) | 2-Run Arithmetic Mean (Reported in Table 1) |
|---|---|---|---|
| Qwen3-VL-4B Base - V*QA () | 67.16 | ||
| Qwen3-VL-8B Base - V*QA () | 71.59 | ||
| Qwen3-VL-8B +HaPRL - Hard () | 20.09 | ||
| Qwen3-VL-8B +HaPRL - V*QA () | 72.73 | ||
| LLaVA-1.5-8B +HaPRL - Easy () | 50.52 | ||
| LLaVA-1.5-8B +HaPRL - Hard () | 27.92 |
Conservative full-denominator lower bound on Hard: To rule out any upward bias from excluding unparseable rollouts, even under the strictest accounting that treats every unparseable rollout as incorrect over the full denominator of 106 and takes the lower of the two runs:
- Qwen3-VL-8B +HaPRL has a lower bound of (2-run mean ), exceeding matched Outcome-RL (15.09%) by +3.78 points at the lower bound (+4.72 on the mean);
- LLaVA-1.5-8B +HaPRL has a lower bound of (2-run mean ), exceeding matched Outcome-RL (12.26%) by +13.21 points at the lower bound (+14.63 on the mean).
Thus, the gains of both 8B models on VisualProbe Hard hold under every denominator convention and individual run. We apologize for omitting the unparseable-instance rule in the initial submission and will state it explicitly in Section 4 and the caption of Table 1.
Sincere Thanks and Follow-up to Reviewer YFFR
Dear Reviewer YFFR,
Thank you again for taking the time to review our manuscript and share your feedback. We have responded to your comments and revised the manuscript accordingly. We hope that our responses and revisions address your concerns. If any questions or concerns remain, we would be happy to address them during the discussion period.
Best regards,
The Authors
Response to Reviewer YFFR [1/2]
We sincerely thank the reviewer for the constructive comments and valuable suggestions. We address each point below and will revise the manuscript accordingly.
W1 & Q1: Paired Validation of Table 2 and Judge Input Clarification
(1) Paired process-score analysis on identical questions. Scoring each policy on its own answer-correct subset means the Hard drop from 71.54 to 65.56 can reflect a change in which questions are scored. On Qwen3-VL-4B, we extract the 707-question three-way intersection answered correctly by SFT, Outcome-RL, and HaPRL and the 801-question two-way intersection answered correctly by Outcome-RL and HaPRL, and compare scores on the exact same questions:
On the exact same 707 (and 801) questions, HaPRL scores higher than Outcome-RL across all six subsets (+2.91 and +2.85 macro). On a 100-question stratified subsample, 3 blinded human raters (83.45 / 84.76 / 86.68, question-level judge-human ρ=0.74) and a third-party GPT-4o judge (83.74 / 84.92 / 86.78) yield the identical ranking.
(2) Held-out references and whether the judge receives cropped images (Q1).
<think>reasoning, each<grounding>call's source view and exact normalized coordinates[x0, y0, x1, y1], and the final answer; training calls also include the human trace andfinal_bboxes). Although cropped image patches are not passed again as separate images, holding both the full original image and exact normalized crop coordinates allows the judge to check directly on I0 whether each crop covers the visual evidence needed for the answer. We will add this input specification and the exact prompts for both judges (qa_process.pyandqa_process_v1.pyin the supplementary code) to the revised manuscript.