- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes VISTA, a RL framework for wide-search tasks. VISTA works by prompting the policy model to output a table at each search turn, allowing intermediate reward signals during RL. These signals are used to redistribute credit within the trajectory, keeping the final outcome as the primary reward signal. Experiments on a wide variety of model families show strong performance, and show that VISTA training also transfers well to multi-hop QA evaluations.
Critical strengths
- Experimentation seems thorough, with a wide range of model families and evaluations tested.
- The key idea behind the method (output partial progress at each stage) is fairly straightforward and reasonable, and the ablations around reward shaping (ensuring dominant terminal state, etc) are useful. I like the comparisons of keeping intermediate answers in/not in the context too, which feels a bit different to recent popular meta-RL work (e.g., https://arxiv.org/abs/2603.11327, https://arxiv.org/pdf/2602.13949).
- The results in Appendix B are especially interesting, digging further into how to correctly structure the process rewards.
Critical weaknesses
- In Figure 3, it seems to show that the GRPO baseline drops after training, and this is reflected also in the Qwen3-8B results table, which feels a bit odd and potentially a sign of poor baseline tuning (e.g., some KL penalty, or adjusting the LR could help perf). Especially since GRPO seems to improve for all other base models.
- VISTA costs more compute to run (extra judge, ignoring decode costs), and reaches the final quality values in Table 19 far slower than GRPO. But there is (AFAIK) no compute-matched comparison of VISTA to GRPO (this could be especially interesting given the VISTA-trained models seem to be more token efficient - does this offset the extra judge cost?). Could it be that a well-tuned GRPO baseline given the same training compute as VISTA performs just as well, if not better? Especially for larger models where VISTA’s relative gains are smaller? What happens at different training budgets?
- This method seems very domain specific, with evaluations limited to agentic search tasks, limiting impact.
Questions for authors
- How were hyperparameters tuned/chosen, and did authors attempt to fix the collapsing/poor later performance in WebLeaper+Format GRPO results, seen in Figure 3?
- What is the standard deviation / error bars on the evaluations? Alot rests on WideSearch, with only ~200 questions, so I’m curious how noisy the evaluation is. The gaps in performance do seem wide, so I think the method is probably effective, but it could be that e.g. the better performance of VISTA on Qwen3.8-27B is within noise.
- What model training curves is Figure 3 showing? I am curious what these curves look like for all models, especially ones where VISTA does not drastically improve over the GRPO baseline.
Thank you for the rebuttal. It addressed most of my concerns. I therefore raised my rating from 2 to 3.