- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper introduces DocArena, an automated pipeline for constructing search-agent training data from raw documents. The pipeline extracts page content, profiles cross-page fact distributions, generates questions using reasoning templates, and validates both answer correctness and evidence-page necessity through leave-one-page-out checks. It produces DocArena-79K, containing 79,623 QA pairs from 8,336 documents. The authors also introduce Doc-Search, which combines a text-based policy with multimodal page retrieval and a reading tool, and trains the policy using answer and incremental evidence-retrieval rewards. Experiments evaluate document QA and transfer to text-based QA. The central contribution is the construction and empirical validation of evidence-aware training targets, rather than a new retrieval architecture or optimization algorithm.
Critical strengths
-
The paper addresses a consequential supervision problem and tests its downstream relevance. Distinguishing correct answers from correct evidence labels is important when the labels directly determine retrieval rewards. The paper goes beyond reporting a data-quality audit: in the equal-size training comparison, removing the leave-one-page-out check reduces EM from 24.18 to 21.39 on MMLongBench-Doc and from 47.96 to 45.41 on SlideVQA. This connects the proposed verification procedure to the performance of the trained agent, providing substantive evidence for the practical value of the contribution.
-
The controlled experiments separate several important sources of improvement. The authors explicitly acknowledge the limitations of comparing released checkpoints and additionally control initialization, tools, rewards, and training schedule in the data comparisons. The reward ablation is particularly informative: adding retrieval supervision improves both recall and EM, whereas retrieval-only training achieves high recall but poor answer accuracy. This supports the use of complementary retrieval and answer objectives and makes the empirical case more convincing than a leaderboard comparison alone.
Critical weaknesses
-
Unexplained discrepancies make parts of the evaluation difficult to reconcile. Figure 2 reports 247 two-page and 22 three-page MMLongBench-Doc questions, whereas Figure 10 reports 186 and 92, respectively. In addition, the full-data SlideVQA results in Figure 11 report F1/precision of 62.04/52.84, compared with 66.95/59.78 in Table 3. These differences may reflect distinct configurations or reporting errors, but the manuscript does not explain them. Clarifying the evaluation subsets and configurations is necessary to assess the evidence-count and scaling conclusions confidently.
-
The multi-page-specific benefits of the central components remain insufficiently isolated. The principal pipeline and reward ablations report aggregate results on evaluation sets containing substantial numbers of single-page questions. Although Figure 10 provides retrieval F1 broken down by evidence-page count, it does not establish whether the gains from evidence verification and retrieval supervision persist specifically on multi-page QA. Reporting the Table 5 and Table 6 comparisons separately for single-page and multi-page questions would clarify how strongly the experiments support the intended cross-page use case.
Questions for authors
-
Clarification of reported results: What explains the different evidence-page-count distributions in Figures 2 and 10, and the different full-data SlideVQA results in Figure 11 and Table 3? Please identify the corresponding checkpoints, evaluation subsets, and inference configurations.
-
Clarification through a breakdown of the existing evaluation: Could the authors report the key reward and pipeline ablations in Tables 5 and 6 separately for questions requiring one versus multiple evidence pages, including both retrieval and answer metrics? This would clarify whether the aggregate gains extend to the cross-page setting emphasized by the paper.
-
Clarification of the training algorithm: Section A.3.1 describes placing retrieval rewards at </search> tokens and the answer reward at the final token. How are these rewards converted into token-level advantages under GRPO? Please specify whether group normalization and credit assignment operate on trajectory-level returns or on action-specific returns; reward placement alone does not fully specify this mechanism.
2 comments
Rebuttal by Authors (Part 2)
Q2: Tables 5/6 reward and pipeline ablations by one versus multiple evidence pages. Both retrieval and answer metrics.
Response to Q2: We revised Section 4.3 and added Table 19 in Appendix A.5.1.
Table T12. MMLongBench-Doc. EM results are copied from Table T10.
| Reward / pipeline ablation | Distinct gold evidence pages | Recall | Precision | F1 | EM |
|---|---|---|---|---|---|
| Full pipeline (Ours) (Table 6) | 1 | 64.89 | 31.97 | 38.76 | 27.51 |
| Naive generation (Table 6) | 1 | 61.81 | 31.84 | 39.07 | 23.44 |
| W/o LOO (Table 6) | 1 | 65.09 | 30.34 | 36.59 | 24.10 |
| QA only / w/o PR (Table 5) | 1 | 59.75 | 32.91 | 39.77 | 25.17 |
| PR only / w/o QA (Table 5) | 1 | 69.20 | 28.73 | 35.71 | 6.95 |
| QA+PR (Table 5) | 1 | 64.89 | 31.97 | 38.76 | 27.51 |
| Full pipeline (Ours) (Table 6) | ≥2 | 55.35 | 37.31 | 42.25 | 20.34 |
| Naive generation (Table 6) | ≥2 | 48.32 | 34.51 | 36.94 | 13.24 |
| W/o LOO (Table 6) | ≥2 | 51.11 | 34.76 | 40.72 | 16.80 |
| QA only / w/o PR (Table 5) | ≥2 | 43.92 | 37.06 | 38.04 | 14.05 |
| PR only / w/o QA (Table 5) | ≥2 | 53.26 | 34.48 | 40.43 | 3.47 |
| QA+PR (Table 5) | ≥2 | 55.35 | 37.31 | 42.25 | 20.34 |
Table T13. SlideVQA. Multi-page F1 and EM are copied from the paper (Table 6b). Full pipeline (Ours) multi-page Recall and Precision are copied from the paper (Table 3). Single-page EM is copied from Table T11.
| Reward / pipeline ablation | Distinct gold evidence pages | Recall | Precision | F1 | EM |
|---|---|---|---|---|---|
| Full pipeline (Ours) (Table 6) | 1 | 89.44 | 54.90 | 60.55 | 49.57 |
| Naive generation (Table 6) | 1 | 87.56 | 53.15 | 58.23 | 47.28 |
| W/o LOO (Table 6) | 1 | 90.78 | 54.13 | 59.50 | 48.15 |
| Full pipeline (Ours) (Table 6) | ≥2 | 86.28 | 59.78 | 66.95 | 47.96 |
| Naive generation (Table 6) | ≥2 | 83.13 | 57.01 | 64.32 | 43.05 |
| W/o LOO (Table 6) | ≥2 | 84.21 | 58.97 | 65.59 | 45.41 |
The aggregate retrieval and QA gains extend to questions requiring multiple evidence pages. Tables T12–T13 report the key reward and pipeline ablations in Tables 5 and 6 separately for questions requiring one versus multiple evidence pages, including both retrieval and answer metrics. Groups count distinct gold evidence pages. Within each group, all policy variants use the same questions and evaluation settings. All scores are percentages, and the EM results match Tables T10–T11.
For the pipeline comparisons, the full pipeline (Ours) improves Recall, F1, and EM over naive generation on multi-page questions in both benchmarks. Retaining the leave-one-page-out check improves both retrieval F1 and EM, with EM gains of 3.54 percentage on MMLongBench-Doc and 2.55 on SlideVQA.
For the reward comparison, adding retrieval supervision to QA-only training improves both retrieval F1 and EM on multi-page MMLongBench-Doc, by 4.21 and 6.29 percentage, respectively.
Q3: Retrieval rewards at </search> and answer reward at the final token → token-level advantages under GRPO. Group normalization and credit assignment: trajectory-level versus action-specific returns.
Response to Q3: Thanks for the valuable comments! We clarified Section 3.3 and Appendix A.3.1 (Eq. 4).
Group normalization is applied to action-boundary rewards, and token-level advantages accumulate the normalized rewards from the current and subsequent actions. Retrieval rewards are recorded at </search> tokens and the QA reward at the final response token.
For each question, we sample trajectories. Let denote the token position of reward event in trajectory , and let be its number of search turns. At search turn , the boundary reward is , with . The terminal reward is . We compute the mean and standard deviation over all search and terminal reward events pooled across the trajectories for question , including zero-valued reward events. We then normalize each boundary reward and compute token-level advantages as follows:
\widetilde{u}_{i,j} &= \frac{u_{i,j}-\mu_q}{\sigma_q+\epsilon},\\\\ A_{i,\ell} &= m^{\mathrm{policy}}_{i,\ell} \sum_{j:\\,b_{i,j}\geq\ell}\widetilde{u}_{i,j}. \end{aligned}$$ Here $m^{\mathrm{policy}}$ selects policy-generated tokens and excludes padding and tool observations, and $\epsilon>0$ is a numerical stabilizer. The factors $\gamma^t$ are already included in the retrieval rewards, so no additional discount is applied in the accumulation. Different search actions receive advantages based on their remaining rewards, enabling action-level credit assignment while preserving the final-answer objective. KL regularization is applied separately in the actor loss, with coefficient 0.001.
Rebuttal by Authors (Part 1)
We appreciate Reviewer
Svya's recognition that our work addresses a consequential supervision problem, provides substantive evidence for the practical value of evidence-aware training targets, and separates several important sources of improvement through controlled experiments. We tried our best to address all the weaknesses and questions raised by the reviewer, as summarized below:W1/Q1: We revised Section 4.1, Appendices A.4.4 and A.5.2, and Figures 10–11, and added Table 20.W2: We revised Section 4.3 and added Table 19 in Appendix A.5.1.Q2: We revised Section 4.3 and added Table 19 in Appendix A.5.1.Q3: We clarified Section 3.3 and Appendix A.3.1 (Eq. 4).W1/Q1: Figures 2/10: different evidence-page-count distributions. Figure 11/Table 3: different full-data SlideVQA F1/precision. Corresponding checkpoints, evaluation subsets, and inference configurations.Response to W1/Q1: Thanks for the valuable comments! We revised Section 4.1, Appendices A.4.4 and A.5.2, and Figures 10–11, and added Table 20.Table T9. Evaluation subsets and inference configurations.
Figure 10 groups questions differently from Figure 2 to analyze data-scaling gains by the number of pages read. Figure 2 reports how many gold evidence pages each question has (40 three-page questions. The count of 22 refers to four-page questions). Figure 10 groups the same 859 questions by the number of pages the model actually reads. For example, a question with two annotated evidence pages may lead the model to read three pages, while one with three annotated pages may involve reading only two. Grouping by gold evidence counts can mix searches that read different numbers of pages, making it harder to determine whether additional training data benefits searches involving more reading.
Figure 11 uses different evaluation settings from Table 3 to isolate the effect of training-data scale. Table 3 reports results on the 567 SlideVQA questions with multiple gold evidence pages and follows the configuration reported in Section 4.1 (Implementation) and Appendix A.4.4 (More Illustrations on the Comparison): two training search turns and three inference search turns.
Figure 11's retrieval panel evaluates all 2,215 SlideVQA questions. It fixes both the training and inference search budgets at two turns and uses the step-150 checkpoint from each run, to study the benefit of additional training data (varying only the training-data fraction) within the same allowed search budget. Table T9 summarizes the evaluation subsets and inference configurations.
W2: Multi-page-specific gains from evidence verification and retrieval supervision. Tables 5/6 comparisons separately for single-page and multi-page questions.Response to W2: We revised Section 4.3 and added Table 19 in Appendix A.5.1.Table T10. Pipeline and reward comparisons separately for single-page and multi-page QA: MMLongBench-Doc, EM (%).
Table T11. Pipeline comparison separately for single-page and multi-page QA: SlideVQA, EM (%). The multi-page results are copied from manuscript Table 6b.
The gains from evidence verification and retrieval supervision persist on multi-page QA (Tables T10–T11). We retain the original policy checkpoints, inference settings, and scorer, and group questions by their original gold evidence-page annotations: one page versus at least two. Within each group, all compared policies use the same questions. On multi-page questions, retaining the leave-one-page-out check improves EM by 3.54 percentage on MMLongBench-Doc and 2.55 on SlideVQA. Adding retrieval supervision to QA-only training improves multi-page MMLongBench-Doc EM by 6.29 percentage.