acceptodds
Under review as a conference paper at ICLR 2027

DocArena: From Raw Documents to Search Training Data Beyond Answer Correctness

Abstract

Search agents need to locate and combine information from document collections through multi-step retrieval and reasoning. We study how existing documents can provide both answer targets and evidence labels for learning this capability. How- ever, correct question-answer pairs do not ensure accurate evidence labels, and incorrect labels can reward irrelevant retrieval or penalize the retrieval of support- ing evidence. To address this challenge, we propose DocArena, a fully automated pipeline that constructs questions, answers, and automatically checked evidence- page labels from raw documents without human training annotations or expert search demonstrations. Specifically, we extract textual and visual page content with a multimodal LLM and profile the distribution of facts across pages to guide evidence selection and multi-page question construction. The pipeline checks an- swer consistency and tests whether each question remains answerable after re- moving each evidence page in turn. We construct DocArena-79K with 79, 623 QA pairs from 8, 336 documents across 16 domains and 49 languages. We fur- ther develop Doc-Search, which decouples multimodal page perception from a text-based policy and uses the constructed answer and retrieval targets for rein- forcement learning. Experiments cover six multimodal document scenarios and seven text-based QA benchmarks. Pipeline ablations show that evidence checks improve training sample quality. Under fixed training settings, DocArena data improve document QA. Further analyses show that retrieval supervision improves evidence coverage and reduces repeated queries, while combining it with answer supervision improves QA. The trained policy also transfers from document col- lections to text-based QA.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
document understanding, document retrieval, search agent, post-training, data curation

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_SvyaOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces DocArena, an automated pipeline for constructing search-agent training data from raw documents. The pipeline extracts page content, profiles cross-page fact distributions, generates questions using reasoning templates, and validates both answer correctness and evidence-page necessity through leave-one-page-out checks. It produces DocArena-79K, containing 79,623 QA pairs from 8,336 documents. The authors also introduce Doc-Search, which combines a text-based policy with multimodal page retrieval and a reading tool, and trains the policy using answer and incremental evidence-retrieval rewards. Experiments evaluate document QA and transfer to text-based QA. The central contribution is the construction and empirical validation of evidence-aware training targets, rather than a new retrieval architecture or optimization algorithm.

Critical strengths
  1. The paper addresses a consequential supervision problem and tests its downstream relevance. Distinguishing correct answers from correct evidence labels is important when the labels directly determine retrieval rewards. The paper goes beyond reporting a data-quality audit: in the equal-size training comparison, removing the leave-one-page-out check reduces EM from 24.18 to 21.39 on MMLongBench-Doc and from 47.96 to 45.41 on SlideVQA. This connects the proposed verification procedure to the performance of the trained agent, providing substantive evidence for the practical value of the contribution.

  2. The controlled experiments separate several important sources of improvement. The authors explicitly acknowledge the limitations of comparing released checkpoints and additionally control initialization, tools, rewards, and training schedule in the data comparisons. The reward ablation is particularly informative: adding retrieval supervision improves both recall and EM, whereas retrieval-only training achieves high recall but poor answer accuracy. This supports the use of complementary retrieval and answer objectives and makes the empirical case more convincing than a leaderboard comparison alone.

Critical weaknesses
  1. Unexplained discrepancies make parts of the evaluation difficult to reconcile. Figure 2 reports 247 two-page and 22 three-page MMLongBench-Doc questions, whereas Figure 10 reports 186 and 92, respectively. In addition, the full-data SlideVQA results in Figure 11 report F1/precision of 62.04/52.84, compared with 66.95/59.78 in Table 3. These differences may reflect distinct configurations or reporting errors, but the manuscript does not explain them. Clarifying the evaluation subsets and configurations is necessary to assess the evidence-count and scaling conclusions confidently.

  2. The multi-page-specific benefits of the central components remain insufficiently isolated. The principal pipeline and reward ablations report aggregate results on evaluation sets containing substantial numbers of single-page questions. Although Figure 10 provides retrieval F1 broken down by evidence-page count, it does not establish whether the gains from evidence verification and retrieval supervision persist specifically on multi-page QA. Reporting the Table 5 and Table 6 comparisons separately for single-page and multi-page questions would clarify how strongly the experiments support the intended cross-page use case.

Questions for authors
  1. Clarification of reported results: What explains the different evidence-page-count distributions in Figures 2 and 10, and the different full-data SlideVQA results in Figure 11 and Table 3? Please identify the corresponding checkpoints, evaluation subsets, and inference configurations.

  2. Clarification through a breakdown of the existing evaluation: Could the authors report the key reward and pipeline ablations in Tables 5 and 6 separately for questions requiring one versus multiple evidence pages, including both retrieval and answer metrics? This would clarify whether the aggregate gains extend to the cross-page setting emphasized by the paper.

  3. Clarification of the training algorithm: Section A.3.1 describes placing retrieval rewards at </search> tokens and the answer reward at the final token. How are these rewards converted into token-level advantages under GRPO? Please specify whether group normalization and credit assignment operate on trajectory-level returns or on action-specific returns; reward placement alone does not fully specify this mechanism.

2 comments
AuthorsOpenReview ↗

Rebuttal by Authors (Part 1)

We appreciate Reviewer Svya's recognition that our work addresses a consequential supervision problem, provides substantive evidence for the practical value of evidence-aware training targets, and separates several important sources of improvement through controlled experiments. We tried our best to address all the weaknesses and questions raised by the reviewer, as summarized below:

  • W1/Q1: We revised Section 4.1, Appendices A.4.4 and A.5.2, and Figures 10–11, and added Table 20.
  • W2: We revised Section 4.3 and added Table 19 in Appendix A.5.1.
  • Q2: We revised Section 4.3 and added Table 19 in Appendix A.5.1.
  • Q3: We clarified Section 3.3 and Appendix A.3.1 (Eq. 4).

W1/Q1: Figures 2/10: different evidence-page-count distributions. Figure 11/Table 3: different full-data SlideVQA F1/precision. Corresponding checkpoints, evaluation subsets, and inference configurations.

Response to W1/Q1: Thanks for the valuable comments! We revised Section 4.1, Appendices A.4.4 and A.5.2, and Figures 10–11, and added Table 20.

Table T9. Evaluation subsets and inference configurations.

ResultEvaluation subsetTrain / inference turns
Figure 10, left panelMMLongBench-Doc: 859 questions, grouped by pages read2 / 2
Figure 11, left panel, 100% dataSlideVQA SP+MP: 2,215 questions2 / 2
Tables 3 and 15SlideVQA MP: 567 questions2 / 3

Figure 10 groups questions differently from Figure 2 to analyze data-scaling gains by the number of pages read. Figure 2 reports how many gold evidence pages each question has (40 three-page questions. The count of 22 refers to four-page questions). Figure 10 groups the same 859 questions by the number of pages the model actually reads. For example, a question with two annotated evidence pages may lead the model to read three pages, while one with three annotated pages may involve reading only two. Grouping by gold evidence counts can mix searches that read different numbers of pages, making it harder to determine whether additional training data benefits searches involving more reading.

Figure 11 uses different evaluation settings from Table 3 to isolate the effect of training-data scale. Table 3 reports results on the 567 SlideVQA questions with multiple gold evidence pages and follows the configuration reported in Section 4.1 (Implementation) and Appendix A.4.4 (More Illustrations on the Comparison): two training search turns and three inference search turns.

Figure 11's retrieval panel evaluates all 2,215 SlideVQA questions. It fixes both the training and inference search budgets at two turns and uses the step-150 checkpoint from each run, to study the benefit of additional training data (varying only the training-data fraction) within the same allowed search budget. Table T9 summarizes the evaluation subsets and inference configurations.

W2: Multi-page-specific gains from evidence verification and retrieval supervision. Tables 5/6 comparisons separately for single-page and multi-page questions.

Response to W2: We revised Section 4.3 and added Table 19 in Appendix A.5.1.

Table T10. Pipeline and reward comparisons separately for single-page and multi-page QA: MMLongBench-Doc, EM (%).

Original comparisonVariantSingle-page: gold = 1Multi-page: gold ≥2
Pipeline — Table 6bFull pipeline (Ours)27.5120.34
Pipeline — Table 6bNaive generation23.4413.24
Pipeline — Table 6bW/o leave-one-page-out24.1016.80
Reward — Table 5aQA only / w/o PR25.1714.05
Reward — Table 5aPR only / w/o QA6.953.47
Reward — Table 5aQA+PR27.5120.34

Table T11. Pipeline comparison separately for single-page and multi-page QA: SlideVQA, EM (%). The multi-page results are copied from manuscript Table 6b.

VariantSingle-page: gold = 1Multi-page: gold ≥2 (Copied from manuscript Table 6b)
Full pipeline (Ours)49.5747.96
Naive generation47.2843.05
W/o leave-one-page-out48.1545.41

The gains from evidence verification and retrieval supervision persist on multi-page QA (Tables T10–T11). We retain the original policy checkpoints, inference settings, and scorer, and group questions by their original gold evidence-page annotations: one page versus at least two. Within each group, all compared policies use the same questions. On multi-page questions, retaining the leave-one-page-out check improves EM by 3.54 percentage on MMLongBench-Doc and 2.55 on SlideVQA. Adding retrieval supervision to QA-only training improves multi-page MMLongBench-Doc EM by 6.29 percentage.

AuthorsOpenReview ↗

Rebuttal by Authors (Part 2)

Q2: Tables 5/6 reward and pipeline ablations by one versus multiple evidence pages. Both retrieval and answer metrics.

Response to Q2: We revised Section 4.3 and added Table 19 in Appendix A.5.1.

Table T12. MMLongBench-Doc. EM results are copied from Table T10.

Reward / pipeline ablationDistinct gold evidence pagesRecallPrecisionF1EM
Full pipeline (Ours) (Table 6)164.8931.9738.7627.51
Naive generation (Table 6)161.8131.8439.0723.44
W/o LOO (Table 6)165.0930.3436.5924.10
QA only / w/o PR (Table 5)159.7532.9139.7725.17
PR only / w/o QA (Table 5)169.2028.7335.716.95
QA+PR (Table 5)164.8931.9738.7627.51
Full pipeline (Ours) (Table 6)≥255.3537.3142.2520.34
Naive generation (Table 6)≥248.3234.5136.9413.24
W/o LOO (Table 6)≥251.1134.7640.7216.80
QA only / w/o PR (Table 5)≥243.9237.0638.0414.05
PR only / w/o QA (Table 5)≥253.2634.4840.433.47
QA+PR (Table 5)≥255.3537.3142.2520.34

Table T13. SlideVQA. Multi-page F1 and EM are copied from the paper (Table 6b). Full pipeline (Ours) multi-page Recall and Precision are copied from the paper (Table 3). Single-page EM is copied from Table T11.

Reward / pipeline ablationDistinct gold evidence pagesRecallPrecisionF1EM
Full pipeline (Ours) (Table 6)189.4454.9060.5549.57
Naive generation (Table 6)187.5653.1558.2347.28
W/o LOO (Table 6)190.7854.1359.5048.15
Full pipeline (Ours) (Table 6)≥286.2859.7866.9547.96
Naive generation (Table 6)≥283.1357.0164.3243.05
W/o LOO (Table 6)≥284.2158.9765.5945.41

The aggregate retrieval and QA gains extend to questions requiring multiple evidence pages. Tables T12–T13 report the key reward and pipeline ablations in Tables 5 and 6 separately for questions requiring one versus multiple evidence pages, including both retrieval and answer metrics. Groups count distinct gold evidence pages. Within each group, all policy variants use the same questions and evaluation settings. All scores are percentages, and the EM results match Tables T10–T11.

For the pipeline comparisons, the full pipeline (Ours) improves Recall, F1, and EM over naive generation on multi-page questions in both benchmarks. Retaining the leave-one-page-out check improves both retrieval F1 and EM, with EM gains of 3.54 percentage on MMLongBench-Doc and 2.55 on SlideVQA.

For the reward comparison, adding retrieval supervision to QA-only training improves both retrieval F1 and EM on multi-page MMLongBench-Doc, by 4.21 and 6.29 percentage, respectively.

Q3: Retrieval rewards at </search> and answer reward at the final token → token-level advantages under GRPO. Group normalization and credit assignment: trajectory-level versus action-specific returns.

Response to Q3: Thanks for the valuable comments! We clarified Section 3.3 and Appendix A.3.1 (Eq. 4).

Group normalization is applied to action-boundary rewards, and token-level advantages accumulate the normalized rewards from the current and subsequent actions. Retrieval rewards are recorded at </search> tokens and the QA reward at the final response token.

For each question, we sample G=5G=5 trajectories. Let bi,jb_{i,j} denote the token position of reward event jj in trajectory ii, and let TiT_i be its number of search turns. At search turn tt, the boundary reward is ui,t=λPRγtri,tPRu_{i,t}=\lambda_{\mathrm{PR}}\gamma^t r^{\mathrm{PR}}_{i,t}, with γ=0.9\gamma=0.9. The terminal reward is ui,Ti+1=λQAriQAu_{i,T_i+1}=\lambda_{\mathrm{QA}}r^{\mathrm{QA}}_i. We compute the mean μq\mu_q and standard deviation σq\sigma_q over all search and terminal reward events pooled across the trajectories for question qq, including zero-valued reward events. We then normalize each boundary reward and compute token-level advantages as follows:

\widetilde{u}_{i,j} &= \frac{u_{i,j}-\mu_q}{\sigma_q+\epsilon},\\\\ A_{i,\ell} &= m^{\mathrm{policy}}_{i,\ell} \sum_{j:\\,b_{i,j}\geq\ell}\widetilde{u}_{i,j}. \end{aligned}$$ Here $m^{\mathrm{policy}}$ selects policy-generated tokens and excludes padding and tool observations, and $\epsilon>0$ is a numerical stabilizer. The factors $\gamma^t$ are already included in the retrieval rewards, so no additional discount is applied in the accumulation. Different search actions receive advantages based on their remaining rewards, enabling action-level credit assignment while preserving the final-answer objective. KL regularization is applied separately in the actor loss, with coefficient 0.001.
Reviewer_QLMUOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper asks how existing document collections can supply both answer targets and evidence labels for training search agents, observing that correct QA pairs do not imply correct evidence labels. DocArena is a four-stage automatic pipeline: (I) document structuring and indexing (144 DPI page images, MLLM-extracted structured semantics, E5-Mistral FAISS page index); (II) cross-page information-distribution profiling, which extracts factual units with concept-level keys and computes a distribution width w(c), where page-specific units (w=1) serve as retrieval anchors enforcing evidence exclusivity, w in {2,3} link pages, and w>3 supply shared context; (III) evidence-grounded reasoning-chain construction over four templates (chain, star, block tree, constraint puzzle) and five operators (LOOKUP, JOIN, FILTER, AGG, ARITH), with a global template counter to prevent type collapse and anchor stratification so the question carries a search clue without revealing pages or the answer; (IV) cascaded quality assurance — deterministic filters, MLLM correctness verification, and a leave-one-page-out necessity test that discards a sample if the question remains answerable after removing any labelled evidence page. Applied to CCpdf, this yields DocArena-79K: 79,623 QA pairs from 8,336 documents across 16 domains and 49 languages, with 92.3% multi-page evidence (mean 2.99 pages) and 63.2% of evidence pages containing tables, figures, or charts. The agent, Doc-Search, couples a Qwen2.5-7B-Instruct text policy with a ColPali visual retriever and a reading tool, trained by GRPO with R = lambda_QA * r_QA + lambda_PR * sum_t gamma^t r_PR^t, where r_PR^t is the recall of newly retrieved pages against the evidence set. Reported results: best Top-1/Recall in all six multimodal document scenarios and highest EM in four; on MMLongBench-Doc MP, Recall 61.38 and EM 24.18; average EM 36.95 across seven text QA benchmarks. Ablations: adding the retrieval reward raises MP Recall from 53.68 to 61.38, EM from 20.37 to 24.18, and cuts near-duplicate queries from 36.36% to 2.97%; with sample count, initialization, retriever, reward, and schedule fixed, the full pipeline gives EM 24.18 versus 21.39 without the necessity test, 19.25 for naive generation, and 20.08 for Search-R1 data at more than twice the size.

Critical strengths
  1. The leave-one-page-out necessity test directly addresses the paper's central observation, and its effect is measured under matched conditions. Table 6b trains the identical Doc-Search configuration on 79,623 samples from each pipeline variant: full pipeline EM 24.18, without the necessity test 21.39, naive generation 19.25 on MMLongBench-Doc MP, and 47.96 / 45.41 / 43.05 on SlideVQA. Because sample count, initialization, retriever, reward, and schedule are held fixed, this isolates data quality from data scale and training configuration, and the ordering matches the independent quality audit (100% human joint pass rate for the full pipeline versus 89%, 82%, and 39% when correctness, exclusivity, and leave-one-out are removed; the GPT-5.4 and Claude-Opus-4.7 judges give the same ordering). The claim that correct QA pairs are not correct retrieval labels is therefore verified rather than asserted.
  2. The training-data ablation is properly controlled and beats a larger text-trained corpus with fewer samples. Against Search-R1 data (NQ + HotpotQA, ~169K pairs, more than twice the size) inside the same Doc-Search infrastructure — same policy initialization, ColPali retriever, reward configuration, and training schedule — DocArena-79K improves Recall by 3.83 and EM by 4.10 on MMLongBench-Doc MP, and F1 by 2.76 and EM by 3.17 on SlideVQA. The reward ablation adds an independent and behaviourally interpretable effect: adding the page-recall term raises MP Recall from 53.68 to 61.38 and EM from 20.37 to 24.18 while reducing near-duplicate queries from 36.36% to 2.97%, whereas retrieval reward alone gives Recall 62.95 but EM 5.88. That pattern is what one would expect if evidence labels, not answer labels, teach coverage and query diversification.
Critical weaknesses
  1. The headline tables are not controlled experiments, but they are presented as the main evidence. Tables 1-4 compare released checkpoints trained in text-based Wikipedia environments with different data, rewards, and schedules under a shared inference interface. Section 4.1 states that "we do not attribute these gaps to any single component", yet the abstract and contribution list foreground best scores across six scenarios and the main text bolds six-of-six top retrieval results. NrDup in particular is used as evidence that retrieval supervision reduces repeated querying, but it is never optimized and varies sharply with the initial policy alone (SlideVQA: 21.21% from the Instruct checkpoint versus 0.95% from the Base checkpoint at comparable Recall, 86.28 versus 88.42), so it largely measures a backbone's lexical habits. Only Tables 5a and 6b are controlled, and there the absolute EM (24.18%) is low enough that the headline framing and the controlled evidence point to quite different effect sizes.
  2. Scalability, on which the pipeline's value depends, is not established, and the single-document filter makes training easier than evaluation. Appendix A.2.2 reports a 1.55% yield (250 QA pairs from 16,156 seed pages; 17.2 MLLM calls and 196 s per successful seed), but the paper never states how many seed pages, MLLM calls, or GPU/API hours produced the 79,623 released pairs — at that yield this is on the order of five million seed pages. More substantively, A.4.5 confirms that all evidence pages come from a single document, while the ColPali retriever at both training and inference searches the full collection without document restriction. Training tuples therefore carry an implicit "the target document is known" prior that evaluation does not provide, and cross-document multi-hop — the setting the paper also claims transfer to on 2WikiMultiHopQA — is never constructed.
Questions for authors

Clarifications (answers could change my assessment):

  1. What was the total cost of producing DocArena-79K: seed pages consumed, total MLLM calls, and GPU or API hours with Qwen2.5-VL-32B? Please reconcile this with the reported 1.55% yield. Without it, neither reproducibility nor the scalability claim can be assessed.
  2. Please report a cross-document evidence variant, or at least split the evaluation into intra- versus inter-document evidence questions. What happens to yield and downstream EM if the single-document filter is removed?
  3. How much of the NrDup reduction is attributable to retrieval supervision rather than to the backbone? The SlideVQA Base/Instruct contrast (0.95% versus 21.21% at comparable Recall) suggests a large backbone component. Also, A.4.4 lists training max response 500 / observation 1,000 tokens versus inference 1,000 / 2,500 — does this mismatch affect the table comparison?
  4. Table 6a shows that removing Layer 2 still leaves 80-89% of samples passing. Does this mean Layer 2 mostly removes already-defective samples, and which layer's removals actually drive the 24.18 -> 19.25 drop? A per-layer downstream training comparison would settle this.

Suggestions for future work (not affecting my assessment): Cross-document multi-hop construction (already noted as an extension); supervising precision, not only recall — Table 1 Precision (33.76) is below IKEA (34.27) and StepSearch (35.24), consistent with a recall-only retrieval reward; larger training search budgets, since Figure 2 shows the SlideVQA T=3 policy gaining +18.0 EM from two extra inference turns while training is capped at two; raising the 1.55% yield, whose largest losses are the answer-length filter (23.0%), the retrieval pre-filter (21.3%), and MLLM correctness verification (18.1%).

5 comments
AuthorsOpenReview ↗

Rebuttal by Authors (Part 1)

We appreciate Reviewer QLMU's recognition that our central observation is verified rather than asserted, the leave-one-page-out necessity test directly addresses this observation, and properly controlled experiments support the training value of our constructed data and evidence labels. We tried our best to address all the weaknesses and questions raised by the reviewer, as summarized below:

  • W1: We revised Sections 4.2–4.3, alongside existing Tables 5a and 6b.
  • W2/Q1: We added Tables 7–8 in Appendices A.2.2–A.2.3 and revised Appendix A.4.5 and Section 4.3.
  • Q2: We revised Appendix A.4.5 and added Table 13.
  • Q3: We revised Section 4.3 and added Table 12 in Appendix A.4.4.
  • Q4: We revised Section 4.3 and added Table 17 in Appendix A.5.1.
  • Suggestions for future work: We revised Appendix A.4.5 and added Table 13.

W1: Tables 1–4: uncontrolled headline evidence, different training data/rewards/schedules. NrDup: never optimized, sharp initialization dependence, and retrieval-supervision attribution. Controlled Tables 5a/6b: low absolute EM (24.18%) versus headline effect sizes.

Response to W1: We revised Sections 4.2–4.3, alongside existing Tables 5a and 6b.

Controlled experiments support our claims about the constructed data and retrieval supervision. Tables 5a and 6b provide the controlled comparisons supporting these contributions. Tables 1–4 report system-level performance under a shared inference interface, retaining differences in the checkpoints’ original training data, rewards, and schedules.

Even though NrDup is not directly optimized, adding retrieval supervision can reduce repeated querying in the controlled comparison with policy initialization fixed. In Table 5a, with the Instruct initialization fixed, NrDup falls from 36.36% to 2.97% on MMLongBench-Doc MP, while Recall rises from 53.68 to 61.38 and EM from 20.37 to 24.18. The Base/Instruct results also show sensitivity to initialization, so our attribution to retrieval supervision is based on the fixed-initialization comparison. We interpret NrDup as a measure of lexical query repetition alongside retrieval and QA performance.

The controlled results support our claim that DocArena data improve QA under fixed training settings. With initialization, retriever, rewards, and training schedule fixed, EM increases from 20.08% to 24.18% on MMLongBench-Doc and from 44.79% to 47.96% on SlideVQA, corresponding to gains of 4.10 and 3.17 percentage.

W2/Q1: DocArena-79K scalability: total seed pages, MLLM calls, GPU/API hours with Qwen2.5-VL-32B, and reconciliation with 1.55% yield. Single-document filter: easier training than full-collection evaluation and implicit “the target document is known” prior. 2WikiMultiHopQA transfer without cross-document multi-hop construction.

Response to W2/Q1: Thanks for the valuable comments! We added Tables 7–8 in Appendices A.2.2–A.2.3 and revised Appendix A.4.5 and Section 4.3.

Table T3. QA-and-evidence construction: dataset size, model use, and human annotation. The per-QA annotation time is a rough dataset-level normalization of formal annotation hours by the final dataset size.

Construction propertyMMLongBench-DocDocArena
QA pairs1,08279,623
MLLM useGPT-4o-assisted quality checkingOpen Qwen2.5-VL-32B-Instruct for generation and verification
Human QA/evidence annotationApproximately 150 person-hours. ≈8 min/QANo per-example manual authoring

Table T4. Cost of constructing the 79,623 released QA pairs. Generation and verification totals include rejected candidates and retries. One-time page preprocessing is reported separately.

Production quantityValue
Seed attempts / unique seed pages≈5,072,000 / 295,000
Total MLLM calls32,395,000
Generation and verification GPU-hours≈6,104 A100 GPU-hours
Hardware3–8 nodes, each with 8 A100 GPUs (24–64 GPUs)
Equivalent continuous-running time95–254 hours (4.0–10.6 days)
Production yield: released QA pairs / seed attempts1.57% (79,623 / ≈5,072,000)
Amortized cost per retained QA pair≈407 MLLM calls. 0.0767 GPU-hours (≈4.6 GPU-minutes)
One-time page extraction budget≈300,000 extraction calls. 184–736 GPU-hours

Scalability: DocArena has produced 79,623 QA pairs without per-example human annotation, using the locally deployable Qwen2.5-VL-32B-Instruct. This open model automatically generates and verifies questions, answers, and evidence labels, so expanding the dataset does not require per-example human authoring or access to proprietary generation models. Table T3 compares dataset size, model use, and human annotation requirements with MMLongBench-Doc.

AuthorsOpenReview ↗

Rebuttal by Authors (Part 2)

Total Cost of Producing DocArena-79K: Table T4 reports the seed pages consumed, total MLLM calls, and GPU-hours used to produce DocArena-79K, including rejected candidates and retries. Amortized over the released dataset, generation and verification cost approximately 4.6 A100 GPU-minutes per retained QA pair, including unsuccessful attempts. This provides a concrete compute budget for expanding automated construction without per-example human annotation, with one-time preprocessing reused across seed attempts.

The generation-and-verification cost is approximately 6,104 GPU-hours, corresponding to 4.0–10.6 days of continuous allocation on 24–64 GPUs. This supports expansion through automated generation and verification within a concrete compute budget. The production yield is 1.57% for 79,623 released QA pairs over approximately 5.072M seed attempts, close to the 1.55% yield on the separate analysis subset. The analysis-subset means—17.2 calls / 196 seconds for successful seeds and 6.6 calls / 147 seconds for failed seeds—describe per-seed outcomes.

More Illustration on the Single-Document Filter: Restricting a QA pair’s evidence to one document (Single-Document Filter) does not tell the policy which document to retrieve. The policy is not given the gold document ID or evidence-page list in advance. Therefor, the policy must still discover the evidence through full-collection search at both training and inference.

About Transfer to 2WikiMultiHopQA: The 2WikiMultiHopQA evaluation tests whether a policy trained on within-document questions can answer cross-document multi-hop questions. The resulting policy reaches 36.82 EM on 2WikiMultiHopQA despite DocArena-79K being constructed within individual documents. Our contribution is to construct effective answer and evidence targets for learning search, supported by the controlled training-data comparisons in Table 6b.

Q2: Cross-document evidence variant or intra-/inter-document evaluation split. Single-document filter removal: yield and downstream EM.

Response to Q2: Thanks for the valuable comments! We revised Appendix A.4.5 and added Table 13.

Table T5. Performance on intra- versus inter-document evidence questions (100 questions per group).

Existing policyIntra-document: Retrieval F1 / QA EM (%)Inter-document: Retrieval F1 / QA EM (%)
DocArena, full pipeline (Ours)44.54 / 26.5441.67 / 19.50
Naive generation39.60 / 17.5134.14 / 15.20

Table T5 reports performance on intra- versus inter-document evidence questions constructed from MMLongBench-Doc. Using Qwen2.5-VL-32B-Instruct, we construct comparison and difference questions requiring two evidence pages from either one or two original documents. Answers are constructed to require information from both evidence pages. The set contains 100 intra-document and 100 inter-document questions. The existing full and naive policies use the same full-collection retriever, reader, inference budget, and EM scorer. The full-minus-naive EM difference on inter-document questions is 4.30 percentage (19.50% vs. 15.20%).

Removing the single-document filter increases the full-set production yield from 1.57% to 2.67%. For downstream training, we recover previously generated questions, answers, and support-page labels from archived construction runs that allowed cross-document evidence. We select these archived candidates without the single-document constraint while retaining the other verification requirements. No new QA pairs are generated. The resulting retained examples are used to train the filter-off policy. With policy initialization, retriever, rewards, and training schedule fixed, the filter-on/filter-off EM scores are 24.18 / 24.00 on MMLongBench-Doc, 47.96 / 42.17 on SlideVQA, and 36.82 / 39.20 on 2WikiMultiHopQA. Removing the filter decreases EM by 0.18 percentage on MMLongBench-Doc and increases it by 2.38 on 2WikiMultiHopQA, while SlideVQA EM decreases by 5.79 percentage. The cross-document variant has a higher score on 2WikiMultiHopQA and a lower score on SlideVQA.

Q3: Amount of NrDup reduction attributable to retrieval supervision vs. backbone, given the SlideVQA Base/Instruct contrast at comparable Recall. Training–inference token mismatch (500/1,000 vs. 1,000/2,500 response/observation tokens): effect on table comparison.

Response to Q3: We revised Section 4.3 and added Table 12 in Appendix A.4.4.

Table T6. Reward ablation on MMLongBench-Doc MP, copied directly from manuscript Table 5a.

RewardRecall ↑Precision ↑F1 ↑NrDup ↓EM ↑Mod ↑PNLS ↑
w/o PR53.6834.1938.4936.3620.3717.2328.29
w/o QA62.9530.7437.195.575.884.4912.04
PR+QA61.3833.7639.682.9724.1819.4333.12
AuthorsOpenReview ↗

Rebuttal by Authors (Part 3)

Table T7. Comparison under both inference token budgets (MMLongBench-Doc MP). The 1,000/2,500-budget entries are copied unchanged from manuscript Tables 1 and 5a.

PolicyInference response 500 / observation 1,000 (matching Doc-Search's training token limits)Inference response 1,000 / observation 2,500 (Copied from manuscript Tables 1 and 5a)
Doc-Search, w/o PR34.27 / 51.60 / 37.56 / 20.0036.36 / 53.68 / 38.49 / 20.37
Doc-Search, PR+QA2.18 / 59.00 / 39.40 / 25.252.97 / 61.38 / 39.68 / 24.18
IKEA23.14 / 52.65 / 36.40 / 23.1923.62 / 54.43 / 38.41 / 21.03

With the Instruct backbone fixed, retrieval supervision accounts for a 33.39-percentage-point NrDup reduction (36.36% → 2.97%) on MMLongBench-Doc MP. As copied from the manuscript, Table T6 isolates the retrieval-supervision contribution by comparing w/o PR with PR+QA on the same backbone. We agree that the SlideVQA Base/Instruct contrast shows a substantial backbone effect, yet the fixed-backbone comparison still supports retrieval supervision's contribution to reducing NrDup. The training–inference token mismatch changes absolute scores but does not change the NrDup or EM rankings. Table T7 performs the evaluation with different token budgets. The 500/1,000 setting matches Doc-Search's training limits. The other inference settings, including the search-turn limit, are fixed.

Q4: 80–89% pass rates without Layer 2: mostly already-defective samples removed by Layer 2? Layer contributions to the 24.18 → 19.25 EM drop. Per-layer downstream training comparison.

Response to Q4: Thanks for the valuable comments! We revised Section 4.3 and added Table 17 in Appendix A.5.1.

Table T8. Training with all examples retained without Layer 2 (new), alongside existing results from manuscript Table 6b. Only the w/o Layer 2 row is new. The other rows are copied from manuscript Table 6b.

Policy training dataMMLongBench-Doc EM (%)SlideVQA EM (%)
Full pipeline (Ours. Copied from manuscript Table 6b)24.1847.96
W/o Layer 2 (correctness verification)22.6846.60
W/o Layer 3 (LOO) (Copied from manuscript Table 6b)21.3945.41
Naive generation (Copied from manuscript Table 6b)19.2543.05

The 11–20% (1 − (80–89%)) is the audit failure rate among samples retained without Layer 2, not Layer 2’s rejection rate. These audits do not establish whether most samples rejected by Layer 2 are defective.

Table T8 adds a downstream training comparison for Layer 2, using the Full policy from manuscript Table 6b as the reference. We recover the original generated candidates from the construction logs, omit Layer 2, and apply the remaining checks, including LOO. The w/o Layer 2 policy uses all retained examples and thus more training data than Full, with the same policy initialization, retriever, rewards, training schedule, and evaluation settings. Using all examples retained after removing Layer 2, the policy scores 1.50 percentage lower on MMLongBench-Doc and 1.36 lower on SlideVQA than Full. These differences describe this training configuration with more retained examples than Full.

For the per-layer downstream training comparison: A complete comparison across all three layers requires separate policy training and benchmark evaluation for each variant. Given the limited rebuttal time and available computational resources, we prioritize Layer 2 and will continue the remaining per-layer experiments to provide a more detailed analysis.

Suggestions for future work: Suggestions not affecting the assessment: cross-document multi-hop construction, supervising precision as well as recall, larger training search budgets, and raising the 1.55% yield.

Response to Suggestions for future work: We revised Appendix A.4.5 and added Table 13.

For “Cross-document multi-hop construction,” our response to Q2 reports the supplementary intra- and inter-document evaluation in Table T5, together with downstream results. For “raising the 1.55% yield,” our response to Q2 reports an increase in the full-set production yield from 1.57% to 2.67% after removing the single-document filter.

We will carefully consider “supervising precision, not only recall”, together with further extensions to cross-document construction and improvements to curation yield, and begin preparing the corresponding experiments in our continued work. The experiments on precision supervision require additional policy training and evaluation. Given the limited rebuttal time and computational resources, we are not yet able to provide results for this direction.

Figure 2 examines training budgets of up to three turns for SlideVQA and four for MMLongBench-Doc. We much appreciate the suggestion and will extend this analysis to larger training budgets. We thank Reviewer QLMU again for the insightful comments and valuable guidance for extending this work.

Reviewer_SYwfOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper proposes DocArena, an automated pipeline for constructing search training data from raw multimodal documents. The pipeline jointly generates questions, answers, and evidence-page labels. It uses cross-page fact-distribution profiling to guide evidence selection, followed by answer-consistency and leave-one-page-out checks to assess evidence sufficiency and page-level necessity. Experiments cover six multimodal document scenarios and seven text-based QA benchmarks. Human audits, data-construction ablations, and reward ablations evaluate the quality of the generated supervision and its downstream training value.

Critical strengths
  1. The paper addresses an important problem in retrieval training data and provides independently validated evidence for its solution.

The paper explicitly distinguishes answer correctness from evidence-label validity and makes evidence verification a central part of data construction. Independent human audits, external model judgments, and pipeline ablations jointly support the reliability of the generated labels. In particular, two annotators with access to the complete source documents report joint audit pass rates of 99% and 100%, respectively. These results provide credible support for the pipeline’s practical value as a method for automatically constructing search training targets.

  1. Controlled experiments provide a convincing connection between data quality and downstream performance.

Table 6b compares data-construction variants using the same source documents, sample count, and training configuration. The complete pipeline outperforms both naive QA generation and the variant without leave-one-page-out verification on two document benchmarks, showing that evidence verification improves downstream training outcomes. The reward ablations further demonstrate that evidence feedback improves retrieval coverage and final QA performance, while evidence rewards alone are insufficient to ensure answer quality. These experiments provide the strongest support for the paper’s main claims.

Critical weaknesses
  1. The incremental contribution of the core design relative to the closest related work requires clearer positioning and isolation.

StepSearch already studies process rewards based on information gained from newly acquired evidence. CiteVQA constructs QA pairs and evidence from raw documents and identifies crucial evidence through element-level masking. The paper should more clearly explain how its page-level necessity checks and overall construction pipeline differ from these approaches. In particular, Table 6a supports the sample-quality benefits of cross-page fact-distribution-guided selection, but the downstream experiments do not isolate this component while retaining the other generation and verification stages at the same training-data scale. Consequently, the current evidence supports the complete pipeline more strongly than the independent contribution of this particular design.

  1. Some analyses supporting the data-scaling and complex multi-page claims contain unexplained inconsistencies.

Figure 10 reports evidence-page-count groups for MMLongBench-Doc that differ from Figure 2, despite both containing 859 questions. For example, the two-page and three-page groups contain 247 and 40 questions in Figure 2, compared with 186 and 92 in Figure 10. In addition, the 100%-data endpoint in Figure 11 reports SlideVQA retrieval F1/Precision of 62.04/52.84, whereas Tables 3 and 11 report 66.95/59.78. These differences may reflect different annotation processing, evaluation subsets, or training runs, but I could not identify an explanation in the manuscript. Clarifying these settings is necessary to assess the scaling conclusions and the reported benefits on questions requiring multiple evidence pages.

Questions for authors
  1. Clarification of the evaluation protocol. Figure 11 selects 249 questions using the full-data model’s EM > 0.5, yet reports an EM of 91.3% for that model on the selected subset. If selection and evaluation use the same predictions and per-question EM is binary, the resulting subset EM should be 100%. Could the authors specify whether this analysis instead uses repeated-generation averages, independent predictions for selection and evaluation, or another aggregation procedure?

  2. Clarification of novelty and component contributions. Relative to CiteVQA’s evidence-masking verification, what are the principal methodological advances of the proposed leave-one-page-out check and overall data-construction pipeline? When reasoning templates, answer verification, and leave-one-page-out checks are retained, how does removing only cross-page fact-distribution-guided selection affect data quality and downstream performance under equal-scale training? This comparison would materially affect my assessment of the contribution of this core design.

  3. Additional validation of the existing cross-page capability claim. On fixed evaluation subsets requiring at least two evidence pages, and particularly three or more pages, does the complete pipeline consistently improve final QA accuracy over naive generation and the variant without leave-one-page-out verification? Please provide results under a consistent evaluation protocol, together with appropriate uncertainty estimates, to establish whether the aggregate gains extend to questions that require integrating evidence across multiple pages.

2 comments
AuthorsOpenReview ↗

Rebuttal by Authors (Part 1)

We appreciate Reviewer SYwf's recognition that this work addresses an important problem in retrieval training data, shows practical value as a method for automatically constructing search training targets, and provides convincing supports for papers's main claims. We tried our best to address all the weaknesses and questions raised by the reviewer, as summarized below:

  • W1/Q2: We revised Sections 2, 3.2, and 4.3 and added Table 16 in Appendix A.5.1.
  • W2: We revised Section 4.1, Appendices A.4.4 and A.5.2, and Figures 10–11, and added Table 20.
  • Q1: We revised Section 4.1, Appendices A.4 and A.5.2, and Figure 11.
  • Q3: We revised Section 4.3 and added Table 18 in Appendix A.5.1.

W1/Q2: Differences from StepSearch’s information-gain process rewards and CiteVQA’s element-level masking. Page-level necessity checks and overall construction pipeline. Removing only cross-page fact-distribution-guided selection: data quality and downstream performance, equal-scale training, reasoning templates/answer verification/leave-one-page-out checks retained.

Response to W1/Q2: Thanks for the valuable comments! We revised Sections 2, 3.2, and 4.3 and added Table 16 in Appendix A.5.1.

Table T1. Independent contribution of cross-page fact-distribution-guided selection, with the other generation and verification stages retained at the same training-data scale.

Policy training dataMMLongBench-Doc EM (%)SlideVQA EM (%)
Full pipeline (Ours. Copied from manuscript Table 6b)24.1847.96
Without distribution-guided selection22.4246.06

Methodological Advances of the Proposed DocArena Relative to StepSearch: DocArena can create training examples for a new document collection without an existing QA dataset. StepSearch instead needs to start with an existing multi-hop QA dataset and uses its decomposed questions to construct reference search queries. Both methods has progress rewards, but StepSearch computes progress rewards against gold documents. DocArena computes them against automatically constructed evidence pages.

Methodological Advances of the Proposed DocArena Relative to CiteVQA: DocArena’s leave-one-page-out necessity check removes each labeled evidence page and rejects the QA pair if the question remains answerable. DocArena verifies that the same labeled page set is sufficient to answer the question and every page is necessary under the verifier. By comparison, CiteVQA’s element-level masking checks the necessity of individual citation elements, but does not require the same page-level evidence set to satisfy both conditions.

The overall pipelines in both works also differ in:

  • Profiling purpose: DocArena uses cross-page fact-distribution profiles to embed page-specific search clues in questions. CiteVQA uses semantic profiles to link documents and sections.
  • Evidence granularity: DocArena labels the evidence pages a search agent should retrieve. CiteVQA labels the bounding-box regions an answering model should cite.

Experiment isolating cross-page fact-distribution-guided selection: Table T1 isolates this component while retaining the other generation and verification stages at the same training-data scale. For downstream performance, cross-page fact-distribution-guided selection improves EM by 1.76 percentage on MMLongBench-Doc and 1.90 on SlideVQA under equal-scale training. Note that this comparison retains reasoning templates, answer verification, and leave-one-page-out checks. Only cross-page fact-distribution-guided selection is removed. Policy initialization, retriever, rewards, and training settings are fixed.

For data quality, the human joint question–answer–evidence pass rate in Table 6a is 100% for the full pipeline (Ours) and 82% without distribution-guided selection, based on 100 audited examples per variant.

AuthorsOpenReview ↗

Rebuttal by Authors (Part 2)

W2: Fig. 2/10: different evidence-page-count groups for the same 859 questions. Fig. 11 vs. Tables 3/11: different full-data SlideVQA F1/Precision. Evaluation settings underlying scaling and multi-page analyses.

Response to W2: We revised Section 4.1, Appendices A.4.4 and A.5.2, and Fig. 10–11, and added Table 20.

Fig. 10 groups questions by different quantities from Fig.2 for its analysis aim. Fig. 2 reports how many gold evidence pages each question has. Fig. 10 groups the same 859 questions by reporting how many pages the model actually reads for the same questions. For example, a question with two annotated evidence pages may lead the model to read three pages, while one with three annotated pages may involve reading only two. The reason why Fig.10 groups differently from Fig.2 owns to the analysis aim: Grouping by gold evidence counts can mix searches that read different numbers of pages, making it harder to determine whether additional training data benefits searches involving more reading.

Figure 11 uses different evaluation settings from Table 3 to isolate the effect of training-data scale. Tables 3 and 15 follow the configuration reported in Section 4.1 (Implementation) and Appendix A.4.4 (More Illustrations on the Comparison): two training search turns and three inference search turns. Instead, Figure 11 fixes both the training and inference search budgets at two turns and uses the step-150 checkpoint from each run, to study the benefit of additional training data (varying only the training-data fraction) within the same allowed search budget.

Q1: Fig. 11: 249 questions selected by full-data EM > 0.5, subset EM 91.3% versus 100% under binary EM and the same predictions. Repeated-generation averages, independent selection/evaluation predictions, or another aggregation procedure.

Response to Q1: We revised Section 4.1, Appendices A.4 and A.5.2, and Fig. 11.

In Fig. 11 (middle), we use EM to denote a score based on exact matching of normalized tokens (token F1), computed using the official SlideVQA evaluation code. Each question has one prediction and one score, and the reported value is the arithmetic mean of the per-question scores.

We use this same official scoring procedure throughout our SlideVQA evaluation. Token F1 captures improvements in answer-token overlap with the reference, even when answers do not match exactly. By comparison, binary exact match would assign zero to all non-exact answers and therefore fail to distinguish these partial improvements. The training QA reward remains binary and is a different quantity from the continuous score in Fig. 11 (middle).

Specifically, we compare all training-data fractions on the same fixed set S\mathcal S of 249 multi-page questions selected by the full-data model’s score s_{100\\%} > 0.5. Selection and evaluation use the same predictions. Selecting scores above 0.5 does not require every selected score to equal 1 or the subset mean to equal 100%, so a mean EM of 91.33% is consistent with this selection rule. This panel reports conditional differences on questions selected by the full-data model.

Q3: Consistent QA gains on fixed subsets requiring ≥2 and ≥3 evidence pages: complete pipeline vs. naive generation and w/o leave-one-page-out verification. Consistent evaluation protocol and appropriate uncertainty estimates.

Response to Q3: Thanks for the valuable comments! We revised Section 4.3 and added Table 18 in Appendix A.5.1.

Table T2. QA scores on fixed ≥2-page and ≥3-page subsets. Brackets indicate 95% confidence intervals for the corresponding differences.

Training data / comparisonMMLong Table 6b (N=859)MMLong gold ≥2 (N=360)MMLong gold ≥3 (N=113)SlideVQA gold ≥2 (N=567)SlideVQA gold ≥3 (N=5)
Full pipeline (Ours. Copied from revised manuscript Table 18)24.1820.3412.4647.9640.00
Naive generation19.2513.246.0043.0520.00
W/o leave-one-page-out21.3916.809.5045.4120.00
Full − naive+4.93 [-1.07, +6.53]+7.10+6.46 [-8.0, +20.00]+4.91 [-2.09, +10.67]+20.00 [-100, +100]
Full − w/o LOO+2.79 [-3.21, +4.67]+3.54+2.96 [-9.5, +16.5]+2.55 [-3.35, +11.25]+20.00 [-100, +100]

Table T2 provides additional validation of the existing cross-page capability under a consistent evaluation protocol with appropriate uncertainty estimates. We retain the Table 6b policies and use the same question IDs, inference settings, and scorer within each subset. The full pipeline (Ours) achieves higher QA scores than both variants on every fixed ≥2-page and ≥3-page subset, including gains of 6.46 and 2.96 percentage over naive generation and w/o LOO on MMLongBench-Doc ≥3-page questions. These results extend the observed gains to multi-page QA, although the reported 95% confidence intervals include zero and the SlideVQA ≥3-page subset contains only five questions.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Sincere Appreciate the Reviewers’ Feedback and Summary

Dear Area Chairs and Reviewers,

We would like to thank the Area Chairs and all reviewers for their time and effort. We appreciate the reviewers’ recognition that this work addresses an important problem (SYwf, Svya), demonstrating the practical value of the evidence-aware training targets (Svya) and the automatic construction (SYwf). The reviewers also highlighted convincing experimental support for the main claims (SYwf), properly controlled ablations (QLMU), and controlled experiments that separate several important sources of improvement (Svya).

In response to the reviewers’ comments and suggestions, we further discussed DocArena’s differences from StepSearch and CiteVQA (Sections 2 and 3.2). We clarified the reward and token-level advantage calculations (Section 3.3 and Appendix A.3.1), as well as experimental settings and result interpretation (Sections 4.1–4.3, Appendices A.4 and A.5.2, and Figure 11). We supplemented the discussion of data-pipeline components (Section 4.3 and Appendices A.4.5 and A.5.1, Tables 16–17 added). We provided additional analyses of evidence-page counts (Section 4.3, Appendices A.5.1–A.5.2, and Figure 10, Tables 18–19 added). We added an inference-token-budget comparison (Section 4.3 and Appendix A.4.4, Table 12 added). We reported data-construction scale and costs (Appendices A.2.2–A.2.3, Table 8 added). We provided a cross-document discussion (Section 4.3 and Appendix A.4.5, Table 13 added).

Once again, we thank all the reviewers for their thoughtful and constructive feedback, which has been instrumental in enhancing the quality and impact of this submission. We hope this summary could assist the evaluation of this submission.

Best regards,

Authors of Submission 8160

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.