- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes DocTrek, an agentic RL framework for multimodal long-document QA. It constructs a hierarchical document tree with typed multimodal elements and trains an agent to navigate this structure using dense process rewards assigned at action-end token positions. Experiments on four benchmarks demonstrate improvements over existing RAG and agentic systems.
Critical strengths
- Formulating multimodal long-document QA as a Markov decision process over a document tree is an interesting idea.
- DocTrek surpasses the chosen baselines across four multimodal long-document QA benchmarks.
Critical weaknesses
- The experiments do not sufficiently validate the proposed method. Several baselines use different model backbones, making direct comparisons less conclusive. Also, the individual reward components are not adequately ablated, leaving their respective contributions unclear.
- The improvement over “w/o Dense Reward” is relatively small (from 0.446 to 0.448 in EM), while average tool calls increase substantially (from 18.43 to 33.20). This raises concerns about whether the modest effectiveness gains are worth the additional inference cost and whether the proposed method achieves a favorable effectiveness-efficiency trade-off.
Minor
- Citations should use \citep{} for better readability.
- The main paper could be more self-contained. For example, the experimental section should at least explicitly list the names of baselines rather than only describing categories and leaving all the details to the appendix.
Questions for authors
- Could the authors provide separate ablations for the two per-turn reward components and ? Further ablations of the individual shaping terms of and sensitivity analyses of the reward coefficients would also help establish their contributions.
- Could the authors clarify whether “w/o Dense Reward” removes the process rewards entirely or simply places them at the final token? The descriptions in the main text and Table 2 caption appear inconsistent, and these two settings test different aspects of the proposed method.
2 comments
Thank you for the clarifications. I disagree that the official configurations of prior baselines necessarily represent their best-performing settings, as backbone choices are often constrained by model availability at the time of publication. However, given the inclusion of recent baselines and the substantial reproduction costs, I can accept the current comparisons. Also, the lack of ablations for individual reward components still remains a major concern. I understand the computational constraints during rebuttal, but they do not justify the absence of experiments essential to validating the proposed reward design and the effectiveness-efficiency trade-off of the method. Therefore, I will maintain my original rating.
Thank you for recognizing the tree-based MDP and the results across four benchmarks. We believe the current evidence supports the combined system and its measured within-system components more strongly than the review's characterization of insufficient validation suggests.
Do differing backbones make the results inconclusive? Table 1 evaluates complete systems under their published configurations and does not isolate backbone or hierarchy. Each baseline's prompts and tool-invocation protocol are closely coupled to the model used in its official implementation; replacing that model alone would change the method's operating conditions rather than faithfully reproduce its best-performing implementation. We therefore compare against the baselines in their official, best-performing configurations. Model size alone is an insufficient account of the pattern: DocTrek (Qwen3-VL-8B) reaches 0.562 average, compared with 0.455 for the 8B MDocAgent and 0.539 for GPT-4o-based DMAP. Against the latter it leads on MMLongBench-Doc (0.501 vs. 0.432), PaperTab (0.422 vs. 0.390), and FetaTab (0.750 vs. 0.725), although DMAP leads on LongDocURL. These are system-level comparisons, not backbone-controlled causal estimates. Separately, the within-DocTrek comparisons use the same main-agent setting: SFT reaches 0.482 accuracy, final-token reward placement 0.488, and action-end placement 0.501; Table 3 further shows 0.486 without cross-page merging and 0.414 without vector retrieval. Therefore the experimental support does not rest solely on comparing 8B with 7B external agents.
Are the reward terms unvalidated? Equation (6) distinguishes the gold-evidence retrieval reward Rret from the heuristic shaping reward Rpri, with coefficients specified in Appendix C. Table 2 keeps these components fixed and changes their token positions, yielding 0.488 → 0.501 accuracy and the retrieval gains in Appendix E; it tests the paper's central claim about action-end credit assignment. The agent's training pipeline requires approximately 120 hours on 24 H20 GPUs. Under this training-time constraint, separately retraining variants for Rret, Rpri, each shaping term, and multiple coefficient settings is beyond the scope of the present study. We will investigate these individual contributions and coefficient sensitivity in future work.
Is the effectiveness–cost trade-off unfavorable? The reviewer is right to distinguish EM (0.446 → 0.448) from accuracy (0.488 → 0.501) and tool calls (18.43 → 33.20). The gain is not restricted to a 0.2-point EM difference: the first ten retrieved evidence leaves improve by 9.13 points in recall and 1.52 points in precision, with improvement already visible at five leaves (Appendix E). This is evidence of better evidence localization at a fixed retrieval cutoff, while the end-task improvement and extra navigation are both reported transparently. It is not a claim that accuracy per call or latency is improved. The 124 s/query online cost is measured on eight H20 GPUs; the offline 13.8 s/page tree construction is reused across questions. Calling the trade-off categorically unfavorable from EM and calls alone overlooks the accuracy, early retrieval quality, and amortized preprocessing reported in the paper.
What exactly does “w/o Dense Reward” test, and can the presentation be clarified? It retains all reward components but puts their accumulated value at the final token, as the Table 2 caption states. Thus it isolates action-end versus final-token placement, not the deletion of process rewards. We will replace the imprecise “terminal, outcome-only” phrase in §5.3 to match the actual experiment. We will also name the baseline methods in §5.1 and use parenthetical citations where appropriate; these presentation edits do not alter the results.