- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 3
Read review
Paper summary
In this paper, the goal is to reduce the high input cost of multimodal document RAG. I would consider this work a multimodal extension of Provence, which performs zero-cost text pruning during reranking. MMProvence extends this idea to both text and page images. To do this, for each query-page pair, the reranker jointly predicts a relevance score, a relevant image region, and the relevant text sentences. The score is used for reranking, while the predicted region and sentences are used to prune the selected pages before they are passed to the final VLM, without requiring an additional pruning-model call. I would also to clarify that the ‘zero-cost text pruning’, specifically means that pruning does not require an additional model call; it does not mean that pruning has no computational cost. In fact, the reranking stage becomes more expensive because the model must generate the bounding box and relevant sentences in addition to the relevance score.
Critical strengths
- The proposed approach is simple, as reranking and multimodal context pruning are performed within the same model call.
- The contextual bounding-box idea is useful.
- The paper provides extensive ablations • The method achieves substantial context reduction.
- MMProvence-2B reduces improves the efficiency while preserving the performance for ViDoRe V3 benchmark dataset.
Critical weaknesses
-
Limited novelty and missing closely related work/comparison. A key contribution of the paper is to reduce unnecessary visual context by identifying the query-relevant region of a retrieved page instead of passing the full-page image to the generator. A closely related idea has already been explored in RegionRAG[1], which performs region-level retrieval to expose query-relevant visual regions rather than full document pages. The main difference is that MMProvence integrates visual-region selection, text pruning, and reranking into the same call. However, RegionRAG is neither cited nor compared against. A direct comparison would help clarify whether MMProvence provides a meaningful improvement over existing region-level visual RAG approaches, particularly in terms of answer quality, context reduction, and end-to-end efficiency.
-
Limited evaluation scope. The end-to-end QA experiments are conducted only on the eight domains of ViDoRe V3. related work such as RegionRAG has been evaluated across a broader range of document QA settings. Evaluating MMProvence on additional datasets such as DocVQA, InfographicsVQA, SlideVQA, or PlotQA would provide stronger evidence that the proposed pruning approach generalizes beyond ViDoRe V3.
-
The term “zero-cost” can be misleading. It is true that MMProvence does not require an additional pruning-model call. However, pruning is not computationally free: generating the bounding box and relevant sentences makes the reranking stage more expensive. For example, MMProvence-2B reduces the generator input by about 75%, but provides only around 11–13% end-to-end latency reduction in the main Qwen3.5-9B generator setting. Moreover, the larger MMProvence-9B variant is actually slower than the full-context pipeline. Therefore, I think it would be more precise to describe the method as“pruning without an additional model call” rather than simply “zero-cost pruning.”
[1] Y. Li, Z. Lu, Z. Liu, Y. Sun, C. Liu, and H. Xie, “RegionRAG: Region-level Retrieval-Augmented Generation for Visual Document Understanding, on arxiv since december 2025, and AAAI 2026.
Questions for authors
2 comments
Dear Reviewer,
Thanks for taking the time to review our work. In the rebuttal, we have added the RegionRAG results in both the original paper setting as well as in a more fair configuration. Yet, our model still outperforms it. Additionally, we have reported results on DocVQA and InfoVQA. Since there is less than 24 hours remaining till the end of the discussion period, we would appreciate it if you can take another look at our responses and let us know if there are any remaining concerns.
We thank the reviewer for their feedback. Below, we reply to the weakness:
Limited Novelty: Our main contribution is integrating the pruning inside the reranker which results in efficiency gains since we don't have to load an additional pruning model in memory or perform pre-filling of the Query+Context prompt. Our sentence index mmprovence full pipeline is 23%/33% faster (with/without thinking) and uses 33% less compute. To our best knowledge, no prior work integrated pruning in the reranker for multimodal RAG applications.
Missing related work and comparison with RegioRAG: We have added RegionRAG as a baseline to the paper. Below are the experimental results against RegionRAG on Vidore benchmark.
For fairness, we compared against RegionRAG in 2 setups: S1 uses the full pipeline from their paper (RegionRET+RegionRAG), while S2 uses MMprovence-2B as the reranker to isolate the cropping effect of both models. We tried our best to choose the RegionRAG threshold that matches the compression rate of MMProvence. We outperform both approaches, especially S1 where we outperform it by 28% relative improvement.
Regarding runtime, our model is 20% faster than the closest RegionRAG setup while outperforming it. The details are below:
Limited evaluation scope: We include MP-DocVQA & InfoVQA from the VisRAG project with image input (no OCR). Below are the results of our model compared to the full context baseline. We will include the results in the main paper. On DocVQA our model achieves ~80% token reduction with minimal drop in performance (0.968 -> 0.939). On the InfoVQA, we achieve ~75% compression while suffering a bigger performance drop (0.93 -> 0.85). Such difference in performance can be attributed to the training data having little infographic samples.
MP-DocVQA
InfoVQA
Zero-Cost claim: We agree that the term needs clarification. By "zero-cost" we mean that the pruning stage needs no additional model call. The box and the relevant sentences are produced by the reranking call itself, so no pruning model is loaded in memory and the Query+Context prompt is not pre-filled a second time. The cost of pruning moves into the reranking stage as output tokens. This is why reranking takes 0.75 / 1.48 / 5.47 s longer at 2B / 4B / 9B, and why MMProvence-9B ends up slower than the full-context pipeline. Most of these output tokens are copied sentences. To address this, we trained a sentence-index variant of MMProvence-2B (Table 5, last row) that outputs sentence numbers instead of copying the sentences. It cuts the reranking output from 9,578 to 2,700 tokens per query (−72%). Below are the efficiency results with the Qwen3.5-9B generator on H100 (median seconds per query at batch size 1, 392 queries; Δ against the full-context pipeline):
With the sentence-index variant, the full pipeline is 23%/33% faster (with/without thinking) and uses 33% less compute. Outputting sentence numbers does not hurt answer quality: −0.002 ± 0.011 LLMeval compared to the copied-sentence model.
We will follow the reviewer's suggestion and use "pruning without an additional model call" in the paper. We will define the term at first use and state clearly in RQ5 that the end-to-end gains depend on model size.