acceptodds
Withdrawn from ICLR 2027

Loud and Clear: Making Visual Evidence Count in Multimodal Document RAG

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Multimodal document retrieval-augmented generation (RAG) has focused largely on retrieval, yet its answers depend on how well a vision-language model reads the retrieved pages. We find that evidence in retrieved pages remains underused even when the correct pages are retrieved. On SciEGQA oracle pages, five prior methods give the generator 0.7K to 2.7K visual tokens per query on average under a common image processor, yet raising the page-only budget from 1K to 32K improves accuracy by 15.22 points. At a fixed 16K budget, adding ground-truth crops improves accuracy by 3.75 points over page-only input. We study what the generator needs from retrieved pages through two requirements: Loud, making full use of visual token capacity, and Clear, refining the page so that its evidence becomes prominent. For Loud, increasing either the visual token budget or visual fidelity, the image resolution those tokens preserve, improves accuracy with the other held fixed, and full-page scaling outperforms repetition and tiling at matched budgets. For Clear, suppressing surrounding content improves accuracy without enlarging selected regions, and adding OCR of these regions to page images outperforms either input alone. We implement both as Loud and Clear (LoC), which combines higher-resolution pages with OCR from query-relevant regions without ground-truth annotations or additional training. LoC outperforms eight prior methods on SciEGQA, ViDoRe-v3, and MMLongBench-Doc, improving over the strongest, ColQwen3, by 8.88, 3.09, and 4.22 points with the same retriever and generator. It also outperforms its page-only variant under the same visual token budget. These results locate a bottleneck beyond retrieval: how the retrieved pages are given to the generator.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Deep Learning, Multimodal retrieval-augmented generation, Document question answering, Vision-language models, Visual token budget

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_cHynOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

The paper studies how input presentation affects evidence use in multimodal document RAG. It proposes LoC, combining higher-resolution page images with OCR from query-relevant regions. Controlled experiments examine visual budgets and content refinement, while evaluations on three benchmarks show improvements over the same retriever–generator baseline.

Critical strengths

The controlled experiments hold retrieved pages and the generator fixed, providing useful evidence that input presentation matters beyond retrieval quality.

LoC improves accuracy on three benchmarks without additional training. Same-budget ablations and uncertainty estimates help substantiate its practical effectiveness.

Critical weaknesses

Limited novelty and incomplete attribution. LoC combines resolution scaling, existing region scoring, and OCR. Much of the improvement comes from increased input budgets, while the specific value of query-aware OCR selection over simpler OCR augmentation remains unclear. Clear’s gain on ViDoRe-v3 is not statistically significant.

Unclear advantage over adaptive alternatives. Existing agentic approaches already support query-dependent OCR or visual inspection through cropping and zooming. Without a comparison under matched computational budgets, it is unclear what LoC’s fixed pipeline offers over these alternatives. Larger inputs also increase observed latency, while end-to-end latency and peak memory remain unmeasured.

Questions for authors
  1. Agentic RAG already supports query-dependent OCR, cropping, and zooming [1–3]. Given this, what does LoC offer beyond a fixed recipe for related tools? Why should we prefer uniformly increasing page resolution and appending selected-region OCR over letting an agent inspect the evidence and decide what needs clarification? Under matched computational budgets, does LoC provide a distinct advantage, or would an agent with these tools do better?

  2. *Much of the improvement comes from increased input budgets. What additional benefit comes specifically from query-aware region selection compared with generic OCR augmentation? How do you distinguish increased token capacity from changes in spatial representation? Given that Clear’s gain on ViDoRe-v3 is not statistically significant, how broadly does the evidence support this component?

  3. Larger input budgets increase observed generation latency, while end-to-end latency and peak memory remain unmeasured. Once region selection and OCR overhead are included, does LoC remain competitive with simpler baselines and adaptive agents under matched computational budgets?

[1] AgenticOCR: Parsing Only What You Need for Efficient Retrieval-Augmented Generation.
[2] VRAG-RL: Empower Vision-Perception-Based RAG for Visually Rich Information Understanding via Iterative Reasoning with Reinforcement Learning.
[3] MDocRAG-RL: Empowering Multi-Modal Document RAG via Complex Visual Reasoning with Reinforcement Learning.

Reviewer_CNaNOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

The paper describes a method to increase the resolution of input images while focusing attention on specific relevant regions, with an application to document visual question answering (DocVQA). The authors explore several alternatives for both increasing input resolution and focusing on relevant image regions. By identifying the alternatives that achieve the best results on the SciEQA benchmark, the authors design a final pipeline that is applied to other benchmarks. The experimental results show that the proposed pipeline improves the accuracy of answers across all three evaluated benchmarks.

Critical strengths
  • The proposed method is simple and it can be applied to a wide variety of DocVQA frameworks.
  • The proposed framework demonstrate consistent accuracy improvements across all evaluated benchmarks when compared directly to the baseline ColQwen3 method.
Critical weaknesses
  • There is no evaluation of the computational cost of the proposed approach. Increasing the token budget will inherently increase computation time. It is necessary to evaluate and discuss the trade-off between the improvement of accuracy and the added computation cost.
  • I believe the comparison with the SoA in Table 2 is not entirely correct. The comparison shown in Table 2 is against other retrieval methods, which does not show that LoC performs better than other similar methods for focusing attention on relevant regions of the image, but just that the baseline ColQwen3 performs better than other retrieval methods. A fair comparison should be carried out against other methods designed to increase token budget and/or focus attention on relevant parts of the region. Furthermore, LoC could have been applied to the other retrieval methods to demonstrate the true generalization of the proposed method.
Questions for authors
  • It is not clear why full-page scaling is selected in the "Loud" configuration versus multi-scale views. Both alternatives give very similar results, and multi-scale views actually outperforms full-page scaling in the SciEQA benchmark. A better motivation of this choice would be necessary. Additionally, both configurations could have been tested and compared in the final experimental results to properly justify this architectural choice.
  • In the analysis of alternatives for "Clear", the paper misses including the option of using full page + OCR for the whole page. Providing this would establish a baseline on the improvement carried out simply by adding OCR to the image. This addition would help analyze the real impact of the "page + crop OCR" technique, isolating whether the improvement comes from cropping the specific region of the image or simply from adding the OCR data.
Reviewer_ZxzmOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies how retrieved pages are presented to the generator in multimodal document RAG. Its training-free framework, LoC, combines higher-resolution full-page images with OCR from query-relevant regions to improve answer accuracy.

Critical strengths
  1. The controlled studies are informative, especially the two-stage resizing experiment and the whiteout comparison. In the latter, the selected evidence pixels remain unchanged while surrounding content is removed.

  2. The comparisons with the same retriever and generator support the effectiveness of the approach. The paper also compares input variants under a shared budget for image and OCR tokens.

Critical weaknesses
  1. The method combines established ideas in high-resolution document reading, region selection, and the use of text alongside images. AgenticOCR is a particularly close precedent, as it already combines region extraction, OCR, and page context. The finer controls in this paper add useful evidence, especially the resizing and whiteout studies, but I find the additional insights too limited to support acceptance in the current form.

  2. The ablations do not isolate the contribution of selecting OCR regions based on the query. They support the benefit of combining page images with selected-region OCR, but do not establish how much of the improvement requires query-dependent selection. It remains unclear whether this selection step is needed to obtain the gains, or whether a simpler OCR augmentation would suffice.

Questions for authors

With the retrieved pages, generator, and combined budget for image and OCR tokens held fixed, how does LoC compare with full-page images accompanied by whole-page OCR? Could the authors also include a query-agnostic region-selection control with a comparable OCR token count to isolate the benefit of using the query to select regions?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.