- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 3(+1)
Read review
Paper summary
This paper proposes DocComp, a framework that reduces the computational cost of document visual question answering through query-conditioned semantic component selection. It first uses a frozen layout detector to generate candidate regions, then adapts ColSmol from page-level retrieval to within-page component ranking using answer-location supervision, same-page negatives, and text-side LoRA. The framework adaptively selects the number of components based on retrieval score gaps and reconstructs the selected evidence for different image processors before passing it to frozen downstream vision-language models.The authors evaluate answer quality and visual-token usage on 41,180 answer-valid BoundingDocs examples across nine Qwen and InternVL models. Retrieval, ablation, and computational analyses additionally use a subset of 35,349 examples with reliable spatial annotations. The results show substantial improvements in within-page retrieval and reduced visual-token usage, with comparable or improved overall ANLS for some models. However, answer quality varies across models and document sources, and practical latency improvements depend on reusing page-level preprocessing.
Critical strengths
S1. Component-level adaptation delivers substantial retrieval gains supported by statistical analysis. On identical candidate regions and positive assignments, DocComp improves within-page R@1 from 37.78% for the base ColSmol retriever to 92.56%, with paired document-cluster bootstrap confidence intervals supporting the improvement. The fixed-Top-3 downstream experiment also improves ANLS from 0.5898 to 0.8179. These results provide substantive evidence for adapting page-level retrievers specifically to within-page component ranking. Although the downstream comparison does not use identical visual-token budgets, this does not diminish the significance of the retrieval gains measured on the same candidate sets.
S2. Processor-aware reconstruction is practically valuable and evaluated across multiple models. The paper considers not only which regions to select, but also how different image processors convert them into visual tokens, including avoiding the enlargement of small crops and reducing redundant overhead in fixed-tile processing. A single retriever serves nine frozen downstream models. For Qwen, the method reduces visual tokens by approximately 62–63%, with overall ANLS changes ranging from −1.90 to +0.84 points. The InternVL reconstruction comparisons further demonstrate the importance of preserving region scale for answer quality. These results support the practical value of the framework as a reusable front end.
Critical weaknesses
W1. The incremental contribution over the closest region-selection approaches is not sufficiently established empirically. The related-work section discusses PinPoint and RegionRAG, but the main compression comparison includes only VScan and VisionZip. Since the central contribution is the combination of component supervision, region selection, and input reconstruction, these comparisons do not sufficiently establish its incremental value over existing region-selection approaches. The experiments convincingly demonstrate an advantage over the evaluated post-hoc pruning configurations, but they do not establish an advantage over the closest region-based alternatives. A limited comparison with clearly specified task and resource conditions, or a substantive explanation of why direct comparison is inappropriate, would considerably strengthen the contribution’s positioning.
W2. Essential experimental protocols and practical efficiency gains are not sufficiently independently verifiable. The reliable subset has 100% candidate coverage, but the paper does not fully specify its filtering procedure, the conversion of multipage documents into QA–page examples, the dataset revision, or document-level split checks explicitly covering all 41,180 evaluation examples. Essential training, detection, and latency-measurement configurations are also incomplete. Moreover, uncached execution is 4.75% slower than Full-page, so practical speedups depend on caching or page-level amortization. It remains unclear whether Full-page uses equivalent opportunities to reuse cacheable visual features. These gaps directly affect the scope of the retrieval results, reproducibility, and the assessment of the efficiency contribution. They do not establish data leakage or an unfair comparison, but they require clarification.
Questions for authors
1.What is the key incremental value of DocComp relative to PinPoint and RegionRAG? Could you provide a limited region-selection comparison with clearly specified training resources, downstream generators, and input budgets, or explain which task or model constraints make a direct comparison inappropriate? This would affect my assessment of the contribution’s significance. 2.Please specify the executable rules and exclusion counts by category for constructing the 41,180 answer-valid examples and the 35,349-example reliable subset. Which dataset revision was used? How were multipage documents converted into QA–page examples, and were input pages selected using answer locations? Is the entire evaluation set document-disjoint from the training and validation sets? These details would affect my assessment of the evaluation scope and protocol reliability. 3.Please report the hardware, batch size, numerical precision, timing boundaries, absolute latency, and a breakdown of detection, component encoding, retrieval, and VLM inference time. Does Full-page reuse cacheable visual features? Does the page-amortized latency advantage persist when both methods operate under consistent reuse conditions? This would affect my assessment of the practical efficiency contribution. 4.Please provide the LoRA, optimizer, training-duration, detector, and context-margin configurations, and clarify whether positive-assignment rules are identical during training and evaluation. Using the existing predictions, could you distinguish failures to retain an answer-bearing component from incorrect answers despite retaining such a component, and provide diagnostics indicating whether the latter involve missing context or reconstruction issues? This would help explain the relationship between high answer-location retention and final QA quality.
2 comments
Response to Reviewer DhgX — Part 2/2
W2 / Q3. Latency.
- Setup. One H200, batch 1, BF16, SDPA, greedy decoding (64 tokens), CUDA-synchronised timers. Fixed 500 reliable QA, 10 warm-up queries.
- Breakdown (ms).
- Full-page 1,068.8 = loading 410.9 + processor 228.8 + vision 231.5 + prefill/decoding 197.6.
- DocComp 1,119.6 = component encoding and ranking 590.8 + detection + VLM path 410.6.
- Detection: 55.1 ms/page.
- Does Full-page reuse cacheable features? Not in the paper. We now cache Full-page visual embeddings per page and measure on the same protocol. Inputs are token-identical to the uncached run.
| ms per QA | Full-page | DocComp |
|---|---|---|
| No reuse | 1,068.8 | 1,119.6 |
| Page-level reuse, DocComp only (paper) | 1,068.8 | 755.3 |
| Page-level reuse, both (new) | 625.7 [571, 686] | 755.3 |
- Does the advantage persist under consistent reuse? No. We remove the latency claim.
- What remains:
- decoder visual tokens −63%;
- VLM stage −45% (642.2 vs 354.0 ms);
- analytical FLOPs −51% cold and −46% to −49% with equal reuse.
W2 / Q4. Configuration, positive rules, failures.
-
Retriever.
- ColSmolVLM-500M (650243e9) with text-side LoRA on q/k/v/o/gate/up/down: r 32, α 32, dropout 0.05.
- Multi-positive InfoNCE (τ 1.0), AdamW with linear warmup and decay, BF16. Batch of 48 queries, ≤ 32 negatives per query.
- 2 epochs: epoch 1 at lr 2e-4 (10,121 steps); epoch 2 at lr 5e-5. One H200.
-
Detector. DocLayout-YOLO (YOLOv10m, D4LA), frozen. Input 1024², confidence 0.01, NMS IoU 0.45.
-
Selection. Top-1 if gap(1,2) ≥ 4.0; else Top-3 if gap(3,4) ≥ 1.5; else Top-5.
-
Context margin. 24 px.
-
Positive rules. They differ between training and evaluation:
- training uses at most one positive: a component covering ≥ 0.8 of the answer anchor and containing its centre;
- evaluation accepts every component covering ≥ 0.8 of any answer box.
Under the stricter training rule, reliable retention is 99.50% (99.53% under the evaluation rule).
-
Failures (reliable subset, exact match, 9,071 errors).
| Case | QA |
|---|---|
| Answer-bearing component not retained | 134 (1.5%) |
| Retained, but wrong | 8,937 (98.5%) |
| — of which Full-page is correct | 1,921 |
We re-ran the 1,921 cases with one change each, keeping the same selection.
| Change | Exact match |
|---|---|
| Unchanged | 0.027 |
| 2× crop resolution | 0.645 |
| 96 px context | 0.523 |
| Top-5 | 0.425 |
Most errors come from resolution and local context, not from missing regions.
Response to Reviewer DhgX — Part 1/2
We thank the reviewer. Corrections are in the general response: GT-free re-evaluation, disjointness and latency.
W1 / Q1. Incremental value over PinPoint and RegionRAG.
All methods were run on all 41,180 known-page QA, with paired document-cluster bootstrap (5,000 resamples).
W2 / Q2. Data protocol.
The rule uses detector candidates, so the 100% coverage is partly by construction. Coverage on all 41,180 QA is 98.16%.