- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper proposes a training-free framework for high-resolution VQA, targeting cases related to small visual objects. It first extracts entities from the question, uses the MLLM’s attention to generate initial candidate regions, and caches entity-specific attention maps. After generating an initial answer, the model selects an entity for further inspection, generates candidate crops using its cached attention map, and ranks them by an MLLM. The selected crops are assembled into a spatially ordered canvas and fed back to the model for iterative refinement, with confidence-based stopping. Experiments show that RECAP outperforms compared methods on three high-resolution benchmarks across four VLM backbones. Ablations show the effectiveness of its iterative refinement phase and spatial composition, while the method incurs additional inference latency.
Critical strengths
- The paper addresses an important practical issue in high-resolution VQA that relevant visual evidence can be overlooked due to background clutter. This training-free pipeline improves performance over baselines across four evaluated backbones.
Critical weaknesses
- The hyperparameters are selected based on V* Bench accuracy (Appendix A.2), while the same benchmark is used for the main evaluation without a documented validation split. This would raise concerns about benchmark-specific tuning.
- The contribution of answer-conditioned entity diagnosis is not directly isolated. The repeated-sampling and sequential self-refinement controls (Table 4b) only match model-request counts. A controlled entity-selection comparison would better support its contribution.
Questions for authors
- Could the authors report the average number of model forward passes and total processed tokens per question for RECAP and the compared baselines? This would provide a more complete accuracy–computation trade-off.
- Why does RECAP underperform HiDe on Qwen3-VL-8B, particularly on HRBench-8K (79.1% vs. 83.9% average accuracy)? Representative failure modes would help clarify this gap.
- Could the authors include a baseline comparison with FOCUS [1], a closely related training-free method that identifies target objects and ranks candidate regions using object-existence confidence?
[1] FOCUS: Internal MLLM Representations for Efficient Fine-Grained Visual Question Answering. NeurIPS 2025.
6 comments
Response to Reviewer w923
Q1: Forward passes and total processed tokens per sample
We report accuracy and inference cost on V* Bench using Qwen2.5-VL-7B. Forward passes, FLOPs, visual tokens, total processed tokens, and inference time are averaged per sample. Total processed tokens sum input and generated tokens over every model call, including repeated conversation context; visual tokens are included in the input-token total and are not added again. The V* Bench baseline accuracies and latencies follow the efficiency comparison in Table 4.
| Method | Accuracy (%) | Forward passes | TFLOPs | Visual tokens | Total processed tokens |
|---|---|---|---|---|---|
| Direct | 79.1 | 1.0 | 97.5 | 4,471.8 | 4,530.7 |
| ViCrop | 83.8 | 3.0 | 251.0 | 13,671.4 | 13,848.2 |
| ERGO | 81.7 | 2.0 | 214.0 | 9,350.7 | 13,487.9 |
| HiDe | 90.6 | 3.0 | 271.5 | 13,777.5 | 15,001.7 |
| RECAP () | 95.3 | 4.2 | 281.3 | 13,811.2 | 16,720.6 |
| RECAP () | 95.8 | 6.6 | 368.3 | 15,987.4 | 18,177.9 |
The per-question averages include the initial global-attention diagnosis pass (), which comprises diagnosis, candidate verification, and a canvas update and achieves 89.5%. sets the maximum number of subsequent entity-specific refinement rounds. Therefore, means one additional round using the diagnosed entity’s cached attention map, yielding 95.3% accuracy.
Compared with HiDe, the one-round configuration improves accuracy by 4.7 percentage points with 3.6% more FLOPs and 11.5% more total processed tokens. Visual-token usage is nearly identical. It averages 4.2 forward passes and 7.7 seconds per sample, compared with HiDe’s 3.0 passes and 6.6 seconds. Default RECAP reaches 95.8% accuracy with 6.6 forward passes, 18,177.9 total processed tokens, and 10.8 seconds per sample.
We have expanded Table 4 to report these costs and clarified the token-counting convention and refinement budget in the efficiency discussion.
Q2: Qwen3-VL-8B performance gap
We traced the Qwen3-VL-8B performance gap to an error in our attention-reconstruction code used with FlashAttention 2. When no explicit attention mask was supplied, the reconstruction failed to apply causal masking. This produced attention maps that differed from the backbone’s attention and affected RECAP’s region proposals.
We fixed the error and reran the evaluation with the same hyperparameters. The table below reports average accuracy (%).
| Benchmark | RECAP (original) | RECAP (corrected) | HiDe (reported) |
|---|---|---|---|
| V* Bench | 89.5 | 92.7 | 89.5 |
| HRBench-4K | 81.1 | 82.0 | 81.2 |
| HRBench-8K | 79.1 | 84.4 | 83.9 |
The original HRBench-8K gap to HiDe (79.1% vs. 83.9%) resulted from the causal-masking bug in attention reconstruction. After fixing the bug, RECAP reaches 84.4% on HRBench-8K and also exceeds HiDe’s reported results on V* Bench and HRBench-4K, with no hyperparameter retuning. Thus, the original gap does not represent the corrected method’s performance. We apologize for the implementation error and have updated the affected results in the main results table.
We also checked the attention-extraction implementations for Qwen3.5-4B and InternVL3.5-8B and evaluated both using the shared hyperparameters without model-specific retuning.
| Backbone | Benchmark | Direct (%) | HiDe (%) | RECAP (%) |
|---|---|---|---|---|
| Qwen3.5-4B | V* Bench | 83.2 | 88.0 | 91.1 |
| Qwen3.5-4B | HRBench-4K | 83.0 | 86.5 | 88.5 |
| Qwen3.5-4B | HRBench-8K | 76.8 | 81.9 | 84.4 |
| InternVL3.5-8B | V* Bench | 81.2 | 79.6 | 88.5 |
| InternVL3.5-8B | HRBench-4K | 73.0 | 74.9 | 77.0 |
| InternVL3.5-8B | HRBench-8K | 66.1 | 71.4 | 74.0 |
These results show that RECAP also benefits newer backbones under the shared hyperparameters. RECAP achieves the highest accuracy among the compared methods on all six backbone–benchmark combinations. We have included the corrected Qwen3-VL-8B results and both newer-backbone evaluations in the main results table.
Response to Reviewer w923
Q3: Comparison with FOCUS
FOCUS identifies target objects and ranks candidate regions using object-existence confidence. RECAP also verifies candidate visibility, and uses intermediate answers to select which entity to refine next. It then generates and verifies candidates for that entity and updates the shared evidence canvas.
The table below compares RECAP with FOCUS’s reported results using Qwen2.5-VL-7B.
| Method | V* Bench (%) | HRBench-4K (%) | HRBench-8K (%) |
|---|---|---|---|
| FOCUS (reported) | 90.58 | 79.25 | 76.25 |
| RECAP | 95.8 | 78.1 | 78.4 |
RECAP achieves higher accuracy on V* Bench and HRBench-8K, while FOCUS performs better on HRBench-4K. These comparisons use results from separate studies rather than matched local evaluations. FOCUS reports direct-inference baseline accuracies of 79.06%, 71.62%, and 68.62%, compared with 79.1%, 72.2%, and 67.0% in our evaluation.
We have added FOCUS’s reported results to the main results table and clarified the methodological distinction in Related Work.
Response to Reviewer w923
Your detailed comments helped us clarify the experimental setup and strengthen the comparisons. Thank you also for pointing us to FOCUS.
To help locate the updates in the manuscript and supplement, we use purple for changes specific to your feedback and blue for those relevant to multiple reviewers.
Our parameter-selection protocol follows the approach used in HiDe [1, §5.4]: select a default configuration on V* Bench and keep it fixed for evaluation on other datasets. We selected hyperparameters using Qwen2.5-VL-7B and InternVL3-8B, then applied the shared configuration across all evaluated backbones and benchmarks without further tuning. No separate validation split was used on V* Bench, and its role in configuration selection is now documented in Appendix A.2.
With this fixed configuration, RECAP improves over direct answering by 5.9 and 11.4 percentage points on HRBench-4K and HRBench-8K with Qwen2.5-VL-7B, and by 7.7 and 14.1 points with InternVL3-8B. These results show that the gains extend beyond the benchmark used for parameter selection.
[1] HiDe: Rethinking The Zoom-IN Method in High Resolution MLLMs via Hierarchical Decoupling.
We compared entity-selection strategies on V* Bench using Qwen2.5-VL-7B. We replaced answer-conditioned selection with choosing the first, last, or a random remaining entity, keeping the rest of the pipeline, refinement cap, and stopping rule unchanged. Random-selection results are averaged over three seeds.
All strategies choose from the remaining entities. “First” and “last” follow the order of the extracted entity list.
Selecting entities based on the current answer improves accuracy by 2.6 percentage points over first-entity selection, 2.1 points over last-entity selection, and 2.3 points over random selection. These results show that the choice of which entity to refine matters, and that the current answer provides a useful basis for this choice.
We have added this comparison to Table 2(c).