acceptodds
Under review as a conference paper at ICLR 2027

Re-Attending to the Missing Pieces: Test-Time Visual Evidence Refinement for High-Resolution Reasoning

Abstract

Multimodal large language models (MLLMs) can process increasingly high-resolution images, yet often miss small, question-relevant evidence in cluttered scenes. In our controlled study across three MLLMs, accuracy generally improves as the surrounding background is removed, while the queried object remains visible at every ratio. This exposes a visual evidence allocation bottleneck: more pixels do not ensure that finite encoding and attention budgets are directed to relevant evidence. Existing training-free methods may select or refine visual regions, but typically do not jointly diagnose missing evidence and verify the resulting crops. We introduce RECAP (Re-attending and Composing Attention-guided Proposals), a training-free closed-loop framework for adaptive test-time visual evidence refinement. RECAP identifies localizable entities and derives source-resolution proposals from their attention maps. Tentative answers direct refinement toward unresolved evidence, while comparative verification ranks proposals by target visibility. Selected regions form compact, order-aware canvases. Confidence-guided routing returns the answer once the model is sufficiently confident. Across V Bench, HRBench-4K, and HRBench-8K, RECAP improves all four evaluated backbones. It reaches 95.8% with Qwen2.5-VL-7B and 92.7% with InternVL3-8B on V*Bench, achieving the best results among the compared methods for both backbones.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Multimodal Large Language Models, High-Resolution Visual Reasoning, Test-Time Refinement, Training-Free, Attention-Guided Localization, Visual Question Answering

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_w923OpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes a training-free framework for high-resolution VQA, targeting cases related to small visual objects. It first extracts entities from the question, uses the MLLM’s attention to generate initial candidate regions, and caches entity-specific attention maps. After generating an initial answer, the model selects an entity for further inspection, generates candidate crops using its cached attention map, and ranks them by an MLLM. The selected crops are assembled into a spatially ordered canvas and fed back to the model for iterative refinement, with confidence-based stopping. Experiments show that RECAP outperforms compared methods on three high-resolution benchmarks across four VLM backbones. Ablations show the effectiveness of its iterative refinement phase and spatial composition, while the method incurs additional inference latency.

Critical strengths
  1. The paper addresses an important practical issue in high-resolution VQA that relevant visual evidence can be overlooked due to background clutter. This training-free pipeline improves performance over baselines across four evaluated backbones.
Critical weaknesses
  1. The hyperparameters are selected based on V* Bench accuracy (Appendix A.2), while the same benchmark is used for the main evaluation without a documented validation split. This would raise concerns about benchmark-specific tuning.
  2. The contribution of answer-conditioned entity diagnosis is not directly isolated. The repeated-sampling and sequential self-refinement controls (Table 4b) only match model-request counts. A controlled entity-selection comparison would better support its contribution.
Questions for authors
  1. Could the authors report the average number of model forward passes and total processed tokens per question for RECAP and the compared baselines? This would provide a more complete accuracy–computation trade-off.
  2. Why does RECAP underperform HiDe on Qwen3-VL-8B, particularly on HRBench-8K (79.1% vs. 83.9% average accuracy)? Representative failure modes would help clarify this gap.
  3. Could the authors include a baseline comparison with FOCUS [1], a closely related training-free method that identifies target objects and ranks candidate regions using object-existence confidence?

[1] FOCUS: Internal MLLM Representations for Efficient Fine-Grained Visual Question Answering. NeurIPS 2025.

6 comments
AuthorsOpenReview ↗

Response to Reviewer w923

Your detailed comments helped us clarify the experimental setup and strengthen the comparisons. Thank you also for pointing us to FOCUS.

To help locate the updates in the manuscript and supplement, we use purple for changes specific to your feedback and blue for those relevant to multiple reviewers.


W1: Concerns about hyperparameter selection and benchmark-specific tuning

Our parameter-selection protocol follows the approach used in HiDe [1, §5.4]: select a default configuration on V* Bench and keep it fixed for evaluation on other datasets. We selected hyperparameters using Qwen2.5-VL-7B and InternVL3-8B, then applied the shared configuration across all evaluated backbones and benchmarks without further tuning. No separate validation split was used on V* Bench, and its role in configuration selection is now documented in Appendix A.2.

With this fixed configuration, RECAP improves over direct answering by 5.9 and 11.4 percentage points on HRBench-4K and HRBench-8K with Qwen2.5-VL-7B, and by 7.7 and 14.1 points with InternVL3-8B. These results show that the gains extend beyond the benchmark used for parameter selection.

[1] HiDe: Rethinking The Zoom-IN Method in High Resolution MLLMs via Hierarchical Decoupling.


W2: Contribution of answer-conditioned entity selection

We compared entity-selection strategies on V* Bench using Qwen2.5-VL-7B. We replaced answer-conditioned selection with choosing the first, last, or a random remaining entity, keeping the rest of the pipeline, refinement cap, and stopping rule unchanged. Random-selection results are averaged over three seeds.

Entity selectionFinal accuracy (%)
Based on the current answer (ours)95.8
First in the list93.2
Last in the list93.7
Randomly from the list93.5

All strategies choose from the remaining entities. “First” and “last” follow the order of the extracted entity list.

Selecting entities based on the current answer improves accuracy by 2.6 percentage points over first-entity selection, 2.1 points over last-entity selection, and 2.3 points over random selection. These results show that the choice of which entity to refine matters, and that the current answer provides a useful basis for this choice.

We have added this comparison to Table 2(c).

AuthorsOpenReview ↗

Response to Reviewer w923

Q1: Forward passes and total processed tokens per sample

We report accuracy and inference cost on V* Bench using Qwen2.5-VL-7B. Forward passes, FLOPs, visual tokens, total processed tokens, and inference time are averaged per sample. Total processed tokens sum input and generated tokens over every model call, including repeated conversation context; visual tokens are included in the input-token total and are not added again. The V* Bench baseline accuracies and latencies follow the efficiency comparison in Table 4.

MethodAccuracy (%)Forward passesTFLOPsVisual tokensTotal processed tokens
Direct79.11.097.54,471.84,530.7
ViCrop83.83.0251.013,671.413,848.2
ERGO81.72.0214.09,350.713,487.9
HiDe90.63.0271.513,777.515,001.7
RECAP (Tmax⁡=1T_{\max}=1)95.34.2281.313,811.216,720.6
RECAP (Tmax⁡=4T_{\max}=4)95.86.6368.315,987.418,177.9

The per-question averages include the initial global-attention diagnosis pass (t=0t=0), which comprises diagnosis, candidate verification, and a canvas update and achieves 89.5%. Tmax⁡T_{\max} sets the maximum number of subsequent entity-specific refinement rounds. Therefore, Tmax⁡=1T_{\max}=1 means one additional round using the diagnosed entity’s cached attention map, yielding 95.3% accuracy.

Compared with HiDe, the one-round configuration improves accuracy by 4.7 percentage points with 3.6% more FLOPs and 11.5% more total processed tokens. Visual-token usage is nearly identical. It averages 4.2 forward passes and 7.7 seconds per sample, compared with HiDe’s 3.0 passes and 6.6 seconds. Default RECAP reaches 95.8% accuracy with 6.6 forward passes, 18,177.9 total processed tokens, and 10.8 seconds per sample.

We have expanded Table 4 to report these costs and clarified the token-counting convention and refinement budget in the efficiency discussion.


Q2: Qwen3-VL-8B performance gap

We traced the Qwen3-VL-8B performance gap to an error in our attention-reconstruction code used with FlashAttention 2. When no explicit attention mask was supplied, the reconstruction failed to apply causal masking. This produced attention maps that differed from the backbone’s attention and affected RECAP’s region proposals.

We fixed the error and reran the evaluation with the same hyperparameters. The table below reports average accuracy (%).

BenchmarkRECAP (original)RECAP (corrected)HiDe (reported)
V* Bench89.592.789.5
HRBench-4K81.182.081.2
HRBench-8K79.184.483.9

The original HRBench-8K gap to HiDe (79.1% vs. 83.9%) resulted from the causal-masking bug in attention reconstruction. After fixing the bug, RECAP reaches 84.4% on HRBench-8K and also exceeds HiDe’s reported results on V* Bench and HRBench-4K, with no hyperparameter retuning. Thus, the original gap does not represent the corrected method’s performance. We apologize for the implementation error and have updated the affected results in the main results table.

We also checked the attention-extraction implementations for Qwen3.5-4B and InternVL3.5-8B and evaluated both using the shared hyperparameters without model-specific retuning.

BackboneBenchmarkDirect (%)HiDe (%)RECAP (%)
Qwen3.5-4BV* Bench83.288.091.1
Qwen3.5-4BHRBench-4K83.086.588.5
Qwen3.5-4BHRBench-8K76.881.984.4
InternVL3.5-8BV* Bench81.279.688.5
InternVL3.5-8BHRBench-4K73.074.977.0
InternVL3.5-8BHRBench-8K66.171.474.0

These results show that RECAP also benefits newer backbones under the shared hyperparameters. RECAP achieves the highest accuracy among the compared methods on all six backbone–benchmark combinations. We have included the corrected Qwen3-VL-8B results and both newer-backbone evaluations in the main results table.

AuthorsOpenReview ↗

Response to Reviewer w923

Q3: Comparison with FOCUS

FOCUS identifies target objects and ranks candidate regions using object-existence confidence. RECAP also verifies candidate visibility, and uses intermediate answers to select which entity to refine next. It then generates and verifies candidates for that entity and updates the shared evidence canvas.

The table below compares RECAP with FOCUS’s reported results using Qwen2.5-VL-7B.

MethodV* Bench (%)HRBench-4K (%)HRBench-8K (%)
FOCUS (reported)90.5879.2576.25
RECAP95.878.178.4

RECAP achieves higher accuracy on V* Bench and HRBench-8K, while FOCUS performs better on HRBench-4K. These comparisons use results from separate studies rather than matched local evaluations. FOCUS reports direct-inference baseline accuracies of 79.06%, 71.62%, and 68.62%, compared with 79.1%, 72.2%, and 67.0% in our evaluation.

We have added FOCUS’s reported results to the main results table and clarified the methodological distinction in Related Work.

Reviewer_e7h7OpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper studies the visual evidence allocation bottleneck in high-resolution visual question answering. It proposes RECAP, a training-free framework that iteratively refines visual evidence through attention-based region proposals, answer-conditioned refinement, and verified evidence composition. Experiments on V* Bench and HRBench demonstrate the effectiveness across five MLLMs.

Critical strengths
  1. The background removal experiment in Figure 1 provides an interesting observation about high-resolution visual understanding.
  2. RECAP requires no additional training and can be applied to different MLLMs, making it a flexible approach to improving fine-grained visual reasoning.
  3. RECAP achieves promising results across multiple benchmarks and MLLMs, demonstrating its effectiveness.
Critical weaknesses
  1. In Fig. 1, the performance improvements from removing background objects may not necessarily result from the claimed visual evidence allocation bottleneck. For certain QA tasks, removing distracting objects may also simplify object identification and relational reasoning. Please provide a breakdown by question type and additional controlled experiments to better support the claim.
  2. RECAP requires multiple MLLM calls, introducing considerable inference overhead. Although Table 4 reports latency and matched-request comparisons, please further evaluate the computational cost (e.g., FLOPs and visual tokens) and provide comparisons with strong baselines under matched inference budgets.
  3. The evaluated MLLMs are relatively outdated. Please consider including more recent models, such as Qwen3.5, to demonstrate the effectiveness and generalizability of RECAP on stronger MLLMs.
Questions for authors
  1. Vision-OPD [R1] addresses a similar fine-grained perception problem but takes a fundamentally different approach by distilling regional perception capabilities into the model. Please discuss the relative advantages and limitations of training-based capability internalization versus RECAP's approach.
  2. RTAF [R2] learns region selection and focusing behaviors through reinforcement learning, whereas RECAP relies on an explicitly designed test-time refinement pipeline. Please discuss the advantages and disadvantages of these two approaches.
  3. RECAP follows a predefined iterative refinement pipeline, while recent agentic frameworks such as SpatialClaw [R3] allow more flexible, dynamically composed operations. Please clarify the key differences between RECAP and these agentic approaches, and discuss the potential limitations of RECAP's fixed workflow for more complex visual reasoning tasks.

[R1] Vision-OPD: Learning to See Fine Details for Multimodal LLMs via On-Policy Self-Distillation

[R2] Reason with Thumbnails, Answer with Focus: An Efficient and Effective Paradigm for Multimodal Grounded Visual Reasoning

[R3] SpatialClaw: Rethinking Action Interface for Agentic Spatial Reasoning

5 comments
AuthorsOpenReview ↗

Response to Reviewer e7h7

Thank you for suggesting additional controls for Figure 1 and a closer comparison of inference costs.

We have highlighted the revisions prompted by your comments in teal, with shared revisions shown in blue, throughout the manuscript and supplement.


W1: In Fig. 1, the performance improvements from removing background objects may not necessarily result from the claimed visual evidence allocation bottleneck.

Following your suggestion, we added a breakdown by question type and a control that keeps the image dimensions and visual-token count fixed, using Qwen2.5-VL-7B on V* Bench. In the tables below, Attribute and Spatial refer to attribute and relative-position questions.

For Figure 1, we report accuracy as background is progressively cropped away while retaining the targets. Here, 100% is the original image and 0% is the tightest crop that preserves the targets.

Background kept (%)Attribute (%)Spatial (%)Overall (%)
10080.976.379.1
9081.779.080.6
8083.575.080.1
7087.076.382.7
6090.480.386.4
5096.580.390.1
4092.286.890.1
3094.888.292.2
2093.986.891.1
1099.182.992.7
099.184.293.2

From the original image to the tightest crop, attribute accuracy rises from 80.9% to 99.1%, and spatial accuracy from 76.3% to 84.2%. Both question types benefit, although the gains are larger for attribute questions.

We then repeated the experiment by replacing background pixels with white instead of cropping. Pixels inside the annotated target boxes, image dimensions, and visual-token counts remain unchanged across conditions. Here, 100% is the original image and 0% has all pixels outside the target boxes replaced with white.

Background kept (%)Attribute (%)Spatial (%)Overall (%)
10080.976.379.1
9081.775.079.1
8083.575.080.1
7081.779.080.6
6085.276.381.7
5087.077.683.3
4084.476.381.2
3086.185.585.9
2087.882.985.9
1090.484.288.0
086.194.789.5

Replacing all background outside the target boxes with white raises overall accuracy from 79.1% to 89.5%. Attribute accuracy increases from 80.9% to 86.1%, and spatial accuracy from 76.3% to 94.7%. Background removal therefore improves accuracy even when target size and visual-token count remain unchanged.

Together, these experiments show that background removal benefits both attribute and spatial questions, even when target size and visual-token count remain fixed. The gains therefore extend beyond the effects of enlarging targets or reducing input tokens, supporting the importance of selecting relevant visual evidence. Easier object identification and relational reasoning may contribute to these gains alongside changes in attention allocation.

We have clarified this interpretation in the Figure 1 discussion.

AuthorsOpenReview ↗

Response to Reviewer e7h7

W2: Further evaluation about the computational cost and comparisons with strong baselines under matched inference budgets.

We evaluated accuracy and inference cost on V* Bench using Qwen2.5-VL-7B, including a lower-cost RECAP configuration with one refinement round. We report average forward passes, FLOPs, cumulative visual tokens, total processed tokens, and inference time per sample. Total processed tokens sum input and generated tokens over every model call; visual tokens are included in the input-token total and are not added again. The V* Bench baseline accuracies and latencies follow the efficiency comparison in Table 4.

MethodAccuracy (%)Forward passesTFLOPsVisual tokensTotal processed tokens
Direct79.11.097.54,471.84,530.7
ViCrop83.83.0251.013,671.413,848.2
ERGO81.72.0214.09,350.713,487.9
HiDe90.63.0271.513,777.515,001.7
RECAP (T_max = 1)95.34.2281.313,811.216,720.6
RECAP (T_max = 4)95.86.6368.315,987.418,177.9

For interpreting this cost comparison, t=0t=0 denotes the global-attention diagnosis pass, which performs diagnosis, candidate verification, and a canvas update and achieves 89.5%. T_max caps the additional entity-specific refinement rounds after this pass. With (T_max = 1), RECAP adds one round using the diagnosed entity’s cached attention map and reaches 95.3%; the reported costs cover this complete configuration.

This one-round variant improves accuracy over HiDe by 4.7 percentage points with 3.6% more FLOPs and nearly identical visual-token usage. Total processed tokens increase by 11.5%, from 15,001.7 to 16,720.6 per sample. It uses 4.2 forward passes versus HiDe’s 3.0, with inference time increasing from 6.6 to 7.7 seconds. Compared with default RECAP, it reduces FLOPs by 23.6%, total processed tokens by 8.0%, and inference time by approximately 29%, with a 0.5-point decrease in accuracy. Thus, the one-round comparison is close in FLOPs and visual tokens, but not in total processed tokens, forward passes, or latency.

We have expanded Table 4 with these cost metrics and the one-round RECAP results, and updated the efficiency discussion accordingly.


W3: Evaluation on newer backbones

Following your suggestion, we extended the evaluation to Qwen3.5-4B and InternVL3.5-8B, using the same shared hyperparameters without model-specific retuning.

BackboneBenchmarkDirect (%)HiDe (%)RECAP (%)
Qwen3.5-4BV* Bench83.288.091.1
Qwen3.5-4BHRBench-4K83.086.588.5
Qwen3.5-4BHRBench-8K76.881.984.4
InternVL3.5-8BV* Bench81.279.688.5
InternVL3.5-8BHRBench-4K73.074.977.0
InternVL3.5-8BHRBench-8K66.171.474.0

The added results extend the evaluation to two newer backbones. With the same shared hyperparameters, RECAP achieves the highest accuracy among the three methods on V* Bench, HRBench-4K, and HRBench-8K for both Qwen3.5-4B and InternVL3.5-8B. For example, on V* Bench it reaches 91.1% and 88.5%, respectively.

We have added these results to the main results table and documented the shared-hyperparameter protocol in Appendix A.2.

AuthorsOpenReview ↗

Response to Reviewer e7h7

Q: Comparison with training-based and agentic approaches

Vision-OPD transfers regional perception capabilities into the model through on-policy self-distillation, avoiding repeated evidence-acquisition calls at inference time. It requires additional training and model updates. RECAP can be applied to a frozen backbone without collecting training data or optimizing its parameters. This is useful when retraining is impractical, and allows evidence selection to respond to the current question and intermediate answer. The trade-off is more computation at inference time and a dependence on attention-based proposals.

RTAF learns where to look through reinforcement learning and can request higher-resolution evidence when needed. Learning this behavior requires policy training and reward design. RECAP uses the frozen model for entity selection and crop verification, avoiding those training steps. Each refinement step can be inspected, and the number of rounds can be capped to control inference expenditure. Its search remains tied to cached attention maps, so it may fail to recover targets that have no useful localization signal.

SpatialClaw lets an agent write and execute code to combine perception and geometry operations as reasoning proceeds. This gives it more freedom to tackle complex spatial tasks, while requiring a code-execution environment and supporting tools. RECAP uses predefined model calls and image operations, without a code-generating agent or specialist perception tools. It is therefore simpler to integrate for high-resolution VQA, but offers less flexibility when a question requires new search targets, additional tools, or geometric operations outside the predefined workflow.

We have added these comparisons to the related-work discussion.

Reviewer_9bAEOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes RECAP, a training-free inference framework for high-resolution vision-language models. RECAP constructs an iterative visual evidence refinement pipeline through entity extraction, current-answer-based entity selection, candidate region verification, and spatial layout-preserving evidence composition, thereby enhancing fine-grained visual understanding capabilities. The authors conduct experiments on benchmarks such as V* Bench and HRBench-4K/8K, and the results demonstrate that RECAP improves the accuracy of multiple vision-language models. The paper also analyzes the effectiveness of the method through component ablation, inference latency comparisons, and model calling budget matching experiments.

Overall, the problem studied in this paper is of practical significance; however, the method's reliance on initial attention localization, performance discrepancies across different base models, and unresolved inconsistencies among certain experimental results still require further explanation.

Critical strengths
  • RECAP leverages model attention, candidate region verification, and evidence composition to improve high-resolution visual understanding without requiring additional training.
  • Unlike approaches that primarily select regions based on question relevance or visual saliency, RECAP uses the tentative answer to diagnose which question entity remains unclear and guides subsequent visual inspection accordingly.
  • RECAP organizes selected crops into a compact visual canvas while preserving their relative spatial order, allowing the model to retain local details without entirely discarding cross-region spatial relationships.
Critical weaknesses
  • Dependence on the initial global attention map may limit the recall of extremely small objects. My primary concern is that both Entity Proposal and the subsequent Target Re-attention rely on entity attention maps extracted and cached from the initial visual input. Although RECAP uses crops during refinement, the candidate regions are still derived from these cached attention maps. This raises a fundamental question about target localization: how can RECAP reliably identify a small or inconspicuous target in the original high-resolution image if the target is not adequately captured by the initial global attention map? If the target receives insufficient attention and is consequently missed during initial proposal generation, subsequent diagnosis and verification may be unable to recover it, since they rely on the same cached attention map.

  • Inconsistent performance across base models. Table 1 shows that on Qwen3-VL-8B, RECAP achieves the same average accuracy as HiDe on V* Bench (89.5%), performs slightly worse than SIEVE on HRBench-4K (81.1% vs. 81.5%), and performs noticeably worse than HiDe on HRBench-8K (79.1% vs. 83.9%). This raises a question: as the native visual capabilities of base models strengthen, is RECAP's additional cropping and evidence composition still beneficial, or might it even lead to a loss of valid context or spatial information in certain scenarios? Moreover, the paper's hyperparameter sensitivity analysis does not cover Qwen3-VL-8B, making it impossible to determine whether the performance discrepancies are related to parameter configurations.

Questions for authors
  • If a target entity has an insufficient response in the initial global attention map, how does RECAP rediscover that target during subsequent refinement? What is the impact of initial localization failure on the final accuracy? Can the authors clarify how the method addresses this potential limitation and provide an analysis of proposal recall, particularly for small objects and targets that receive weak initial attention.

  • Why do RECAP's gains vary substantially across base models? For example, the improvements are much smaller on Qwen2.5-VL-32B than on Qwen2.5-VL-7B, and RECAP also underperforms some existing methods on Qwen3-VL-8B. Are these differences related to the base models' native visual capabilities, information loss caused by evidence selection and composition, or the transfer of hyperparameters across models?

4 comments
AuthorsOpenReview ↗

Response to Reviewer 9bAE

We appreciate your careful review, particularly your questions about attention-based localization and performance across models.

Revisions addressing your comments are marked in dark orange in the manuscript and supplement. Blue marks changes that also address other reviewers’ comments.


W1 / Q1: Recovering initially missed targets

An initial proposal can miss a target even when its location is still indicated in the cached attention maps. RECAP initializes proposals using the sum of all entity attention maps, then refines them using individual entity maps. This allows refinement to pick up targets whose responses are weak in the combined map and pass the resulting candidates to verification.

Overall proposal recall increases from 91.4% to 95.5% after refinement, showing that some initial misses can be recovered. RECAP is an attention-guided method: when a target has a weak but usable response in an entity-specific cached attention map, refinement can use that signal to generate better localized proposals, even if the target was missed by the initial proposals from the aggregated map. If no cached attention map contains a usable signal for the target, RECAP cannot recover it; this is a limitation of attention-based localization.

W1 / Q1: Proposal recall for small targets

We evaluated proposal recall on V* Bench with Qwen2.5-VL-7B, grouping targets by their bounding-box area as a percentage of the image. A target is recalled if at least one candidate covers 50% or more of its annotated area. The table compares the highest-scoring initial candidate (Top-1), the full initial candidate set (Initial set), and the combined initial and refinement candidates (After refinement).

Target sizeTop-1 recall (%)Initial-set recall (%)Recall after refinement (%)
≤0.01%53.373.386.7
>0.01% and ≤0.1%62.590.394.4
>0.1%77.996.598.8
All67.391.495.5

For the smallest targets, recall rises from 53.3% with the top-1 candidate to 73.3% with the full initial set, then to 86.7% after refinement. These results show that using multiple candidates improves small-target coverage, and refinement finds additional targets missed by the initial candidate set.

We have added the proposal-recall results to Table 6 and the evaluation protocol to Appendix B.1.

W1 / Q1: Final accuracy under incomplete initial target coverage

Using the same evaluation records, we split questions into two groups: those whose initial candidates cover all annotated targets, and those with at least one target missed.

Initial coverageQuestions (%)Direct accuracy (%)RECAP accuracy (%)
All targets covered90.680.397.1
At least one target missed9.466.783.3
All questions100.079.195.8

When at least one target is initially missed, RECAP reaches 83.3% accuracy, compared with 97.1% when all targets are covered. On these missed-target questions, it improves over direct answering by 16.7 percentage points. RECAP therefore remains effective even when the initial proposals do not cover every target.

We have added these results to Appendix B.2, Table A4.

AuthorsOpenReview ↗

Response to Reviewer 9bAE

W2 / Q2: Corrected Qwen3-VL-8B results

We traced the Qwen3-VL-8B performance gap to an error in our attention-reconstruction code used with FlashAttention 2. When no explicit attention mask was supplied, the reconstruction failed to apply causal masking. This produced attention maps that differed from the backbone’s attention and affected RECAP’s region proposals.

We fixed the error and reran the evaluation without changing the hyperparameters.

BenchmarkRECAP (original)RECAP (corrected)HiDe (reported)
V* Bench89.592.789.5
HRBench-4K81.182.081.2
HRBench-8K79.184.483.9

The corrected results exceed the reported HiDe averages on all three benchmarks and SIEVE’s reported 81.5% on HRBench-4K.

We apologize for this implementation error and have updated the corrected results in the main table.

W2 / Q2: Why RECAP’s gains vary across backbones

RECAP’s gain is smaller on Qwen2.5-VL-32B than on Qwen2.5-VL-7B: 4.8 points versus 16.7 points on V* Bench. This may be because the 32B model already has stronger visual capabilities, leaving less room for improvement.

We also evaluated Qwen3.5-4B and InternVL3.5-8B using the shared hyperparameters, after checking their attention-extraction implementations.

BackboneBenchmarkDirect (%)HiDe (%)RECAP (%)
Qwen3.5-4BV* Bench83.288.091.1
Qwen3.5-4BHRBench-4K83.086.588.5
Qwen3.5-4BHRBench-8K76.881.984.4
InternVL3.5-8BV* Bench81.279.688.5
InternVL3.5-8BHRBench-4K73.074.977.0
InternVL3.5-8BHRBench-8K66.171.474.0

These results show that RECAP remains beneficial on both newer backbones, although the size of the gain varies by model and benchmark. On V* Bench, RECAP improves over direct answering by 7.9 points with Qwen3.5-4B and 7.3 points with InternVL3.5-8B. The results also show that refinement does not eliminate all errors: evidence composition can omit part of a reference region needed for spatial reasoning, as illustrated by the Qwen2.5-VL-7B example in Appendix D.4. We have clarified this model-dependent behavior and limitation in the revised manuscript.

We have added the detailed newer-backbone results to the main table and discussed the variation in gains across backbones.

W2 / Q2: Hyperparameter transfer across models

Our parameter-selection protocol is similar to HiDe [1, §5.4]: select a default configuration on V* Bench and keep it fixed for subsequent evaluations. We selected hyperparameters using Qwen2.5-VL-7B and InternVL3-8B, then applied them without model-specific retuning to Qwen3-VL-8B and the two newer backbones.

The Qwen3 rerun used the original hyperparameters, so its accuracy improvement follows from correcting the attention reconstruction, without parameter retuning. The results on Qwen3.5-4B and InternVL3.5-8B further show that the shared configuration works across backbones.

We have clarified the hyperparameter-selection and transfer protocol in Appendix A.2.

[1] HiDe: Rethinking The Zoom-IN Method in High Resolution MLLMs via Hierarchical Decoupling.

AuthorsOpenReview ↗

Follow-up on our response

Dear Reviewer 9bAE, thank you again for your comments on our paper. As the discussion period is coming to an end, we wanted to ask whether our responses, including the small-target recall analysis and corrected Qwen3-VL results, have addressed your concerns. If any questions remain, we would be happy to clarify them. We would appreciate your feedback.

Dear Reviewer 9bAE, as the discussion closes today, we would appreciate any feedback on our responses, particularly the added small-target recall analysis and corrected Qwen3-VL results. If any concerns remain, we would be happy to clarify them before the deadline. Thank you for your time.

Comments on the paper

1 comment
Area_Chair_Pgf3OpenReview ↗

ICLR 2027 Significance Reminder

Dear Reviewers,

Please review the authors' rebuttal to your comments and the other reviews and to respond.

Please remember to update your ratings and final justification if your concerns have been addressed. If ratings are not updated, clearly explain any remaining concerns in your final justification.

As a reminder, the discussion period ends on November 18th, 11:59 PM AoE.

Best, Your AC

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.