acceptodds
Under review as a conference paper at ICLR 2027

Pay More Attention To Text In High-Resolution MLLMs

Abstract

Failures of high-resolution MLLMs are commonly attributed to a visual problem, motivating zooming, cropping, and related visual interventions to recover fine-grained evidence or suppress interference. Yet recent studies suggest that relevant visual evidence is already encoded in intermediate representations, indicating that visual-side improvements alone insufficient. This raises a natural question: does the remaining bottleneck lie in the text that guides visual search? We identify a previously overlooked linguistic bottleneck: questions formulated for answering do not necessarily specify the visual evidence required for localization. To address this mismatch, we introduce EviSpec, a training-free compiler that derives complementary evidence specifications while preserving the original question for final reasoning. We further validate it through matched-control experiments that isolate the roles of evidence specification and localization. With the search budget fixed, structured evidence specifications yield an 8.6% relative gain over generic requests. With evidence geometry matched, the evidence localized by EviSpec yields a 14.8% relative gain over random evidence. Together, these controls isolate the benefit of specifying what evidence to seek rather than merely expanding visual access. Across all five MLLMs, EviSpec consistently improves upon the corresponding baseline on each of the three benchmarks, yielding average relative gains of 10.4%, 8.8%, and 12.4% on V*Bench, HR-Bench-4K, and HR-Bench-8K, respectively. Beyond high-resolution reasoning, EviSpec also achieves state-of-the-art performance on VQA and hallucination-focused benchmarks.

Primary area:
foundation or frontier models, including LLMs
Keywords:
High-Resolution VQA, MLLMs, Evidence Specification, Question Compilation, Training-Free Inference

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_RRXbOpenReview ↗
Rating
2
Confidence
4
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

The paper argues that failures in high-resolution multimodal large language models (MLLMs) should not be attributed solely to insufficient visual resolution or visual distraction. Some question formulations are not well suited to localizing the visual evidence needed for answering, a mismatch the paper terms the evidence-specification gap. EviSpec is introduced, which uses a frozen model to compile questions into three types of localization requests: Target-role (T) extracts answer-bearing targets and necessary reference entities; Canonical full-query (C) extracts canonical entity expressions from the full question, including answer options; and Stem-only (S) performs the same canonicalization after removing the answer options. EviSpec then performs localization using text-to-visual-token attention, combining instruction-prefix prior calibration, Yen thresholding, connected-component selection, and adaptive expansion to construct compact evidence views. Finally, the original image, evidence views, and unmodified question are jointly provided to the model for answering. The paper reports relative improvements across five models and three benchmarks, with additional evaluations on five other visual tasks.

Critical strengths

1.Separating localization requests from the original question used for final answering is a novel angle on the problem. In particular, distractor options and relational descriptions in multiple-choice questions may interfere with attention-based localization.

2.The gains on high-resolution tasks are substantial. If these results can be reproduced under a consistent, clearly specified configuration, they would constitute a meaningful contribution.

3.The method requires neither external detectors nor training additional parameters. Its lower cost relative to multi-round visual search methods gives it some practical deployment value.

Critical weaknesses

1.The SOTA claims go beyond the experimental evidence. The method does not achieve the best performance on every evaluated task. The paper could simply report its demonstrated improvements without claiming SOTA.

2.The method has some novelty, but the “paradigm reframing” claim is clearly overstated. The paper presents the three TCS requests as a new localization interface. However, extracting entities from questions, decomposing complex questions into localizable queries, and jointly feeding cropped images, the original image, and the original question into a model for final answering are ideas already covered by prior work such as V* and FOCUS.

3.Some tabulated results require verification. For example, it is unclear how the overall accuracy of 89.5% in the last row of Table 18 was calculated. Weighting the reported subset accuracies by their sample counts gives (94.8%×115+84.2%×76)/191=90.6%. Please carefully check the reported values.

Questions for authors
  1. Please provide the details necessary to reproduce the experiments for each backbone model–benchmark combination, including the localization layer indices and prompts. Was an independent development set used to select these configurations? Was V*Bench used repeatedly to select or refine these design choices?
  2. Please clarify the scope of the option-free conclusion. Does the benefit of the C branch come from canonicalization, additional evidence views, or answer-option-conditioned search?
6 comments
AuthorsOpenReview ↗

Reply for RRXb [Weak1]: We sincerely thank the reviewer for recognizing our design rationale and insights! We have addressed all concerns and hope our responses will merit a higher rating.

[W1] Clarification of SOTA Claims

We thank the reviewer for highlighting the overly broad SOTA claims. We acknowledge this concern and will revise the statements to accurately reflect the experimental evidence.

[1] Experimental Evidence. Although EviSpec does not achieve the best performance in every setting, it demonstrates strong overall results:

  • Table 1: EviSpec outperforms the corresponding base model in all 15 model-benchmark combinations, achieving the highest accuracy among compared methods in 13/15 settings.
  • Table 2: EviSpec achieves the best benchmark-level aggregate results among compared methods across five additional benchmarks on both evaluated models.

[2] Planned Revisions. We will revise the Abstract, Introduction, Contributions, Experiments, and Conclusion to remove unsupported SOTA claims and clearly distinguish overall improvements from individual best-performing results.

We appreciate the reviewer's suggestion, which helps us present more precise and evidence-grounded conclusions.

AuthorsOpenReview ↗

Reply for RRXb [Weak2]: We sincerely thank the reviewer for recognizing our design rationale and insights! We have addressed all concerns and hope our responses will merit a higher rating.

[W2] Clarifying Paradigm Reframing and Differences from V* and FOCUS

We thank the reviewer for raising this concern. While EviSpec shares aspects of the visual reasoning pipeline with V* and FOCUS, the key distinction lies in the research question and the primary intervention point.

[1] Evidence Specification Rather Than a New Visual Search Paradigm

We do not claim that EviSpec is the first to use linguistic information to guide visual search. V* focuses on how to perform question-guided visual search, while FOCUS emphasizes identifying target-relevant visual regions. In contrast, EviSpec investigates how to transform an original question into structured linguistic requests better suited for visual evidence localization, while keeping the downstream localization mechanism fixed.

We term this the Evidence-Specification Gap: an original question specifies the final judgment but may not adequately express the target roles, entity attributes, and relational cues required for accurate localization.

Furthermore, EviSpec requires fewer hyperparameters: V* (CVPR 2024) involves 7, FOCUS (NeurIPS 2025) approximately 9, and HiDe (ICML 2026) approximately 3, whereas EviSpec introduces only one hyperparameter.

Table W2-1. Methodological comparison with V and FOCUS.*

DimensionV*FOCUSEviSpec
Core QuestionHow to perform guided visual searchHow to identify target-relevant regionsHow to specify required visual evidence
InterventionVisual search processTarget relevance and region selectionLinguistic requests before localization
MechanismQuestion-guided visual searchTarget relevance maps and cropsT/C/S Evidence Specifications
RepresentationSearch results and visual regionsRelevance maps and cropsComplementary linguistic requests
Primary FocusVisionVisionLanguage
Final ReasoningRetrieved visual informationSelected visual evidenceOriginal image, evidence views, and question

Thus, our contribution is not the invention of entity extraction, question rewriting, or cropping, but the systematic formulation and evaluation of structured linguistic requests for visual evidence localization.

[2] Direct Comparison with V* and FOCUS [Additional]

To further evaluate EviSpec against existing visual search methods, we conduct direct comparisons on Qwen2.5-VL-7B and Qwen3-VL-8B across three high-resolution benchmarks.

Table W2-2. Performance comparison on Qwen2.5-VL-7B (Additional).

MethodV* Bench (%)HR-Bench-4K (%)HR-Bench-8K (%)
V*79.0671.7566.13
FOCUS89.5375.5072.75
EviSpec92.1080.9081.60
Gain over FOCUS+2.57+5.40+8.85
Gain over V*+13.04+9.15+15.47

Table W2-3. Performance comparison on Qwen3-VL-8B (Additional).

MethodV* Bench (%)HR-Bench-4K (%)HR-Bench-8K (%)
V*84.8276.8873.75
FOCUS87.4378.1374.38
EviSpec91.6082.5080.40
Gain over FOCUS+4.17+4.37+6.02
Gain over V*+6.78+5.62+6.65

Across all six settings, EviSpec achieves average absolute accuracy improvements of 5.23 percentage points over FOCUS and 9.45 percentage points over V*. These comparisons provide additional evidence of EviSpec's effectiveness, while our matched-control experiments under identical downstream localization mechanisms directly support the contribution of Evidence Specification.

[3] Clarifying Paradigm Reframing and Planned Revisions

Following the reviewer's suggestion, we will revise the Abstract, Introduction, Contributions, and Related Work to:

  1. Remove overly broad paradigm-reframing claims, without implying that entity extraction, question decomposition, or visual cropping are newly introduced techniques.
  2. Clarify our core contribution as structured Evidence Specification for visual localization, rather than a fundamentally new visual search paradigm.

In summary, EviSpec does not reinvent visual search; instead, it explicitly investigates how the required visual evidence should be specified before localization. We appreciate the reviewer's suggestion, which helps us more precisely position our contribution relative to V* and FOCUS.

AuthorsOpenReview ↗

Reply for RRXb [Weak3]: We sincerely thank the reviewer for recognizing our design rationale and insights! We have addressed all concerns and hope our responses will merit a higher rating.

[W3] Verification and Correction of Reported Accuracy

We thank the reviewer for carefully examining our experimental results and identifying the inconsistency in Table 18. We have rechecked the underlying predictions and statistical calculations and confirmed that the reported overall accuracy of 89.5% was incorrect.

[1] Source of the Discrepancy

The error arose during table compilation, where the accuracies of the attribute and spatial subsets were incorrectly averaged without accounting for their different sample sizes:

Incorrect: (94.8% + 84.2%) / 2 = 89.5%.

However, V* Bench contains 115 attribute questions and 76 spatial questions. According to the original prediction records, 109 and 64 questions were answered correctly, respectively. Thus, the correct overall accuracy is:

Correct: (109 + 64) / (115 + 76) = 173 / 191 = 90.6%.

Therefore, the correct overall accuracy is 90.6%, rather than 89.5%. This discrepancy resulted from an incorrect aggregation method, not changes in the underlying model predictions.

[2] Corrections and Verification

We have corrected the aggregation procedure and rechecked the overall accuracies for all configurations in Table 18, ensuring that they are calculated directly from the total number of correct predictions and samples.

This correction does not affect our main experimental comparisons or conclusions. We apologize for this statistical oversight and sincerely appreciate the reviewer's careful examination, which has helped improve the accuracy and reliability of our reported results.

Reviewer_sXh7OpenReview ↗
Rating
2(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper introduces a new training-free runtime augmentation for existing VLMs, named EviSpec. It verbalizes three different types of textual component of the original query: 1/ Target-role 2/ Canonical full-query including options, 3/ Stem-only. Then it uses the full attention map of three different query to identifie the relevant regions of the original query

Critical strengths

The study on effectiveness of three augmented sub-query is interesting (includign both diversity and varying image budget), which shows the importance of different semantic aspects.

Critical weaknesses
  1. The computation cost of this method seems to be massive; three different sub-querie's attention maps are considered separately plus another full forward pass. It weakens the practical value of this work
  2. the question to evidence request construction (Section 3.2) is heuristic-based; why it should be partitioned into 3 different roles, why not 2 or 4?
  3. The localization method using attention map lacks novelty, meanwhile there are bunch of attention study work of VLMs that has been published this year (e.g. VERA, IJCAI 2026).
Questions for authors
  1. How much do we sacrifice inference speed for demonstrated performance improvements ?

  2. Can your method be applied to larger models? I suspect the method is only neccessary when the model size is small (capability is limited), can you try it on qwen3.6-27B/35BA3B or Gemma4 similar sizes?

7 comments
AuthorsOpenReview ↗

Reply for sXh7 [Weak1]: We sincerely thank the reviewer for recognizing our design rationale and insights! We have addressed all concerns and hope our responses will merit a higher rating.

[W1] Computational Cost and Practical Applicability

We thank the reviewer for raising concerns about the computational overhead of EviSpec. We would like to clarify three key points. First, EviSpec incurs computational costs comparable to existing methods. Second, the additional computation enables EviSpec to acquire complementary visual evidence for challenging questions, yielding a 9.4 percentage-point accuracy improvement. Third, EviSpec extracts attention maps from only a single intermediate layer, rather than processing attention across all model layers.

To further address this concern, we provide additional analyses and experiments examining the actual number of localization requests, component-wise computational overhead, the effectiveness of adaptive deduplication, and the benefits of evidence specification, together with existing end-to-end efficiency comparisons.

[1] Actual Number of Executed Localization Requests [Existing]

As discussed in Section 3.2, although EviSpec constructs three candidate specifications (T/C/S), it adaptively adjusts the number of retained sub-query attention maps according to the question type and difficulty.

As shown in Table W1-1, EviSpec retains only 1.9 distinct localization requests per question on average, rather than consistently executing three localization operations. Specifically, 85.8% of questions retain fewer than three requests. For spatial reasoning questions, 50.0% require only one request, while merely 2.6% retain all three requests.

Table W1-1. Distribution of retained requests (Appendix Table 17).

Question Type1 Request2 Requests3 Requests
Overall20.9%64.9%14.1%
Attribute1.7%76.5%21.7%
Spatial50.0%47.4%2.6%

[2] Component-wise Computational Cost Breakdown [Additional]

To quantify the computational overhead, we further decompose EviSpec's end-to-end inference latency into compilation, model forward passes, attention processing, evidence construction, and final answer generation.

Table W1-2. Component-wise latency.

ComponentLatency (s)Percentage
Compilation1.6125.04%
Forward2.7242.30%
Attention processing0.030.47%
Evidence construction1.0416.17%
Final answering1.0316.02%
Total6.43100%

The results show that the primary computational cost arises from forward passes through the frozen MLLM, rather than attention-map processing, which accounts for only 0.47% of the total latency. Furthermore, Appendix F.7 already provides a computational complexity analysis of attention processing.

[3] Effectiveness of Adaptive Deduplication for T/C/S Specifications [Additional]

To directly quantify the computational benefits of adaptive deduplication, we conduct a paired experiment on the same set of questions, comparing EviSpec with and without deduplication.

As shown in Table W1-3, adaptive deduplication reduces the average number of localization calls to 1.9, yielding a 13.1% reduction in inference latency without compromising final answer accuracy.

Table W1-3. Computational effect of deduplication.

ConfigurationAvg. Localization CallsLatency (s)Accuracy (%)
Dedup ON1.96.4392.1
Dedup OFF2.97.4092.1

[4] Additional Computation Enables More Relevant Visual Evidence [Existing]

The performance gains of EviSpec primarily stem from more effective evidence specification. As shown in Table W1-4, EviSpec achieves a 9.4 percentage-point accuracy improvement over generic requests while retaining fewer distinct localization requests on average.

Table W1-4. Structured vs. generic requests (Appendix Table 8).

MetricGeneric-3EviSpec TCS
Accuracy (%)79.188.5
Exclusive successes422
Avg. distinct requests2.61.9

Furthermore, an additional matched request-interface experiment reported in Appendix Table 9 demonstrates that EviSpec obtains substantially more compact visual evidence while achieving higher accuracy.

Table W1-5. Evidence compactness (Appendix Table 9).

MetricGenericEviSpec
Accuracy (%)77.084.3
Selected image area (%)69.68.0

[5] End-to-End Inference Efficiency [Existing]

The paper already includes an efficiency comparison with existing methods, demonstrating that EviSpec achieves comparable inference latency while delivering higher accuracy. Notably, EviSpec is approximately 2× faster than DeepEyes.

Table W1-6. End-to-end efficiency on NVIDIA A800 (Appendix Table 19).

MethodLatency (s/question)Peak Memory
HiDe5.3622 GB
ViCrop5.9821 GB
DeepEyes12.5621 GB
DeepScan60.3128 GB
EviSpec6.4322 GB
AuthorsOpenReview ↗

Reply for sXh7 [Weak2]: We sincerely thank the reviewer for recognizing our design rationale and insights! We have addressed all concerns and hope our responses will merit a higher rating.

[W2] Why Three Evidence Specifications?

We thank the reviewer for raising this important question. To clarify the rationale behind the three specifications, we further examine the number and complementarity of specifications through existing combination ablations and additional experiments introducing a fourth specification.

[1] Why 3 > 4 Specifications? [Additional]

To directly address this concern, we introduce an additional candidate specification:

E (Evidence-cue specification): Explicit representation of visual evidence cues.

Unlike T/C/S, which primarily focus on entity identification, entity normalization, and textual scope, E explicitly preserves the visual attributes or spatial relation predicates that need to be examined.

For example:

  • Attribute question: For "What is the color of the guard's glove?", E specifies the glove and the color attribute to be examined.
  • Spatial question: For "Is the scooter on the left or right side of the cyclist?", E specifies the scooter, the cyclist, and the left-right relation to be examined.

We evaluate three configurations:

  1. TCS: The original complete method.
  2. TCS+E: Adds the evidence-cue specification.
  3. TCS+G: Adds a generic request.

Table W2-1. Performance and efficiency comparison between three- and four-specification configurations (Additional experiment, Qwen2.5-VL-7B).

ConfigurationV* BenchHR-Bench-4KHR-Bench-8KLatency (s)
TCS92.180.981.66.43
TCS+E89.078.478.89.23
TCS+G89.578.178.88.88

The results show that adding a fourth specification does not improve performance. Both TCS+E and TCS+G exhibit slight accuracy degradation across all three benchmarks. Meanwhile, adding E increases inference latency from 6.43s to 9.23s, indicating additional computational overhead without corresponding accuracy gains.

[2] Why 3 > 2 Specifications? [Existing]

We have already systematically evaluated all seven non-empty combinations of T/C/S in Table 3 of the original paper.

Table W2-2. Ablation study of all seven T/C/S combinations (Existing, Table 3).

ConfigurationV* BenchHR-Bench-4KHR-Bench-8K
T88.578.378.6
C88.076.076.0
S88.077.677.9
TC89.079.078.1
TS89.579.078.4
CS90.178.177.5
TCS92.180.981.6

The results demonstrate that the complete TCS configuration consistently achieves the best performance across all three benchmarks. More importantly, the incremental ablation study in Appendix Table 10 further reveals the independent contribution of the third specification.

Table W2-3. Incremental contribution of the third specification (Existing, Appendix Table 10).

BenchmarkTC − TTS − TTCS − TCTCS − TS
V* Bench+0.5+1.0+3.1+2.6
HR-Bench-4K+0.7+0.7+1.9+1.9
HR-Bench-8K-0.5-0.2+3.5+3.2

Adding the third specification yields consistent performance improvements across all three benchmarks.Furthermore, the controlled experiment under the same request generation budget (Appendix Table 8) shows that TCS achieves 88.5% accuracy, substantially outperforming Generic-3 (79.1%), while retaining only 1.9 distinct requests on average, compared with 2.6 for Generic-3. This further confirms that the advantage of TCS arises from more effective semantic organization, rather than simply generating more queries.

[3] Relationship Between Request Count and Computational Cost [Existing]

As already demonstrated in Appendix Table 17, T/C/S does not require three fixed localization operations. Instead, EviSpec adaptively adjusts the number of retained requests according to the question, retaining only 1.9 distinct localization requests per question on average.

Table W2-4. Distribution of effective localization requests across question types (Existing, Appendix Table 17).

Question Group1 Request2 Requests3 Requests
Overall20.9%64.9%14.1%
Attribute1.7%76.5%21.7%
Spatial50.0%47.4%2.6%

[4] Design Rationale for Three Complementary Specifications

T/C/S are complementary representations designed to address distinct linguistic requirements in visual evidence localization. Taken together, the additional fourth-specification experiments demonstrate that introducing an evidence-cue specification does not provide further accuracy gains and incurs higher inference costs.

We will further clarify the design principles of the three specifications in Section 3.2 of the revised manuscript and incorporate the incremental ablation results, request diversity analysis, and fourth-specification comparisons into the main text to provide a more systematic justification for the T/C/S design.

AuthorsOpenReview ↗

Reply for sXh7 [Weak3]: We sincerely thank the reviewer for recognizing our design rationale and insights! We have addressed all concerns and hope our responses will merit a higher rating.

[W3] Novelty of EviSpec and Differences from VERA

We thank the reviewer for highlighting recent attention-based visual evidence localization studies. We will discuss VERA (IJCAI 2026) in Related Work. We clarify that EviSpec's primary novelty lies in how linguistic inputs guide visual evidence localization, rather than attention-based localization itself.

Moreover, EviSpec requires fewer hyperparameters: VERA uses 2, V* (CVPR 2024) uses 7, FOCUS (NeurIPS 2025) uses approximately 9, and HiDe (ICML 2026) uses 3. In contrast, EviSpec introduces only one hyperparameter, avoiding extensive tuning.

[1] Methodological Differences from VERA

VERA investigates Visual Evidence Retrieval (VER) Heads, uses model uncertainty to trigger retrieval, and textualizes relevant visual patches. In contrast, EviSpec focuses on structuring linguistic inputs to improve question-relevant visual evidence localization.

[2] Independent Contribution of Evidence Specification [Existing]

Our core claim is that improving linguistic specifications enhances visual evidence retrieval without changing the underlying attention-based localization mechanism. We validate this through matched-control experiments.

Table W3-1. Comparison under the same request generation budget (Existing, Appendix Table 8).

MetricGeneric-3EviSpec TCS
Accuracy (%)79.188.5
Exclusive successes422
Average distinct requests2.61.9

TCS achieves a 9.4 percentage-point accuracy improvement while retaining fewer requests, confirming that its gains do not simply arise from more localization operations.

Table W3-2. Matched request-interface comparison (Existing, Appendix Table 9).

MetricDirectGenericEviSpec TCS
Accuracy (%)78.577.084.3
Average requests—2.91.9
Selected area (%)—69.68.0

Under identical downstream localization mechanisms, TCS improves accuracy while reducing the selected image area from 69.6% to 8.0%.

[3] Cross-Localizer Evaluation [Additional]

To verify that these gains are not specific to one attention-based localizer, we evaluate three strategies:

  1. Vanilla: Standard multi-head attention without calibration.
  2. Calibrated: EviSpec's attention calibration and region formation.
  3. VERA Head: VERA-style visual evidence retrieval head selection.

All configurations use Qwen2.5-VL-7B, V* Bench, and identical experimental settings.

Table W3-3. Evidence specification across localization strategies (Additional experiment).

Localization StrategyRaw QuestionGeneric RequestTCSTCS Gain (pp)
Vanilla75.977.083.2+6.2
Calibrated78.077.584.8+7.3
VERA Head80.181.287.4+6.2

TCS consistently outperforms generic requests across all three localization strategies, yielding +6.2, +7.3, and +6.2 percentage-point gains, respectively. Notably, TCS remains effective without EviSpec's attention calibration and also improves VERA-style head selection. These results demonstrate that language-side evidence specification is effective across different localization mechanisms, rather than depending on a specific attention-based algorithm.

[4] Verifying the Relevance of Retrieved Evidence [Existing]

We further verify whether performance gains genuinely arise from question-relevant visual evidence through matched replacement and masking experiments.

Table W3-4. Matched evidence controls (Existing, Appendix Table 13).

ConfigurationAccuracy (%)
Direct78.5
Selected Evidence91.6
Exact-geometry Random Evidence79.8
Selected-region Mask43.5
Exact-geometry Random Mask77.1

With matched region geometry, EviSpec's selected evidence improves accuracy by 11.8 percentage points over random evidence. Moreover, masking selected regions reduces accuracy to 43.5%, compared with 77.1% for random masking. These results confirm that EviSpec retrieves genuinely relevant visual evidence, rather than benefiting merely from additional image inputs.

[5] Clarifying Novelty and Planned Revisions

We will revise the manuscript to:

  1. Discuss VERA and clarify the methodological differences in Related Work.
  2. Strengthen the motivation for Question-to-Evidence Specification in Section 3.2.
  3. Move the matched request-budget and request-interface experiments into the main text.
  4. Incorporate the additional cross-localizer results.

In summary, we do not claim novelty in attention-based localization itself. Instead, EviSpec identifies the Evidence-Specification Gap and addresses it through structured linguistic evidence requests, requiring only one hyperparameter.

Reviewer_Y3geOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies whether the wording used for visual search limits high-resolution VQA. EviSpec compiles a question into target-role, canonical full-query, and option-free specifications, localizes them using calibrated attention, and supplies compact evidence views for answering the original question. It is training-free. Results cover five models on three high-resolution benchmarks and additional visual tasks, with gains over the base model in all 15 main model-benchmark combinations.

Critical strengths
  1. The controls are a strong part of the paper. Equal-attempt generic requests test whether more search calls explain the gains, while geometry-matched random crops and masking test whether the selected content matters (Section 4; Appendices B and C). These are more informative than an accuracy-only comparison of a crop pipeline.

  2. The gains extend across model sizes and architectures. For Qwen2.5-VL-7B, Table 1 improves HR-Bench-8K from 67.9 to 81.6. The branch ablation and transfer tasks add support that the evidence interface is useful beyond one model or benchmark.

Critical weaknesses
  1. It is difficult to map the controlled results to one reproducible configuration. On V*Bench, the main TCS result is 92.1, the equal-search-budget control is 88.5, the matched prompt-family result is 84.3, and the component diagnostic reaches 93.2. The appendices acknowledge different executions and wrappers, but do not provide a clear configuration map for every contrast. This weakens the otherwise appealing attribution of the gain to request structure.

  2. Most detailed mechanism controls use one model and only 191 V*Bench questions. Broader benchmark gains are useful, but do not directly establish that request structure is the cause across settings. The runtime cost also matters: Table 19 reports 6.43 seconds per question versus 2.20 for Base and 5.36 for HiDe. The practical tradeoff deserves a more explicit assessment.

Questions for authors
  1. Clarification: Please map Tables 1, 8, 9, and 11 to their exact prompts, localization wrapper, evidence packing, resolution, and decoding settings. Which single configuration reproduces the main-table EviSpec result, and which matched contrast isolates its compiler contribution?

  2. Assessment-relevant validation: Does the structured-versus-generic request advantage hold on HR-Bench under the same configuration and matched total inference cost? A paired uncertainty estimate would help determine whether the request-specific gain extends beyond the small V*Bench control.

7 comments
AuthorsOpenReview ↗

Reply for Y3ge [Weak1]: We sincerely thank the reviewer for recognizing our design rationale and insights! We have addressed all concerns and hope our responses will merit a higher rating.

[W1] Configuration Mapping and Reproducibility of Controlled Experiments

We thank the reviewer for raising concerns about experimental consistency and reproducibility. We clarify that 92.1%, 88.5%, 84.3%, and 93.2% correspond to different experimental protocols, rather than repeated evaluations under identical settings. These experiments respectively evaluate the full method, equal-search-budget controls, matched prompt-family controls, and localization components.

Importantly, (1) Appendices B.1–B.3 already provide the prompts, execution settings, and certain evidence packing differences for these experiments. (2) We have also submitted fully reproducible code with our manuscript. However, we acknowledge that these details are scattered across sections. We will provide a unified configuration mapping.

All diagnostic controls use Qwen2.5-VL-7B on the same 191 V* Bench questions, with shared settings including USE_ALLATT=False, BF16, FlashAttention-2, greedy decoding, and a maximum processor budget of 16,384 × 28². Experiments run on NVIDIA A800 GPUs with Python 3.11.13, PyTorch 2.8.0+cu128, and Transformers 4.57.1.

[1] Mapping Experimental Results to Configurations [Existing]

Table W1-1. Configuration mapping of reported V Bench results.*

ExperimentTCS Acc.ControlEvaluation PurposeSource
Main Evaluation92.1%T/C/S AblationsFull method and branch contributionsTables 1, 3
Equal-search-budget88.5%Generic-3: 79.1%Control for request countTable 8
Matched Prompt-family84.3%Generic: 77.0%Structured vs. generic requestsTable 9
Component Diagnostic93.2%Legacy+Legacy: 89.5%Attention calibration and region formationTable 11

Appendix B.1 specifies Localization Layer 15, BF16, FlashAttention-2, 448×448 evidence views, identical original images and final questions, and greedy decoding for matched controls.

Appendix B.2 further distinguishes One-canvas Union from Separate-view Execution, which respectively achieve 88.5% and 90.6% accuracy in the corresponding request-interface comparison.

These results indicate that performance depends not only on request construction but also on evidence packing and localization wrappers. Therefore, absolute accuracies across different experimental protocols should not be directly compared.

[2] Attribution Through Matched Controls [Existing]

Our conclusions regarding Evidence Specification are supported by two within-protocol matched comparisons, rather than differences between the four reported TCS accuracies.

Table W1-2. Matched controls for evidence specification.

ExperimentGenericTCSImprovement
Equal-search-budget79.1%88.5%+9.4 pp
Matched Prompt-family77.0%84.3%+7.3 pp

Equal-search-budget: Both configurations attempt three requests with identical downstream localization and answering procedures. TCS achieves +9.4 percentage points while retaining fewer distinct requests (1.9 vs. 2.6).

Matched Prompt-family: Under matched input scopes, output labels, and downstream processing, TCS achieves +7.3 percentage points while reducing the selected image area from 69.6% to 8.0%.

Thus, the contribution of structured evidence requests is established through matched within-protocol comparisons, not by comparing 92.1%, 88.5%, and 84.3% across different protocols.

Notably, the 93.2% result in Table 11 evaluates the combined effects of Attention Calibration and Adaptive Region Formation, rather than the independent contribution of T/C/S request construction.

[3] Configuration Mapping and Planned Revisions

We will introduce a unified Configuration Mapping Table for Tables 1, 8, 9, and 11, specifying:

  1. Request configuration: Compiler prompts and localization layers.
  2. Localization: Attention processing and region formation.
  3. Evidence packing: One-canvas Union or Separate-view Execution.
  4. Visual settings: Image resolution and visual input budgets.
  5. Inference: Final answering prompts, decoding settings, and corresponding execution commands.

We will explicitly distinguish the full EviSpec pipeline from diagnostic configurations, linking each reported result to its exact code configuration.

In summary, the four accuracies reflect distinct experimental protocols, not contradictory estimates under identical settings. The independent contribution of Evidence Specification is supported by matched improvements of +9.4 and +7.3 percentage points.

We appreciate the reviewer's suggestion and will improve the transparency and reproducibility of our experimental reporting.

AuthorsOpenReview ↗

Reply for Y3ge [Weak2]: We sincerely thank the reviewer for recognizing our design rationale and insights! We have addressed all concerns and hope our responses will merit a higher rating.

[W2] Cross-Benchmark Mechanism Validation and Accuracy–Efficiency Trade-off

We thank the reviewer for raising concerns about the scope of mechanism and practical inference costs. First, we would like to clarify that the T/C/S ablations in our paper are not limited to the V* , but have already been systematically evaluated on V* Bench, HR-Bench-4K, and HR-Bench-8K.

We agree that extending the stricter Generic-vs-TCS matched controls and analyzing their costs would further strengthen the attribution of performance gains to request structure across benchmarks.

[1] Incremental Benefits of Request Structure Beyond V* Bench [Existing]

Table 3 of the paper already evaluates all seven non-empty T/C/S combinations.

Table W2-1. Cross-benchmark ablation of Evidence Specifications (Existing, Table 3).

ConfigurationV* BenchHR-Bench-4KHR-Bench-8K
T88.578.378.6
C88.076.076.0
S88.077.677.9
TC89.079.078.1
TS89.579.078.4
CS90.178.177.5
TCS92.180.981.6

The complete TCS configuration outperforms every two-branch combination. Notably, on HR-Bench-8K, TCS improves accuracy by 3.5 and 3.2 % over TC and TS, respectively. These results demonstrate that the complementarity of request structures is not unique to V* Bench.

However, we agree with the reviewer's distinction: combination ablations establish the incremental contributions of T/C/S but cannot fully replace strictly matched comparisons between structured and generic requests.

[2] Extending Generic-vs-TCS Matched Controls to HR-Bench [Additional]

Appendix Table 8 already reports an Equal-search-budget control on V* Bench. Under the same three-request generation budget, TCS improves accuracy from 79.1% (Generic-3) to 88.5% (+9.4 percentage points), while retaining fewer distinct requests on average (1.9 vs. 2.6).

To further address this concern, we extend this matched experiment to HR-Bench-4K and HR-Bench-8K, and additionally evaluate Qwen3-VL-8B for cross-backbone validation. We also compare actual localization calls and latency under matched candidate generation budgets. (All comparisons fix the Backbone, Layer, Processing, Region, Evidence, Budget, Prompt, and Settings, varying only the Request Construction strategy.)

Table W2-2. Structured-vs-Generic matched controls (Additional experiment).

BackboneBenchmarkGeneric-3 (%)TCS (%)Δ Accuracy (pp)95% Paired CI
Qwen2.5-VL-7BV*Bench82.292.1+9.9[+4.1, +15.1]
Qwen2.5-VL-7BHR-Bench-4K75.580.9+5.4[+2.6, +8.2]
Qwen2.5-VL-7BHR-Bench-8K74.181.6+7.5[+4.5, +10.5]
Qwen3-VL-8BV*Bench82.791.6+8.9[+3.4, +14.4]
Qwen3-VL-8BHR-Bench-4K78.082.5+4.5[+1.8, +7.2]
Qwen3-VL-8BHR-Bench-8K74.480.4+6.0[+3.2, +8.8]

[3] Joint Analysis of Inference Efficiency and Accuracy Gains [Existing]

Appendix Table 19 reports end-to-end inference latency on NVIDIA A800 GPUs. We integrate these results with Table 1 to provide a comparison.

Table W2-3. Accuracy and inference efficiency comparison on Qwen2.5-VL-7B.

MethodV* Bench (%)HR-Bench-4K (%)HR-Bench-8K (%)Latency (s)Peak Memory
Base79.171.567.92.20—
ViCrop82.274.069.35.9821 GB
DeepEyes90.175.172.612.5621 GB
DeepScan90.675.072.460.3128 GB
HiDe91.176.876.35.3622 GB
EviSpec92.180.981.66.4322 GB

(1) Accuracy gains over Base. EviSpec incurs an additional 4.23 s latency (2.20 → 6.43 s), while improving accuracy by 13.0, 9.4, and 13.7 percentage points across the three benchmarks.

(2) Trade-off against HiDe. Compared with the more directly comparable training-free method HiDe, EviSpec adds only 1.07 s latency (20.0%), while achieving accuracy gains of 1.0, 4.1, and 5.3 percentage points, respectively. Both methods require 22 GB peak GPU memory.

(3) Efficiency advantages.

  • DeepEyes: EviSpec reduces latency by 48.8%, while improving V* Bench accuracy by 2.0 percentage points.
  • DeepScan: EviSpec reduces latency by 89.3%, achieving 9.38× faster inference, while improving accuracy by 1.5, 5.9, and 9.2 percentage points across the three benchmarks.

Overall, EviSpec introduces additional computation over direct inference but achieves a accuracy-efficiency trade-off. We do not position it as a replacement for direct inference, but as a training-free enhancement for accuracy-sensitive high-resolution visual reasoning.

We will move these joint comparisons into the main text to make the performance gains and associated computational costs more transparent.

AuthorsOpenReview ↗

Reply for Y3ge [Question1]: We sincerely thank the reviewer for recognizing our design rationale and insights! We have addressed all concerns and hope our responses will merit a higher rating.

[Q1] Configuration Mapping and the Compiler's Independent Contribution

We thank the reviewer for requesting clarification. The full EviSpec TCS pipeline reproduces 92.1%, while Appendix Table 9's Matched Prompt-family Comparison most directly isolates the Compiler's contribution (84.3% vs. 77.0%). Appendix Tables 8 and 9 establish the contribution of structured requests under different matched protocols. We also add a Structured-vs-Generic comparison under the main configuration.

Appendices B.1–B.3 document T/C/S and Generic prompts, input scopes, output formats, localization, and evidence packing. We submitted the code and will consolidate these details in a unified configuration map.

[1] Configuration Mapping for Tables 1, 8, 9, and 11

Table Q1-1. Configuration mapping of V Bench experiments.*

ConfigurationTable 1 / 3Table 8Table 9Table 11
PurposeMain EvaluationEqual-search-budgetMatched Prompt-familyComponent Diagnostic
BackboneQwen2.5-VL-7BQwen2.5-VL-7BQwen2.5-VL-7BQwen2.5-VL-7B
DatasetV* Bench (191)V* Bench (191)V* Bench (191)V* Bench (191)
CompilerT/C/ST/C/S vs. G1/G2/G3T/C/S vs. Matched GenericFixed T/C/S across all four configurations
Localization Layer15151515
Attention ProcessingCalibratedCalibratedCalibratedLegacy / Calibrated
Region FormationAdaptiveAdaptiveAdaptiveLegacy / Adaptive
Evidence PackingSeparate ViewsSingle CanvasSingle CanvasSeparate Views
Evidence Resolution448×448448×448448×448448×448
Image Processor Budget16,384×28²16,384×28²16,384×28²16,384×28²
DecodingGreedyGreedyGreedyGreedy

[2] Configuration Reproducing the Main 92.1% Result

The main result uses the full EviSpec pipeline, not a diagnostic configuration:

  1. Fixed T/C/S prompts generate Target-role, Canonical full-query, and Stem-only requests.
  2. Remove empty/exact duplicates; retain valid requests in T/C/S order.
  3. Extract intermediate-layer attention; apply Prompt Calibration and Adaptive Region Formation.
  4. Build one Compact Evidence View per valid request; remove exact duplicates.
  5. Pass the original image, retained views, and unchanged question to the final Answering Stage.

This configuration yields 92.1% on V Bench with Qwen2.5-VL-7B* (Tables 1 and 3). We will provide its complete configuration identifier, including the Localization Layer, Evidence Resolution, Image Processor Budget, Evidence Packing, and Decoding Settings, ensuring a unique and explicit execution configuration.

[3] Matched Experiments Isolating the Compiler's Contribution

Appendix Table 9's Matched Prompt-family Comparison provides the clearest isolation, not cross-protocol accuracy comparisons: T/C/S and Generic share input scopes, output labels, localization, and final answering; only Request Construction varies.

Table Q1-2. Matched control for the Compiler's contribution (Existing).

MetricMatched GenericEviSpec TCS
Accuracy (%)77.084.3
Exclusive Successes721
Avg. Distinct Requests2.91.9
Selected Area (%)69.68.0

TCS gains 7.3 percentage points, retaining fewer requests and selecting more compact regions. Thus, request organization affects evidence and final accuracy under fixed downstream procedures.

Appendix Table 8 provides an independent equal-search-budget control:

Table Q1-3. Equal-search-budget comparison (Existing, Appendix Table 8).

MetricGeneric-3EviSpec TCS
Accuracy (%)79.188.5
Avg. Distinct Requests2.61.9
Improvement—+9.4 pp

Both attempt three requests, yet TCS is more accurate with fewer retained requests. The Compiler's contribution is supported by matched gains of +7.3 and +9.4 pp, not cross-protocol comparisons of 92.1%, 88.5%, 84.3%, and 93.2%.

Table 11's 93.2% diagnoses Attention Processing and Region Formation, not the independent contribution of Request Construction.

[4] Revisions and Reproducibility

The original paper provides prompts, algorithms, and settings, but their cross-protocol mapping needs clarification. We will add a Configuration Mapping Table linking each experiment to its configuration file and listing prompts, Localization Wrapper, Evidence Packing, Resolution, Decoding, outputs, and execution commands for every reported result where applicable.

In summary, the full EviSpec configuration reproduces 92.1%, while matched Prompt-family and Equal-search-budget comparisons isolate the Compiler's contribution under fixed downstream protocols. Mechanism attribution should rely on these controlled structured-vs-generic gains, not cross-protocol accuracies.

We thank the reviewer for emphasizing reproducibility. We will clarify these mappings in the revised manuscript.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.