- Rating
- 2
- Confidence
- 4
- Informativeness
- 3
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper argues that failures in high-resolution multimodal large language models (MLLMs) should not be attributed solely to insufficient visual resolution or visual distraction. Some question formulations are not well suited to localizing the visual evidence needed for answering, a mismatch the paper terms the evidence-specification gap. EviSpec is introduced, which uses a frozen model to compile questions into three types of localization requests: Target-role (T) extracts answer-bearing targets and necessary reference entities; Canonical full-query (C) extracts canonical entity expressions from the full question, including answer options; and Stem-only (S) performs the same canonicalization after removing the answer options. EviSpec then performs localization using text-to-visual-token attention, combining instruction-prefix prior calibration, Yen thresholding, connected-component selection, and adaptive expansion to construct compact evidence views. Finally, the original image, evidence views, and unmodified question are jointly provided to the model for answering. The paper reports relative improvements across five models and three benchmarks, with additional evaluations on five other visual tasks.
Critical strengths
1.Separating localization requests from the original question used for final answering is a novel angle on the problem. In particular, distractor options and relational descriptions in multiple-choice questions may interfere with attention-based localization.
2.The gains on high-resolution tasks are substantial. If these results can be reproduced under a consistent, clearly specified configuration, they would constitute a meaningful contribution.
3.The method requires neither external detectors nor training additional parameters. Its lower cost relative to multi-round visual search methods gives it some practical deployment value.
Critical weaknesses
1.The SOTA claims go beyond the experimental evidence. The method does not achieve the best performance on every evaluated task. The paper could simply report its demonstrated improvements without claiming SOTA.
2.The method has some novelty, but the “paradigm reframing” claim is clearly overstated. The paper presents the three TCS requests as a new localization interface. However, extracting entities from questions, decomposing complex questions into localizable queries, and jointly feeding cropped images, the original image, and the original question into a model for final answering are ideas already covered by prior work such as V* and FOCUS.
3.Some tabulated results require verification. For example, it is unclear how the overall accuracy of 89.5% in the last row of Table 18 was calculated. Weighting the reported subset accuracies by their sample counts gives (94.8%×115+84.2%×76)/191=90.6%. Please carefully check the reported values.
Questions for authors
- Please provide the details necessary to reproduce the experiments for each backbone model–benchmark combination, including the localization layer indices and prompts. Was an independent development set used to select these configurations? Was V*Bench used repeatedly to select or refine these design choices?
- Please clarify the scope of the option-free conclusion. Does the benefit of the C branch come from canonicalization, additional evidence views, or answer-option-conditioned search?
6 comments
Reply for RRXb [Weak2]: We sincerely thank the reviewer for recognizing our design rationale and insights! We have addressed all concerns and hope our responses will merit a higher rating.
[W2] Clarifying Paradigm Reframing and Differences from V* and FOCUS
We thank the reviewer for raising this concern. While EviSpec shares aspects of the visual reasoning pipeline with V* and FOCUS, the key distinction lies in the research question and the primary intervention point.
[1] Evidence Specification Rather Than a New Visual Search Paradigm
We do not claim that EviSpec is the first to use linguistic information to guide visual search. V* focuses on how to perform question-guided visual search, while FOCUS emphasizes identifying target-relevant visual regions. In contrast, EviSpec investigates how to transform an original question into structured linguistic requests better suited for visual evidence localization, while keeping the downstream localization mechanism fixed.
We term this the Evidence-Specification Gap: an original question specifies the final judgment but may not adequately express the target roles, entity attributes, and relational cues required for accurate localization.
Furthermore, EviSpec requires fewer hyperparameters: V* (CVPR 2024) involves 7, FOCUS (NeurIPS 2025) approximately 9, and HiDe (ICML 2026) approximately 3, whereas EviSpec introduces only one hyperparameter.
Table W2-1. Methodological comparison with V and FOCUS.*
| Dimension | V* | FOCUS | EviSpec |
|---|---|---|---|
| Core Question | How to perform guided visual search | How to identify target-relevant regions | How to specify required visual evidence |
| Intervention | Visual search process | Target relevance and region selection | Linguistic requests before localization |
| Mechanism | Question-guided visual search | Target relevance maps and crops | T/C/S Evidence Specifications |
| Representation | Search results and visual regions | Relevance maps and crops | Complementary linguistic requests |
| Primary Focus | Vision | Vision | Language |
| Final Reasoning | Retrieved visual information | Selected visual evidence | Original image, evidence views, and question |
Thus, our contribution is not the invention of entity extraction, question rewriting, or cropping, but the systematic formulation and evaluation of structured linguistic requests for visual evidence localization.
[2] Direct Comparison with V* and FOCUS [Additional]
To further evaluate EviSpec against existing visual search methods, we conduct direct comparisons on Qwen2.5-VL-7B and Qwen3-VL-8B across three high-resolution benchmarks.
Table W2-2. Performance comparison on Qwen2.5-VL-7B (Additional).
| Method | V* Bench (%) | HR-Bench-4K (%) | HR-Bench-8K (%) |
|---|---|---|---|
| V* | 79.06 | 71.75 | 66.13 |
| FOCUS | 89.53 | 75.50 | 72.75 |
| EviSpec | 92.10 | 80.90 | 81.60 |
| Gain over FOCUS | +2.57 | +5.40 | +8.85 |
| Gain over V* | +13.04 | +9.15 | +15.47 |
Table W2-3. Performance comparison on Qwen3-VL-8B (Additional).
| Method | V* Bench (%) | HR-Bench-4K (%) | HR-Bench-8K (%) |
|---|---|---|---|
| V* | 84.82 | 76.88 | 73.75 |
| FOCUS | 87.43 | 78.13 | 74.38 |
| EviSpec | 91.60 | 82.50 | 80.40 |
| Gain over FOCUS | +4.17 | +4.37 | +6.02 |
| Gain over V* | +6.78 | +5.62 | +6.65 |
Across all six settings, EviSpec achieves average absolute accuracy improvements of 5.23 percentage points over FOCUS and 9.45 percentage points over V*. These comparisons provide additional evidence of EviSpec's effectiveness, while our matched-control experiments under identical downstream localization mechanisms directly support the contribution of Evidence Specification.
[3] Clarifying Paradigm Reframing and Planned Revisions
Following the reviewer's suggestion, we will revise the Abstract, Introduction, Contributions, and Related Work to:
- Remove overly broad paradigm-reframing claims, without implying that entity extraction, question decomposition, or visual cropping are newly introduced techniques.
- Clarify our core contribution as structured Evidence Specification for visual localization, rather than a fundamentally new visual search paradigm.
In summary, EviSpec does not reinvent visual search; instead, it explicitly investigates how the required visual evidence should be specified before localization. We appreciate the reviewer's suggestion, which helps us more precisely position our contribution relative to V* and FOCUS.
Reply for RRXb [Weak3]: We sincerely thank the reviewer for recognizing our design rationale and insights! We have addressed all concerns and hope our responses will merit a higher rating.
[W3] Verification and Correction of Reported Accuracy
We thank the reviewer for carefully examining our experimental results and identifying the inconsistency in Table 18. We have rechecked the underlying predictions and statistical calculations and confirmed that the reported overall accuracy of 89.5% was incorrect.
[1] Source of the Discrepancy
The error arose during table compilation, where the accuracies of the attribute and spatial subsets were incorrectly averaged without accounting for their different sample sizes:
Incorrect: (94.8% + 84.2%) / 2 = 89.5%.
However, V* Bench contains 115 attribute questions and 76 spatial questions. According to the original prediction records, 109 and 64 questions were answered correctly, respectively. Thus, the correct overall accuracy is:
Correct: (109 + 64) / (115 + 76) = 173 / 191 = 90.6%.
Therefore, the correct overall accuracy is 90.6%, rather than 89.5%. This discrepancy resulted from an incorrect aggregation method, not changes in the underlying model predictions.
[2] Corrections and Verification
We have corrected the aggregation procedure and rechecked the overall accuracies for all configurations in Table 18, ensuring that they are calculated directly from the total number of correct predictions and samples.
This correction does not affect our main experimental comparisons or conclusions. We apologize for this statistical oversight and sincerely appreciate the reviewer's careful examination, which has helped improve the accuracy and reliability of our reported results.
Reply for RRXb [Weak1]: We sincerely thank the reviewer for recognizing our design rationale and insights! We have addressed all concerns and hope our responses will merit a higher rating.
We thank the reviewer for highlighting the overly broad SOTA claims. We acknowledge this concern and will revise the statements to accurately reflect the experimental evidence.
We appreciate the reviewer's suggestion, which helps us present more precise and evidence-grounded conclusions.