- Rating
- 4
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
This paper presents UHR-Micro, a diagnostic benchmark designed to systematically evaluate micro-level perception in Earth observation VLMs, exposing a fundamental failure mode termed the "resolution illusion"—where higher input resolution fails to translate into effective micro-evidence grounding. The core contributions and empirical findings are:
- UHR-Micro Benchmark: As the central contribution, UHR-Micro provides 11,072 deterministic instructions grounded in 1,212 native 4K+ remote sensing images across 16 tasks. By constraining target objects to occupy under 0.01% of the canvas on average and enforcing verifiable outputs, the dataset strictly isolates spatial grounding bottlenecks from language priors and coarse scene-context shortcuts.
- Diagnostic Evaluation & Empirical Findings: Extensive testing across open/closed general VLMs and geospatial models reveals pervasive failures in visual evidence access. The benchmarks show that simply increasing input resolution, scaling model parameters, or adopting domain-specific tuning fails to resolve the resolution illusion.
- Reference Baseline (MAP): To validate that evidence access—rather than underlying recognition capacity—is the primary bottleneck, the authors introduce a simple, training-free reference baseline (Micro-evidence Active Perception, or MAP). By dynamically maintaining a spatially registered evidence state, MAP yields a consistent +8.65% average gain across backbones, confirming the benchmark's diagnostic insights on UHR reasoning.
Critical strengths
- A Sharp and Grounded Problem Perspective: The paper brings much-needed clarity to Earth observation VLMs by questioning a common assumption: that higher input resolution automatically yields better fine-grained perception. Pinpointing the gap between nominal resolution and actual micro-evidence grounding provides a solid, realistic perspective on the field's current bottleneck.
- Disciplined Benchmark Construction: UHR-Micro isolates micro-level reasoning effectively. By restricting targets to <0.01% of the canvas and enforcing deterministic outputs, the benchmark strips away language priors and dataset shortcuts, serving as a strict diagnostic stress test for spatial grounding.
- Sound and Practical Active Perception Paradigm: MAP offers a logical alternative to costly sliding-window search. Maintaining a spatially registered evidence state and querying additional crops only when necessary treats UHR perception as targeted evidence acquisition, yielding a consistent +8.65% boost across backbones under modest query budgets.
Critical weaknesses
While the paper is technically solid and well-executed, I would encourage the authors to clarify two minor points during the rebuttal:
- Part-Level Segmentation Performance and Pipeline Bottlenecks: As shown in Table 3 and Table 17, performance on Component Segmentation (CS) remains noticeably low across all evaluated models, with MAP yielding smaller relative gains compared to object-level tasks. It would be helpful if the authors could clarify whether this limitation stems from VLM feature extraction bottlenecks at the part level, or if the current Box-to-SAM2 prompt conversion protocol incurs spatial error amplification on minuscule targets.
- Prompt Sensitivity of the Baseline MAP: The MAP baseline uses prompt-guided queries to direct active crop selection. It would be helpful if the authors could briefly comment on the robustness of this mechanism to prompt variation—specifically, whether minor changes in prompt template phrasing impact the consistency of crop location proposals across different VLM backbones.
Questions for authors
- Could the authors clarify whether the lower performance on Component Segmentation (CS) is primarily driven by VLM localization errors (imprecise bounding boxes) or SAM2's prompt sensitivity when processing ultra-small target crops?
- In the MAP baseline, does the evidence registration accuracy vary significantly depending on whether the underlying VLM uses static patching (e.g., CLIP-based) versus dynamic resolution vision encoders (e.g., AnyRes/InternVL)?
Dear Reviewer,
Thank you again for your thoughtful feedback. We’ve posted our response and updated the manuscript in light of your comments. We’d be happy to provide further clarification if helpful.