- Rating
- 2
- Confidence
- 2
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 3
Read review
Paper summary
The paper investigates object-to-image (O2I) ratio mismatch in visual instance retrieval. Using controlled renderings of 3,021 Objaverse objects at five camera distances, it argues that differences in the fraction of the image occupied by an object explain much of the retrieval degradation commonly attributed to resolution loss. It also reports directional asymmetry between tight-to-wide and wide-to-tight retrieval. Motivated by these observations, the authors evaluate query-side scale averaging, attention-masked augmentation, detector-based crop reranking, and LoRA adaptation. On ILIAS 100M, masked augmentation improves mAP@1000 from 26.5 to 29.2, while crop reranking achieves 42.0.
Critical strengths
-
A useful diagnostic question. Distinguishing lost visual detail from sensitivity to object framing is relevant to understanding global retrieval representations. The controlled rendering setup and fixed-pixel canvas experiment provide complementary evidence that resolution loss alone cannot explain the observed behavior.
-
Segnificant experimental coverage. The study evaluates twelve backbones and includes augmentation controls, query-expansion comparisons, O2I-stratified results, and additional experiments on Oxford and Paris. These analyses go beyond simply demonstrating that cropping improves retrieval.
-
Practical improvements. Query-side augmentation improves full-scale ILIAS retrieval while preserving the existing gallery index. The LoRA experiment also suggests that some of these gains can be obtained with a single inference pass, although it requires adapting the gallery representation.
Critical weaknesses
- The quantitative causal attribution is stronger than the experimental controls support. Section 4.2 interprets the improvement after cropping and resizing as the contribution of O2I mismatch, and the remaining error as resolution-only failure. However, cropping also changes spatial sampling, object coverage across tokens, and the subsequent resizing pipeline. The rendering setup additionally retains perspective differences, as acknowledged in Section 3. Thus, the experiment demonstrates that canonical cropping recovers substantial performance, but does not uniquely establish a decomposition of errors into O2I and resolution components. A factorial experiment varying object detail and frame occupancy independently would provide stronger support for the headline attribution.
Similarly, Appendix L.2 compares attention masking with oracle cropping. The difference includes changes in object sampling and spatial layout, so interpreting it as a lower bound attributable specifically to O2I requires additional justification. Moreover, this experiment uses ground-truth boxes and the ILIAS core gallery; it does not directly decompose the actual OWLv2 reranking improvement on ILIAS 100M.
-
Limited methodological novelty and insufficient direct comparison with closely related methods. Scale averaging, regional descriptors, and crop-based maximum-similarity matching are established retrieval techniques. Patchify, including its detector-proposal variant, is particularly close to the proposed reranker. The paper acknowledges this relationship but does not compare these methods under a shared backbone, shortlist, and computational budget. The comparison against a PE-L + AMES leaderboard result changes multiple components simultaneously and therefore does not isolate the advantage of the proposed approach. The paper’s contribution must consequently rest primarily on the diagnostic analysis, making the attribution concerns particularly consequential.
-
The fixed-pixel evidence is narrower than the broader claims. Section 4.3 provides a useful control, but evaluates only DINOv3-L and reports cosine-similarity drift rather than retrieval accuracy. A reduction in self-similarity does not necessarily imply worse rankings if similarities to other instances also change. Reporting retrieval metrics under this control, and extending it to representative additional backbones, would better connect the mechanism to the claimed retrieval failure.
-
The asymmetry is observed but not sufficiently explained. Cosine similarity between a query and its positive is symmetric. Directional differences in retrieval therefore depend on how the query compares with competing gallery instances. The results establish asymmetric rankings under the chosen protocol, but the interpretation as a directional property of object-support encoding needs analysis of positive and negative similarity distributions, rather than aggregate MRR alone.
-
The practical efficiency comparison remains incomplete. GP and MSA require five and three query encodings, respectively. The reranker uses approximately 31 crop encodings per candidate image, in addition to detection. Caching can amortize this work, but does not establish practical latency or storage requirements for previously unseen candidates. End-to-end latency, cache assumptions, storage, and accuracy–cost comparisons are needed. Appendix M also shows that LoRA requires re-encoding the gallery; its single-pass inference advantage should therefore be distinguished from the index-preserving benefit of query-side augmentation.
-
Related work could better distinguish O2I mismatch from task-dependent resolution requirements. CARES (Context-Aware Resolution Selector for VLMs) studies selecting the minimal sufficient resolution for a given image-question pair. Although it addresses VQA rather than instance retrieval, discussing this work would help clarify the distinction between preserving task-relevant visual detail and matching object-to-image ratios. Using such aproach could work in O2I as well, with classifier training and I think that without implementing it, the authors failed to mention it.
Questions for authors
I acknowledge I might not be an expert but please adress the weaknesses i've stated.