acceptodds
Under review as a conference paper at ICLR 2027

Beyond Fixed Sampling: Uncertainty-Aware Retrieval and Spatially-Grounded Reasoning for Fine-Grained Visual Recognition

Abstract

Recent fine-grained visual recognition methods increasingly adopt a retrieve-then-reason paradigm powered by multimodal large language models. However, this paradigm is often hindered by two critical bottlenecks: Rigid Selection Inflexibility and Ungrounded Speculation. Specifically, conventional top- retrieval fails to adapt to varying sample difficulties, frequently introducing irrelevant noise or missing crucial candidates. Furthermore, holistic image processing during reasoning lacks fine-grained spatial grounding, which often triggers hallucinations and yields unverified visual evidence. To overcome these issues, we propose \ours, a framework designed to refine both the retrieval and reasoning trajectories. First, we introduce Dynamic-, an uncertainty-aware candidate selection mechanism that adapts the probability budget to sample-level confidence and determines the candidate set through cumulative-probability selection. Second, we develop a Discriminative Attribute Anchor (DAA) mechanism for the reasoning phase. By synergizing spatially-grounded evidence with decoupled class attributes, DAA guides the model to anchor onto discriminative local regions while filtering out non-essential semantic features. Comprehensive experiments demonstrate that \ours establishes new state-of-the-art benchmarks across 11 fine-grained datasets, achieving a significant average accuracy improvement of .

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Fine-Grained Visual Recognition, Multimodal Large Language Models

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_qrJ4OpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This article proposes an untrained P-Gaze framework to address the problems of fixed sampling rigidity and unfounded speculation in the fine-grained visual recognition retrieval inference paradigm. Design a dynamic uncertainty aware candidate selection mechanism and adaptively determine the candidate set based on sample confidence; The inference end combines attribute decoupling and spatial evidence verification, filters redundant features, and binds local image regions to alleviate the illusion of large models

Critical strengths

1.This article proposes an untrained P-Gaze framework to address the problems of fixed sampling rigidity and unfounded speculation in the fine-grained visual recognition retrieval inference paradigm. 2.Design a Dynamic-p uncertainty aware candidate selection mechanism, adaptively determine the candidate set based on sample confidence, and use the inference end combined with comparative attribute decoupling and spatial evidence verification to filter redundant features and bind local image regions, alleviating the illusion of large models

Critical weaknesses
  1. For a fine‑grained visual recognition work, the method descriptions are overly verbose. In particular, the figures contain excessive intertwined logical branches, which makes it difficult for readers to quickly grasp the core workflow.
  2. The overall novelty is limited. This work mainly assembles existing off‑the‑shelf modules and can be regarded as an engineering‑optimized pipeline built upon multiple large models. It combines VLM, MLLM and LLM collaboratively for inference, which deviates from the core spirit of fine‑grained visual recognition that emphasizes visual‑centric kernel design. This assembled framework also brings considerable computational overhead.
Questions for authors
  1. For a fine‑grained visual recognition work, the method descriptions are overly verbose. In particular, the figures contain excessive intertwined logical branches, which makes it difficult for readers to quickly grasp the core workflow.
  2. The overall novelty is limited. This work mainly assembles existing off‑the‑shelf modules and can be regarded as an engineering‑optimized pipeline built upon multiple large models. It combines VLM, MLLM and LLM collaboratively for inference, which deviates from the core spirit of fine‑grained visual recognition that emphasizes visual‑centric kernel design. This assembled framework also brings considerable computational overhead.
  3. There are fairness concerns in experimental comparisons. Comparing this multi‑large‑model composite system against previous standalone single‑model baselines is inappropriate. It is analogous to deploying a multi‑agent system to beat problems originally targeted for single‑model solutions. Such evaluation setup is not a desirable paradigm for academic papers.
  4. Theoretical analysis is shallow. Many conclusions are directly drawn from superficial statistical observations. For instance, Figure 2 only demonstrates commonplace phenomena and cannot strongly support the core argument. Besides, heavy reliance on LLMs throughout the research pipeline is quite noticeable.
Reviewer_pCaCOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes P-Gaze, a framework addressing candidate set rigidity in fine-grained recognition through adaptive retrieval and spatial reasoning. Dynamic-p adjusts candidate scope based on classifier uncertainty, CAD contrasts discriminative attributes across candidate classes, and S-GEV extracts local evidence using predefined body part lists. The method outperforms SCAN baseline on most datasets.

Critical strengths

1. Explicit incorporation of uncertainty into retrieval strategy
Existing methods apply fixed candidate counts or cumulative probability thresholds uniformly across all samples. P-Gaze uses ambiguity U to modulate per-sample thresholds, narrowing candidates for high-confidence samples while expanding search space for ambiguous ones. This advances the retrieve-then-reason paradigm from fixed parameters to sample-adaptive selection without additional training.

2. Division of labor between retrieval and reasoning
Dynamic-p adaptively selects candidate sets, CAD contrasts discriminative attributes of candidate classes, and S-GEV extracts spatial evidence in local regions. This design allows each module to focus on specific subtasks: retrieval controls search space, attribute comparison filters shared features, and spatial localization mitigates hallucination. Synergy between Dynamic-p and S-GEV emerges in specific scenarios.

Critical weaknesses

1. Limited innovation depth in Dynamic-p
The method essentially adds confidence weighting to fixed top-p. Ambiguity U depends only on top-1 probability and class count, with candidate sets still determined by cumulative probability. The paper does not explain how best-k determines k values per sample, making it difficult to quantify the gap between the current method and the theoretical upper bound.

2. Questionable scalability of spatial reasoning module
S-GEV requires manually defined body part lists for each dataset, depending on domain knowledge with high transfer cost. The paper does not explore whether part concepts can be automatically extracted from CAD attributes, nor does it analyze principles for selecting part granularity. Ablation experiments quantifying the contribution of manual part lists are missing.

3. Experimental design and presentation issues
The abstract claims state-of-the-art on fine-grained datasets, yet 5 of 11 datasets are general classification tasks, and CUB fails to surpass the SCAN baseline on both backbones. Hallucination evaluation relies primarily on automatic metrics without systematic human verification to validate reliability.

Questions for authors
  1. Table 3 shows the best fixed k setting averages only 63.43%, far below best-k's 78.21%, indicating best-k must be a per-sample varying upper bound rather than a global fixed value. The paper does not explain how this k is determined: does it use ground-truth class labels? If true labels are indeed used, can the model still misclassify after selecting this k?

  2. Would S-GEV remain effective if body part lists were automatically generated from CAD attributes rather than manually defined?

  3. Although hallucination evaluation provides automatic metrics and error cases, can human verification results be added to validate the reliability of automatic evaluation?

Reviewer_ym8eOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

The paper proposes P-Gaze, an interpretable framework designed to jointly refine both the retrieval and reasoning processes in fine-grained visual recognition. P-Gaze consists of a three-stage pipeline: (i) Dynamic Candidate Selection, which performs adaptive retrieval of candidate categories inspired by evidential deep learning; (ii) Contrastive Attribute Decoupling, which leverages a large language model (LLM) to identify discriminative features among the retrieved categories; and (iii) Spatially Grounded Evidence Verification, which grounds the extracted evidence using the spatial detection capabilities of a multimodal large language model (MLLM). P-Gaze demonstrates consistent improvements across 11 benchmarks, including 6 fine-grained recognition datasets.

Critical strengths
  1. The idea of dynamic candidate selection, along with its proposed approach grounded in evidential deep learning, is interesting and well-motivated.
  2. The ablation studies are thorough, clearly demonstrating the contribution of each stage in the pipeline.
Critical weaknesses
  1. Insufficient motivation for retrieval-augmented reasoning in FGVC: The paper does not sufficiently motivate why a retrieve-then-reason paradigm is advantageous over end-to-end approaches for fine-grained visual classification. The authors should provide a clearer discussion, supported by analysis or failure cases, illustrating when and why decoupling retrieval from reasoning leads to better fine-grained discrimination compared to monolithic classification or direct MLLM inference.

  2. Limited novelty: Only the dynamic candidate selection stage appears to introduce substantial novelty and demonstrate meaningful improvements. Contrastive Attribute Decoupling (CAD) has been previously explored in FuDD (https://arxiv.org/abs/2311.07593), while the gains from Spatially Grounded Evidence Verification (S-GEV) appear marginal based on Table 2.

  3. Missing baselines: The authors do not compare against a straightforward baseline in which an MLLM serves as both the retriever and the reasoner. Such a comparison is important to establish the benefits of the proposed retrieval and reasoning pipeline.

Questions for authors

I previously reviewed this work at another venue, and the current submission appears largely unchanged, apart from the addition of some baselines in Table 1. Specifically, the baseline using an MLLM as both the retriever and reasoner is still missing. Could the authors clarify what substantive changes have been made since the previous submission and how the feedback from the earlier review has been addressed?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.