acceptodds
Under review as a conference paper at ICLR 2027

KG-ReflectGround: Evidence-Guided Zero-Shot 3D Visual Grounding

Abstract

Zero-shot 3D visual grounding (3DVG) requires reliable spatial reasoning without task-specific training, yet achieving both strong grounding performance and computational efficiency remains challenging. Existing LLM-based approaches generally incur lower inference costs but remain limited in capturing visually critical cues, whereas MLLM-based approaches provide stronger visual disambiguation at the expense of substantially higher computational overhead. To reconcile these competing requirements, we propose KG-ReflectGround, an efficient zero-shot 3DVG framework that adaptively allocates spatial reasoning and visual verification according to evidence gain. The framework constructs a query-aware sparse spatial knowledge graph over candidate 3D object instances, prioritizing discriminative relations to provide compact, query-relevant structural evidence. Relation-ordered evidence-gain reasoning progressively refines the candidate set and adaptively halts when additional relational evidence provides limited benefit, thereby reducing reasoning cost. An evidence-aware verification policy further assesses whether the accumulated structural evidence is sufficient and selectively invokes MLLM-based visual verification only for ambiguous or appearance-dependent queries. Experiments on ScanRefer and Nr3D demonstrate a favorable balance between zero-shot grounding performance and multimodal computation. KG-ReflectGround achieves 52.0% Overall [email protected] and 47.8% Overall [email protected] on ScanRefer, with an average inference latency of 3.48 s per query.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
zero-shot 3D visual grounding, 3D scene understanding, spatial reasoning, multimodal reasoning, vision-language models, adaptive inference

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_3Ze3OpenReview ↗
Rating
2
Confidence
4
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

The paper proposes KG-ReflectGround, a zero-shot 3D visual grounding framework that combines query-aware sparse spatial graph reasoning with selective multimodal verification. It constructs a compact spatial knowledge graph over candidate objects, performs relation-ordered evidence-gain reasoning with adaptive halting, and invokes MLLM-based visual verification only when structural evidence is deemed insufficient.

The main contributions are: (1) an efficient zero-shot 3DVG framework that integrates explicit spatial reasoning with selective MLLM verification; (2) a query-aware sparse spatial knowledge graph and relation-ordered evidence-gain reasoning with adaptive halting; and (3) an evidence-aware verification strategy that selectively activates multimodal reasoning when structural evidence is insufficient.

The method is validated on ScanRefer and Nr3D under the ground-truth-object setting using Qwen3-VL-8B as the backbone. The paper reports zero-shot ScanRefer Overall [email protected]/0.5 of 52.0/47.8 and Nr3D Overall accuracy of 49.2, together with comparisons to supervised and zero-shot methods, ablations, adaptive-halting analysis, and qualitative examples.

Critical strengths
  • Clear practical motivation. The paper addresses the accuracy–computation trade-off in zero-shot 3D grounding by combining structural reasoning with selective visual verification. Allocating multimodal computation according to query requirements is a practically relevant direction.
  • Several relevant experimental controls. The paper includes a same-backbone SeeGround comparison using Qwen3-VL-8B, component ablations, and fixed-depth versus adaptive-halting comparisons. These provide an initial basis for assessing the proposed components.
Critical weaknesses

Major concerns

  1. The candidate object setting may lead to an unfair comparison.
    Appendix A.1 states that ScanRefer and Nr3D are evaluated under the ground-truth-object setting using annotated ScanNet object instances as candidates. However, Table 1 does not specify the candidate source used by each baseline. According to the public implementations of methods such as SeeGround, they use object lists predicted by Mask3D. If this is the case, the annotated object list used in this paper provides a stronger prior in terms of instance localization, category, and object count, and the two settings are not directly comparable. The authors should clarify the candidate source for each method and indicate which rows in Table 1 are comparable under the same setting. Annotated objects are available for ScanNet-based benchmarks, but an accurate object list cannot be assumed in real deployment; the conclusions should be qualified accordingly.

  2. The speed comparison does not specify the conditions under which each method was measured, so the latency difference cannot be attributed to the algorithm.
    Table 1 includes methods with different backbone sizes, such as Qwen2-VL-72B and Qwen3-VL-8B. If a larger-model baseline is compared with the 8B model used in this paper, the latency difference does not demonstrate algorithmic efficiency. The manuscript also does not specify the timing boundary: whether the reported time includes query parsing, neighbor search, graph construction, edge reweighting, candidate filtering, rendering, and text/vision model calls. A unified timing protocol and a breakdown into scene-level and query-level costs are needed; otherwise the efficiency claim is not supported.

  3. The core operations are not fully defined, and the figures are not consistent with the algorithmic procedure.
    The geometric scoring function ϕr\phi_r (Eq. 5), the candidate compatibility function mm (Eq. 10), and the matched anchor object aka_k (Eq. 11) are not fully defined. It is also unclear what aa and bb in the text–graph conflict formula in Eq. (17) correspond to in the query. Figure 3 shows “Halting-weighted Evidence” with candidate reweighting factors such as ×1.6 and ×1.2, but the equations and pseudocode do not explain how these factors are obtained, at which step they are applied, or whether they affect the final prediction. The candidate counts and normalized scores in Figures 4 and 5 also cannot be reproduced from the manuscript. These issues directly affect whether the method can be independently implemented and verified.

  4. The visual routing claim is too strong, and the current evidence does not support “invoking vision only when truly needed.”
    Candidate entropy, top-two margin, text–graph conflict, and appearance dependency are heuristic rules designed by the authors; they do not guarantee that the selected queries actually require visual information. The manuscript also does not provide verifiable evidence that the routed cases correspond to queries that need visual observation. Showing that performance drops when images are removed is still insufficient: a close score may indicate that color information is genuinely needed, or that the spatial relation was computed incorrectly; a query containing “red” may not require vision if there is only one chair in the scene; and if the true target has already been incorrectly pruned, looking at the remaining candidates may not help. The authors therefore need to provide evidence supporting this claim or weaken the statement.

  5. The kNN geometric graph is reasonable but conventional, and the current evidence is insufficient to support strong novelty and generalization claims.
    Building a kNN graph from object geometry and organizing spatial relations as a graph is a reasonable but relatively conventional design; this alone does not establish strong novelty. ScanRefer and Nr3D are both based on ScanNet RGB source and cannot be regarded as fully independent or heterogeneous datasets. Therefore, results on multiple benchmarks alone cannot demonstrate transferability across data sources or deployment conditions. The authors should more precisely define the contribution of the method or narrow the generalization claim.

  6. The relation coverage, complex constraint composition, and unsupported queries are not explained.
    The manuscript describes a framework that derives pairwise relation weights from object geometry, but it does not provide the full relation set, the scoring function for each relation, or the composition algorithm for complex constraints. It also does not explain how unknown relations are handled. The relations shown in Figure 6, such as between, middle, center, and opposite, need to be mapped to the implementation item by item. If these relations are actually handled by the subsequent LLM/MLLM rather than by the graph, the responsibility boundary between graph reasoning and model reasoning should be clarified. Fixed kNN and irreversible candidate deletion may also miss necessary reference objects; the coverage capability and failure cases need to be discussed.

Minor

  • The Overall aggregation in Table 1 needs to be checked. Under the official ScanRefer evaluation and the public implementations of the compared methods, Overall should be the query-count-weighted accuracy of Unique and Multiple, with the same subset proportions used for both IoU thresholds. Some rows in Table 1 do not satisfy this relationship, and the discrepancy cannot be explained simply by rounding in the displayed precision. I am not assuming a specific source of error, but this affects the credibility of the main performance results. The authors should provide the number of queries and correct predictions for each subset, the aggregation formula, and verify whether there is a transcription or aggregation error in Table 1.
  • The fonts and annotations in Figures 2 and 3 are too small to read at normal page size.
  • An actual execution example would help readers verify the query parsing, anchor objects, candidate changes, halting signals, and visual verification process.
Questions for authors

See Weakness.

Reviewer_UneROpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

This work decomposes the localization reasoning based on 3D segmentation results in 3DVG into two parts. One part relies on an LLM to perform reasoning using textual information derived from 3D segmentation results, and the other part leverages an MLLM for reasoning with visual assistance, so as to address the high latency of localization reasoning. The research problem has certain value. However, the contributions of this paper only lie in optimizations on the reasoning execution pipeline and candidate set. It adopts relation based evidence augmented reasoning to iteratively refine the candidate set, which remains an optimization of the multi step execution strategy for each localization reasoning. Compared with one step reasoning, the paradigm adopted in this paper that achieves localization through multiple rounds of interaction with large models is inherently inefficient. In addition, the design of deploying multiple large models simultaneously is also inefficient, such as separate LLMs for language parsing and MLLMs for visual reasoning, resulting in relatively limited overall contributions. Furthermore, some of the experimental results are questionable.

Critical strengths

Research on accelerating 3DVG reasoning is a worthwhile direction to explore. The design proposed in this paper that splits localization reasoning based on 3D segmentation results into textual information reasoning and visual assisted reasoning exhibits certain innovation.

Critical weaknesses

(1) The latency experimental results designed in this paper are questionable. For example, in the latency experiment of Table 5 Per query latency, the latency of SeeGround is higher than the method proposed in this paper. In the reasoning process of SeeGround, each localization task accesses the VLM twice in total, once for text parsing which is the same as this paper, no visual image is submitted and only one round of text parsing is performed and once for visual reasoning. By contrast, the method in this paper requires an average of 3.2 reasoning steps as shown in Table 6. Can its reasoning time be far less than the time cost of one VLM invocation in SeeGround? It is difficult to convince readers of the authenticity of the experimental data provided in this paper. Besides, the execution pipeline of this paper is far more complicated than that of SeeGround, casting doubts on the reliability of its experimental results.
(2) This paper does not specify the version and model of the adopted LLM. Is it the language module within Qwen3-VL-8B or an independent language module? Without visual auxiliary information, reasoning merely from 3D textual information of mask segmentation on a small LLM of 8B cannot easily achieve high accuracy, let alone surpass the reasoning accuracy with visual information assistance. Nevertheless, results in Table 4 show that invoking the MLLM for only 31.6% of queries can greatly outperform methods fully relying on MLLMs. This constitutes another major experimental concern.
(3) Following comment (2), this paper should report the accuracy under full MLLM invocation to explore the upper bound of accuracy for the proposed method.
(4) The proposed method uses an additional lightweight LLM for text parsing, yet the corresponding model version is not specified. Meanwhile, deploying multiple large models concurrently is impractical and highly redundant for real world applications.

Questions for authors

We hope the authors can respond to the concerns raised in the above Critical Weaknesses.

Reviewer_G6vkOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces KG-ReflectGround, an efficient inference framework for "zero-shot 3D visual grounding" (3DVG). The framework addresses the limitations of existing approaches: methods based on large language models (LLMs) often lack sufficient visual information, while those based on multimodal large language models (MLLMs) incur excessively high computational costs.

Critical strengths
  1. The overall framework design is clear, with modules that effectively complement one another.
  2. The method achieves competitive results on two benchmark datasets.
  3. The paper provides a preliminary quantitative analysis of reasoning efficiency.
Critical weaknesses

W1. There is a lack of key comparative experiments to directly validate the effectiveness of "Adaptive Verification."

W2. "Progressive Candidate Pruning" carries the risk of irreversible error propagation. According to Equation (10), the candidate set is filtered incrementally via hard thresholding, such that .

W3. The experimental setup weakens the claims regarding the method's generalization capability.

W4. Comparisons of computational efficiency require more rigorous controls.

Questions for authors

see the weakness

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.