- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes Render-and-Verify (RaV), a training-free framework that improves frozen supervised 3D visual grounders through VLM-based candidate verification. RaV constructs diverse candidate hypotheses using rotated scenes, selects candidate-specific viewpoints with visibility constraints, and refines rendered images using a frozen diffusion corrector. A frozen VLM then verifies each candidate against the referring expression and re-ranks the predictions. Experiments on ScanRefer, Nr3D, and Sr3D demonstrate improvements across multiple grounding backbones, with reported state-of-the-art results under the detection protocol.
Critical strengths
(1) The paper addresses the difficulty of distinguishing same-class distractors in 3D visual grounding. Using a VLM to verify hypotheses from an existing supervised grounder is a sensible way to combine geometric localization with complementary visual reasoning.
(2) RaV can be applied to pretrained grounders without modifying their parameters or training objectives. Candidate-specific viewpoint selection provides useful visual evidence for verification, and the ablations demonstrate its contribution to grounding performance.
(3) The paper evaluates multiple grounding backbones and VLM families across ScanRefer, Nr3D, and Sr3D. Component ablations, verification-format comparisons, prediction-flip analyses, and computational-cost measurements provide a reasonably comprehensive assessment of the framework.
Critical weaknesses
(1) RaV combines orientation-based candidate generation, visibility-based viewpoint selection, an existing diffusion corrector, and VLM-based re-ranking. Although the integration is useful and supported by ablations, the novelty beyond applying pretrained components to candidate verification remains modest. The distinction from existing candidate-based VLM grounding approaches needs to be established more clearly.
(2) VCF explicitly optimizes candidate visibility, but does not assess whether the surrounding objects needed to verify the referring expression are visible. Moreover, the comparisons with candidate-local selectors change both the model and evidence representation, making it difficult to attribute the gains specifically to relational context. Controlled experiments using the same VLM with full-scene, candidate-only, and context-masked inputs would strengthen this claim.
(3) Improvements on Sr3D are small (+0.7 for TSP3D and +0.2 for PV-Ground), and the appendix shows that ungated verification substantially degrades PV-Ground on this benchmark. This suggests that verification is not uniformly reliable and that conservative re-ranking is important to the reported results. Although margin sensitivity is provided, the selection procedure for the dataset-specific thresholds should be clarified, including whether evaluation labels were used.
(4) RaV adds three grounder forward passes, candidate rendering, diffusion-based correction, and multiple VLM verifications to a supervised grounding pipeline. Although the appendix reports component costs, a direct comparison of end-to-end latency and throughput against the base grounder on matched hardware is missing. Such a comparison is needed to assess whether the accuracy gains justify the additional computation, particularly for applications requiring fast inference.
(5) Lack of comparison with recent zero-shot 3D grounding methods [1]-[4]
[1] Spazer: Spatial-semantic progressive reasoning agent for zero-shot 3d visual grounding, 2025 NIPS
[2] Think, Act, Build: An Agentic Framework with Vision Language Models for Zero-Shot 3D Visual Grounding, 2026 EMNLP
[3] Seeground: See and ground for zero-shot open-vocabulary 3d visual grounding, 2025 CVPR
[4] Seqvlm: Proposal-guided multi-view sequences reasoning via vlm for zero-shot 3d visual grounding, 2025 ACMMM
Questions for authors
Please see the weakness
6 comments
Response to Reviewer kHWn (2/3) - Margin selection and latency (W3, W4)
W3. Sr3D, ungated degradation, and the selection of τ
We agree that the gains on Sr3D are small, and we would like to explain the reason. Sr3D is difficult for VLM-based grounding in general, since its templated spatial expressions carry little appearance information, and the recent zero-shot VLM methods (e.g., SeeGround [2], SPAZER [3], SeqVLM [4], TAB [5], and VLM-Grounder [6]) report ScanRefer and Nr3D only. The verifier is also the least discriminative on Sr3D. The probability that it scores a correct candidate above an incorrect one is 58% on Sr3D, compared with 66% on ScanRefer. This is exactly where the conservative margin is designed to act. It turns the 3.8-point loss of the ungated rule into a small but safe gain, so that RaV never degrades the grounder. To the best of our knowledge, RaV is the first VLM-based method to report Sr3D. We also note that the gains on ScanRefer and Nr3D do not rely on the margin. Table A11 of the main paper shows that RaV without any margin improves EG-3DVG on ScanRefer from 57.07 to 61.57 and PV-Ground on Nr3D from 51.12 to 52.35. The verification itself is therefore reliable on these benchmarks.
Regarding the selection of τ, we apologize for the confusion caused by the brief description. τ is a single scalar per dataset chosen from the coarse grid of Tab. A11 on the validation split, which is the usual hyperparameter sweep in vision tasks. Because RaV is training-free, we did not design a learnable or scene-adaptive margin. To address the concern directly, we re-selected τ with a held-out protocol. In this setting, the validation scenes are split into two halves, τ is chosen on one half and evaluated on the other, and the results of both directions are averaged. The results are shown in Tab. R2.
Table R2. Held-out selection of the margin τ (Overall [email protected]).
| Configuration | τ from the grid sweep (paper) | Held-out selection | Base |
|---|---|---|---|
| ScanRefer, EG-3DVG | 61.67 | 61.61 | 57.07 |
| Nr3D, PV-Ground | 53.27 | 53.13 | 51.12 |
| Sr3D, PV-Ground | 56.39 | 56.39 | 56.18 |
The accuracies change by at most 0.14 points, so the reported gains do not depend on the selection procedure. The final version will report the held-out numbers and the procedure.
W4. End-to-end latency and throughput on matched hardware
We thank the reviewer for this suggestion. Table R3 reports the end-to-end latency per query on one A6000 at batch size 1, using the amortized stage costs of Tab. A10 in the main paper with the early exit included.
Table R3. End-to-end latency per query on one A6000 (ScanRefer, seconds).
| Pipeline | Grounder | Pool (3 passes) | Render + correct | Verify | Total | [email protected] / @0.5 |
|---|---|---|---|---|---|---|
| PV-Ground two-stage | 0.23 | – | – | – | 0.23 | 59.87 / 47.56 |
| + RaV 7B, no corrector | 0.23 | 0.69 | 0.28 | 2.2 | 3.4 | 60.85 / 47.91 |
| + RaV 7B | 0.23 | 0.69 | 2.1 | 2.2 | 5.2 | 61.65 / 48.47 |
| + RaV 72B | 0.23 | 0.69 | 2.1 | 17.0 | 20.0 | 62.20 / 49.05 |
| EG-3DVG two-stage | 0.27 | – | – | – | 0.27 | 58.50 / 52.40 |
| + RaV 7B | 0.27 | 0.82 | 2.1 | 2.2 | 5.4 | 61.24 / 54.33 |
| + RaV 72B | 0.27 | 0.82 | 2.1 | 17.0 | 20.2 | 61.87 / 54.93 |
RaV adds a few seconds per query, and the full validation split runs in 85 minutes on eight A6000s with the 7B verifier. This cost should be weighed against training. Each supervised grounder is trained separately for ScanRefer, Nr3D and Sr3D, and a single run of TSP3D on ScanRefer with the authors' recipe takes 128 GPU-hours on our A6000s. RaV needs no training on any dataset and improves these grounders by 1.6 to 3.4 points with the 7B verifier at about 11 GPU-hours of inference for the whole validation split. Among methods that use a VLM at inference time, RaV is also the fastest. The zero-shot pipelines in Tab. R4 report 23.5 to 50.3 seconds per query, whereas RaV takes about 5 seconds with the 7B verifier because it issues one small-VLM call per candidate on a single rendered view. We will add Tab. R3 to the main paper.
Response to Reviewer kHWn (3/3) - Comparison with zero-shot methods (W5)
W5. Comparison with recent zero-shot methods
We thank the reviewer for pointing us to these works. Table R4 adds the four methods together with IVSGround [1]. Since their evaluation protocols differ in split and annotation, we list the protocol of each method in the table. All numbers are taken from the respective papers.
Table R4. Comparison with VLM-based 3D visual grounding methods on the ScanRefer dataset (Overall [email protected] / [email protected]).
| Method | VLM | Split / annotation | Time per query | [email protected] / @0.5 |
|---|---|---|---|---|
| Full validation split | ||||
| SeeGround (CVPR'25) [2] | Qwen2-VL-72B | full val | – | 44.1 / 39.4 |
| IVSGround (ECCV'26) [1] | Qwen3-VL-32B | full val | – | 56.2 / 49.7 |
| RaV 7B + TSP3D (ours) | Qwen2.5-VL-7B | full val | 4.6 s | 59.86 / 49.42 |
| RaV 72B + PV-Ground two-stage (ours) | Qwen2.5-VL-72B | full val | 20.0 s | 62.20 / 49.05 |
| RaV 72B + EG-3DVG two-stage (ours) | Qwen2.5-VL-72B | full val | 20.2 s | 61.87 / 54.93 |
| 250-sample subset of VLM-Grounder | ||||
| VLM-Grounder (CoRL'24) [6] | GPT-4o | 250-sample subset | 50.3 s | 51.6 / 32.8 |
| SeqVLM (ACM MM'25) [4] | Doubao-1.5-vision-pro | 250-sample subset | – | 55.6 / 49.6 |
| SPAZER (NeurIPS'25) [3] | GPT-4o | 250-sample subset | 23.5 s | 57.2 / 48.8 |
| TAB (EMNLP'26) [5] | Qwen3-VL-32B | 250-sample subset, author-refined annotations | – | 71.2 / 46.4 |
| RaV 72B + TSP3D (ours) | Qwen2.5-VL-72B | 250-sample subset | 19.4 s | 61.5 / 47.4 |
| RaV 72B + EG-3DVG two-stage (ours) | Qwen2.5-VL-72B | 250-sample subset | 20.2 s | 59.0 / 51.0 |
RaV times are per query on a single A6000, including the grounder. The 72B verifier does not fit on one A6000 and is sharded over four GPUs, so for the 72B rows we report GPU-seconds summed over the four GPUs, which is a conservative estimate (the wall-clock time per query is about 7 seconds).
On the full validation split with the original annotations, the strongest zero-shot result is IVSGround at 56.2 / 49.7, and RaV is 6.0 points higher in [email protected] with PV-Ground two-stage and 5.2 points higher in [email protected] with EG-3DVG two-stage. For a like-for-like comparison with the methods evaluated on the 250-sample subset of VLM-Grounder, we re-scored RaV on the same queries. RaV reaches 61.5 [email protected] with TSP3D and 51.0 [email protected] with EG-3DVG two-stage, which are both higher than those of SPAZER and SeqVLM.
TAB [5] reports a higher [email protected] of 71.2, but it is evaluated on a different version of the benchmark. The authors manually reviewed the 250-sample subset and rewrote its annotations. In particular, ambiguous references that admit several valid candidates were resolved by adding exclusive contextual anchors, and expressions with category or spatial errors were corrected. Removing the ambiguity among same-class objects makes the task considerably easier, and this ambiguity is precisely the failure mode that RaV is designed to resolve. TAB does not report results on the original annotations or an ablation of the refinement, so its numbers cannot be compared with results on the original benchmark. The reported times come from different hardware and should be read as order-of-magnitude comparisons. We will add Tab. R4 and this discussion to Section 4.2.
References
[1] Chiang et al., Where to look matters: Learning influential views for VLM-based 3D visual grounding. ECCV 2026.
[2] Li et al., SeeGround: See and ground for zero-shot open-vocabulary 3D visual grounding. CVPR 2025.
[3] Jin et al., SPAZER: Spatial-semantic progressive reasoning agent for zero-shot 3D visual grounding. NeurIPS 2025.
[4] Lin et al., SeqVLM: Proposal-guided multi-view sequences reasoning via VLM for zero-shot 3D visual grounding. ACM MM 2025.
[5] Wang et al., Think, act, build: An agentic framework with vision language models for zero-shot 3D visual grounding. EMNLP 2026.
[6] Xu et al., VLM-Grounder: A VLM agent for zero-shot 3D visual grounding. CoRL 2024.
Response to Reviewer kHWn (1/3) - Novelty and context ablation (W1, W2)
We sincerely thank the reviewer for recognizing that verifying a supervised grounder with a VLM is a sensible design and that RaV applies to pretrained grounders without modification. In brief, we add a same-VLM context ablation and an anchor-visibility analysis for W2, a held-out margin selection for W3, end-to-end latency on matched hardware for W4, and a comparison with recent zero-shot methods for W5.
W1. Novelty and distinction from candidate-based VLM grounding
We would like to clarify that the contribution of RaV lies in the design of the whole verification pipeline rather than in a single module. Using a VLM to verify a supervised grounder raises problems that do not appear when a VLM is used as a grounder on its own. The grounder outputs a single box, each hypothesis needs a view that also shows its context, and the VLM should override the grounder only on clear evidence. RaV addresses these with orientation-based candidate generation, geometric viewpoint selection (VCF), and per-candidate verification under a conservative margin. Since no component is trained and neither the grounder nor the VLM is modified, the same pipeline can be applied to future supervised grounders and VLMs as they appear.
Existing candidate-based VLM grounders such as IVSGround [1], SeeGround [2] and SPAZER [3] are zero-shot pipelines that replace the supervised grounder. Their candidates come from a segmenter over all objects in the scene, their views are a rendered overview or captured frames, which IVSGround selects with a VLM view selector fine-tuned on the ScanRefer training split, and the VLM compares the candidates against each other. In contrast, RaV keeps the supervised grounder and its box precision. It synthesizes a dedicated viewpoint for each hypothesis from scene geometry and verifies each hypothesis independently without extra training. These differences matter in practice. In ScanNet, a candidate box is visible in only 33% of its nearest captured frames, and 32% of the candidates are not visible in any of the four nearest frames. By placing the camera for each candidate, RaV makes the candidate visible in a single rendered view for 89.2% of the candidates, compared with 58.7% for a fixed canonical view. In addition, as shown in Tab. A8 of the main paper, independent verification gives +3.41 [email protected] on the same pool, compared with +1.78 for pairwise comparison and +1.39 for selecting among all candidates at once. We will add this comparison to the Related Work section.
W2. Visibility of surrounding objects and attribution of the context gain
We thank the reviewer for the constructive suggestion. To isolate the contribution of the relational context, we ran the controlled experiment the reviewer proposed. The TSP3D candidate pool, the Qwen2.5-VL-7B verifier, the prompt and the scoring rule are all fixed, and only the image given to the VLM changes. The candidate-only crop keeps only the marked candidate with a small margin, the context-masked view keeps the same framing but blacks out all pixels outside the marker, and the full scene is our default input. The results are shown in Tab. R1.
Table R1. Verification with the same VLM under different visual evidence (TSP3D, ScanRefer, 7B verifier).
The candidate-only crop already improves the grounder by about 2.2 points. Compared with the context-masked view, the full scene provides an additional +1.08 / +1.28 points. The crop and the masked view perform almost the same. The additional gain therefore comes from the visible context itself and not from where the candidate appears in the image, which supports the context-aware verification of RaV. We will present Tab. 5 together with this decomposition.
To examine whether the objects mentioned in the expression are visible, we matched the nouns of each expression to ScanNet labels, excluding the target and structural classes, and checked their visibility in the rendered views. Of the ScanRefer validation queries, 8,197 (86%) mention such an anchor object in their context. Over all candidates, at least one anchor is visible in the VCF view for 82.1% of them, compared with 65.2% for a fixed canonical view. Although VCF has no explicit term for anchors, its free-space and target-visibility constraints move the camera to open positions that also include the neighboring objects. This indicates that VCF already provides the visibility of surrounding objects in most cases, which is the mechanism through which RaV benefits from context. We will add this analysis to the appendix and mention an explicit anchor-visibility term as a natural extension.