acceptodds
Under review as a conference paper at ICLR 2027

Render-and-Verify for Training-Free Hypothesis Verification in 3D Visual Grounding

Abstract

Supervised 3D visual grounding (3DVG) has achieved substantial progress, yet its performance remains highly uneven across scene complexity. Modern grounders reliably localize targets when the referred object is unique, but their accuracy drops sharply when multiple same-class distractors are present. We find that many errors involving same-class distractors cannot be resolved by object appearance or grounder confidence alone. Resolving such ambiguities often requires observing the candidate together with surrounding objects, and the usefulness of these relational cues can depend on the viewpoint. Based on this observation, we propose RaV (Render-and-Verify) framework, a training-free plug-in that improves a frozen 3D grounder by verifying its candidate hypotheses with a frozen 2D vision-language model (VLM). Rather than replacing the 3D grounder, RaV preserves its geometric localization capability and utilizes the VLM only for candidate verification. Specifically, RaV first constructs diverse hypotheses from multiple transformed views. For each candidate, Visibility-constrained Canonical Framing (VCF) selects a candidate-conditioned viewpoint that maintains target visibility while preserving the surrounding context required for relational verification. Each rendering is cleaned by a frozen diffusion corrector, then verified by the VLM and re-ranked accordingly. We evaluate RaV on ScanRefer with TSP3D, PV-Ground, and EG-3DVG, and on Nr3D and Sr3D with TSP3D and PV-Ground. RaV improves the Overall accuracy of every evaluated configuration on ScanRefer and achieves state-of-the-art results on all three benchmarks under the detection protocol. The source code will be made publicly available upon acceptance.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
3D Visual Grounding, Vision-Language Models, Training-Free Adaptation

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_kHWnOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes Render-and-Verify (RaV), a training-free framework that improves frozen supervised 3D visual grounders through VLM-based candidate verification. RaV constructs diverse candidate hypotheses using rotated scenes, selects candidate-specific viewpoints with visibility constraints, and refines rendered images using a frozen diffusion corrector. A frozen VLM then verifies each candidate against the referring expression and re-ranks the predictions. Experiments on ScanRefer, Nr3D, and Sr3D demonstrate improvements across multiple grounding backbones, with reported state-of-the-art results under the detection protocol.

Critical strengths

(1) The paper addresses the difficulty of distinguishing same-class distractors in 3D visual grounding. Using a VLM to verify hypotheses from an existing supervised grounder is a sensible way to combine geometric localization with complementary visual reasoning.

(2) RaV can be applied to pretrained grounders without modifying their parameters or training objectives. Candidate-specific viewpoint selection provides useful visual evidence for verification, and the ablations demonstrate its contribution to grounding performance.

(3) The paper evaluates multiple grounding backbones and VLM families across ScanRefer, Nr3D, and Sr3D. Component ablations, verification-format comparisons, prediction-flip analyses, and computational-cost measurements provide a reasonably comprehensive assessment of the framework.

Critical weaknesses

(1) RaV combines orientation-based candidate generation, visibility-based viewpoint selection, an existing diffusion corrector, and VLM-based re-ranking. Although the integration is useful and supported by ablations, the novelty beyond applying pretrained components to candidate verification remains modest. The distinction from existing candidate-based VLM grounding approaches needs to be established more clearly.

(2) VCF explicitly optimizes candidate visibility, but does not assess whether the surrounding objects needed to verify the referring expression are visible. Moreover, the comparisons with candidate-local selectors change both the model and evidence representation, making it difficult to attribute the gains specifically to relational context. Controlled experiments using the same VLM with full-scene, candidate-only, and context-masked inputs would strengthen this claim.

(3) Improvements on Sr3D are small (+0.7 for TSP3D and +0.2 for PV-Ground), and the appendix shows that ungated verification substantially degrades PV-Ground on this benchmark. This suggests that verification is not uniformly reliable and that conservative re-ranking is important to the reported results. Although margin sensitivity is provided, the selection procedure for the dataset-specific thresholds should be clarified, including whether evaluation labels were used.

(4) RaV adds three grounder forward passes, candidate rendering, diffusion-based correction, and multiple VLM verifications to a supervised grounding pipeline. Although the appendix reports component costs, a direct comparison of end-to-end latency and throughput against the base grounder on matched hardware is missing. Such a comparison is needed to assess whether the accuracy gains justify the additional computation, particularly for applications requiring fast inference.

(5) Lack of comparison with recent zero-shot 3D grounding methods [1]-[4]

[1] Spazer: Spatial-semantic progressive reasoning agent for zero-shot 3d visual grounding, 2025 NIPS

[2] Think, Act, Build: An Agentic Framework with Vision Language Models for Zero-Shot 3D Visual Grounding, 2026 EMNLP

[3] Seeground: See and ground for zero-shot open-vocabulary 3d visual grounding, 2025 CVPR

[4] Seqvlm: Proposal-guided multi-view sequences reasoning via vlm for zero-shot 3d visual grounding, 2025 ACMMM

Questions for authors

Please see the weakness

6 comments
AuthorsOpenReview ↗

Response to Reviewer kHWn (1/3) - Novelty and context ablation (W1, W2)

We sincerely thank the reviewer for recognizing that verifying a supervised grounder with a VLM is a sensible design and that RaV applies to pretrained grounders without modification. In brief, we add a same-VLM context ablation and an anchor-visibility analysis for W2, a held-out margin selection for W3, end-to-end latency on matched hardware for W4, and a comparison with recent zero-shot methods for W5.

W1. Novelty and distinction from candidate-based VLM grounding

We would like to clarify that the contribution of RaV lies in the design of the whole verification pipeline rather than in a single module. Using a VLM to verify a supervised grounder raises problems that do not appear when a VLM is used as a grounder on its own. The grounder outputs a single box, each hypothesis needs a view that also shows its context, and the VLM should override the grounder only on clear evidence. RaV addresses these with orientation-based candidate generation, geometric viewpoint selection (VCF), and per-candidate verification under a conservative margin. Since no component is trained and neither the grounder nor the VLM is modified, the same pipeline can be applied to future supervised grounders and VLMs as they appear.

Existing candidate-based VLM grounders such as IVSGround [1], SeeGround [2] and SPAZER [3] are zero-shot pipelines that replace the supervised grounder. Their candidates come from a segmenter over all objects in the scene, their views are a rendered overview or captured frames, which IVSGround selects with a VLM view selector fine-tuned on the ScanRefer training split, and the VLM compares the candidates against each other. In contrast, RaV keeps the supervised grounder and its box precision. It synthesizes a dedicated viewpoint for each hypothesis from scene geometry and verifies each hypothesis independently without extra training. These differences matter in practice. In ScanNet, a candidate box is visible in only 33% of its nearest captured frames, and 32% of the candidates are not visible in any of the four nearest frames. By placing the camera for each candidate, RaV makes the candidate visible in a single rendered view for 89.2% of the candidates, compared with 58.7% for a fixed canonical view. In addition, as shown in Tab. A8 of the main paper, independent verification gives +3.41 [email protected] on the same pool, compared with +1.78 for pairwise comparison and +1.39 for selecting among all candidates at once. We will add this comparison to the Related Work section.

W2. Visibility of surrounding objects and attribution of the context gain

We thank the reviewer for the constructive suggestion. To isolate the contribution of the relational context, we ran the controlled experiment the reviewer proposed. The TSP3D candidate pool, the Qwen2.5-VL-7B verifier, the prompt and the scoring rule are all fixed, and only the image given to the VLM changes. The candidate-only crop keeps only the marked candidate with a small margin, the context-masked view keeps the same framing but blacks out all pixels outside the marker, and the full scene is our default input. The results are shown in Tab. R1.

Table R1. Verification with the same VLM under different visual evidence (TSP3D, ScanRefer, 7B verifier).

InputOverall @0.25 / @0.5Multiple @0.25 / @0.5Δ Overall
Candidate-only crop58.67 / 48.4753.52 / 44.30+2.22 / +1.76
Context-masked58.78 / 48.1453.74 / 44.06+2.33 / +1.43
Full scene (ours)59.86 / 49.4254.95 / 45.30+3.41 / +2.71

The candidate-only crop already improves the grounder by about 2.2 points. Compared with the context-masked view, the full scene provides an additional +1.08 / +1.28 points. The crop and the masked view perform almost the same. The additional gain therefore comes from the visible context itself and not from where the candidate appears in the image, which supports the context-aware verification of RaV. We will present Tab. 5 together with this decomposition.

To examine whether the objects mentioned in the expression are visible, we matched the nouns of each expression to ScanNet labels, excluding the target and structural classes, and checked their visibility in the rendered views. Of the ScanRefer validation queries, 8,197 (86%) mention such an anchor object in their context. Over all candidates, at least one anchor is visible in the VCF view for 82.1% of them, compared with 65.2% for a fixed canonical view. Although VCF has no explicit term for anchors, its free-space and target-visibility constraints move the camera to open positions that also include the neighboring objects. This indicates that VCF already provides the visibility of surrounding objects in most cases, which is the mechanism through which RaV benefits from context. We will add this analysis to the appendix and mention an explicit anchor-visibility term as a natural extension.

AuthorsOpenReview ↗

Response to Reviewer kHWn (2/3) - Margin selection and latency (W3, W4)

W3. Sr3D, ungated degradation, and the selection of τ

We agree that the gains on Sr3D are small, and we would like to explain the reason. Sr3D is difficult for VLM-based grounding in general, since its templated spatial expressions carry little appearance information, and the recent zero-shot VLM methods (e.g., SeeGround [2], SPAZER [3], SeqVLM [4], TAB [5], and VLM-Grounder [6]) report ScanRefer and Nr3D only. The verifier is also the least discriminative on Sr3D. The probability that it scores a correct candidate above an incorrect one is 58% on Sr3D, compared with 66% on ScanRefer. This is exactly where the conservative margin is designed to act. It turns the 3.8-point loss of the ungated rule into a small but safe gain, so that RaV never degrades the grounder. To the best of our knowledge, RaV is the first VLM-based method to report Sr3D. We also note that the gains on ScanRefer and Nr3D do not rely on the margin. Table A11 of the main paper shows that RaV without any margin improves EG-3DVG on ScanRefer from 57.07 to 61.57 and PV-Ground on Nr3D from 51.12 to 52.35. The verification itself is therefore reliable on these benchmarks.

Regarding the selection of τ, we apologize for the confusion caused by the brief description. τ is a single scalar per dataset chosen from the coarse grid of Tab. A11 on the validation split, which is the usual hyperparameter sweep in vision tasks. Because RaV is training-free, we did not design a learnable or scene-adaptive margin. To address the concern directly, we re-selected τ with a held-out protocol. In this setting, the validation scenes are split into two halves, τ is chosen on one half and evaluated on the other, and the results of both directions are averaged. The results are shown in Tab. R2.

Table R2. Held-out selection of the margin τ (Overall [email protected]).

Configurationτ from the grid sweep (paper)Held-out selectionBase
ScanRefer, EG-3DVG61.6761.6157.07
Nr3D, PV-Ground53.2753.1351.12
Sr3D, PV-Ground56.3956.3956.18

The accuracies change by at most 0.14 points, so the reported gains do not depend on the selection procedure. The final version will report the held-out numbers and the procedure.

W4. End-to-end latency and throughput on matched hardware

We thank the reviewer for this suggestion. Table R3 reports the end-to-end latency per query on one A6000 at batch size 1, using the amortized stage costs of Tab. A10 in the main paper with the early exit included.

Table R3. End-to-end latency per query on one A6000 (ScanRefer, seconds).

PipelineGrounderPool (3 passes)Render + correctVerifyTotal[email protected] / @0.5
PV-Ground two-stage0.23–––0.2359.87 / 47.56
+ RaV 7B, no corrector0.230.690.282.23.460.85 / 47.91
+ RaV 7B0.230.692.12.25.261.65 / 48.47
+ RaV 72B0.230.692.117.020.062.20 / 49.05
EG-3DVG two-stage0.27–––0.2758.50 / 52.40
+ RaV 7B0.270.822.12.25.461.24 / 54.33
+ RaV 72B0.270.822.117.020.261.87 / 54.93

RaV adds a few seconds per query, and the full validation split runs in 85 minutes on eight A6000s with the 7B verifier. This cost should be weighed against training. Each supervised grounder is trained separately for ScanRefer, Nr3D and Sr3D, and a single run of TSP3D on ScanRefer with the authors' recipe takes 128 GPU-hours on our A6000s. RaV needs no training on any dataset and improves these grounders by 1.6 to 3.4 points with the 7B verifier at about 11 GPU-hours of inference for the whole validation split. Among methods that use a VLM at inference time, RaV is also the fastest. The zero-shot pipelines in Tab. R4 report 23.5 to 50.3 seconds per query, whereas RaV takes about 5 seconds with the 7B verifier because it issues one small-VLM call per candidate on a single rendered view. We will add Tab. R3 to the main paper.

AuthorsOpenReview ↗

Response to Reviewer kHWn (3/3) - Comparison with zero-shot methods (W5)

W5. Comparison with recent zero-shot methods

We thank the reviewer for pointing us to these works. Table R4 adds the four methods together with IVSGround [1]. Since their evaluation protocols differ in split and annotation, we list the protocol of each method in the table. All numbers are taken from the respective papers.

Table R4. Comparison with VLM-based 3D visual grounding methods on the ScanRefer dataset (Overall [email protected] / [email protected]).

MethodVLMSplit / annotationTime per query[email protected] / @0.5
Full validation split
SeeGround (CVPR'25) [2]Qwen2-VL-72Bfull val–44.1 / 39.4
IVSGround (ECCV'26) [1]Qwen3-VL-32Bfull val–56.2 / 49.7
RaV 7B + TSP3D (ours)Qwen2.5-VL-7Bfull val4.6 s59.86 / 49.42
RaV 72B + PV-Ground two-stage (ours)Qwen2.5-VL-72Bfull val20.0 s62.20 / 49.05
RaV 72B + EG-3DVG two-stage (ours)Qwen2.5-VL-72Bfull val20.2 s61.87 / 54.93
250-sample subset of VLM-Grounder
VLM-Grounder (CoRL'24) [6]GPT-4o250-sample subset50.3 s51.6 / 32.8
SeqVLM (ACM MM'25) [4]Doubao-1.5-vision-pro250-sample subset–55.6 / 49.6
SPAZER (NeurIPS'25) [3]GPT-4o250-sample subset23.5 s57.2 / 48.8
TAB (EMNLP'26) [5]Qwen3-VL-32B250-sample subset, author-refined annotations–71.2 / 46.4
RaV 72B + TSP3D (ours)Qwen2.5-VL-72B250-sample subset19.4 s61.5 / 47.4
RaV 72B + EG-3DVG two-stage (ours)Qwen2.5-VL-72B250-sample subset20.2 s59.0 / 51.0

RaV times are per query on a single A6000, including the grounder. The 72B verifier does not fit on one A6000 and is sharded over four GPUs, so for the 72B rows we report GPU-seconds summed over the four GPUs, which is a conservative estimate (the wall-clock time per query is about 7 seconds).

On the full validation split with the original annotations, the strongest zero-shot result is IVSGround at 56.2 / 49.7, and RaV is 6.0 points higher in [email protected] with PV-Ground two-stage and 5.2 points higher in [email protected] with EG-3DVG two-stage. For a like-for-like comparison with the methods evaluated on the 250-sample subset of VLM-Grounder, we re-scored RaV on the same queries. RaV reaches 61.5 [email protected] with TSP3D and 51.0 [email protected] with EG-3DVG two-stage, which are both higher than those of SPAZER and SeqVLM.

TAB [5] reports a higher [email protected] of 71.2, but it is evaluated on a different version of the benchmark. The authors manually reviewed the 250-sample subset and rewrote its annotations. In particular, ambiguous references that admit several valid candidates were resolved by adding exclusive contextual anchors, and expressions with category or spatial errors were corrected. Removing the ambiguity among same-class objects makes the task considerably easier, and this ambiguity is precisely the failure mode that RaV is designed to resolve. TAB does not report results on the original annotations or an ablation of the refinement, so its numbers cannot be compared with results on the original benchmark. The reported times come from different hardware and should be read as order-of-magnitude comparisons. We will add Tab. R4 and this discussion to Section 4.2.

References

[1] Chiang et al., Where to look matters: Learning influential views for VLM-based 3D visual grounding. ECCV 2026.

[2] Li et al., SeeGround: See and ground for zero-shot open-vocabulary 3D visual grounding. CVPR 2025.

[3] Jin et al., SPAZER: Spatial-semantic progressive reasoning agent for zero-shot 3D visual grounding. NeurIPS 2025.

[4] Lin et al., SeqVLM: Proposal-guided multi-view sequences reasoning via VLM for zero-shot 3D visual grounding. ACM MM 2025.

[5] Wang et al., Think, act, build: An agentic framework with vision language models for zero-shot 3D visual grounding. EMNLP 2026.

[6] Xu et al., VLM-Grounder: A VLM agent for zero-shot 3D visual grounding. CoRL 2024.

Reviewer_gD9AOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary
  • Targets the multiple-instance (same-class distractor) problem in 3D visual grounding.

  • Proposes a training-free framework. A frozen 3D grounder generates candidate hypotheses, and a visibility-constrained algorithm picks a camera view for each candidate. The renders are cleaned with Difix3D+ to remove artifacts, and a frozen VLM verifies each candidate to re-rank them.

  • Provides comprehensive evaluation and ablation studies across grounders, VLMs, and hyperparameters, demonstrating the effectiveness of each component.

Critical strengths
  • The paper is well written, and the figures clearly illustrate the pipeline.

  • The idea is simple and practical. RaV plugs into existing 3D grounders without additional training and consistently improves all evaluated configurations on ScanRefer.

  • The experiments are comprehensive, covering three grounders, multiple VLM families and scales, and three benchmarks, with detailed ablation and sensitivity analyses.

Critical weaknesses

[W1] The technical novelty is limited. The pipeline mostly combines existing components and is close to IVSGround which also use VLM and verification, but the paper gives limited comparison with it.

[W2] τ is tuned separately for each dataset without a held-out selection procedure. This raises concerns about generalization, especially on Sr3D, where the gain depends heavily on τ.

[W3] Difix3D+ brings only a small gain (about +0.5 [email protected]) while adding latency comparable to the 7B verification step.

Questions for authors

[Q1] Have you tried lighter artifact correctors, or other restoration models, in place of Difix3D+? Table 4 shows it adds only about +0.5 [email protected], while its cost (about 0.8 GPU-s per image) is comparable to Qwen2.5-VL-7B verification.

[Q2] Could you add inference time or FPS for the baselines and other methods in Table 1? That would make the accuracy–efficiency trade-off of RaV clear.

6 comments
AuthorsOpenReview ↗

Response to Reviewer gD9A (1/3) - Novelty and held-out margin selection (W1, W2)

We sincerely thank the reviewer for the encouraging comments on the clear writing and figures, the simple and practical design of RaV, and the comprehensive experiments. In brief, we add (i) a held-out margin-selection protocol for W2, (ii) a comparison of lighter and alternative restoration models for W3/Q1, and (iii) an accuracy and inference-speed table covering the Tab. 1 of our main manuscript and the VLM-based pipelines for Q2.

W1. Novelty and closeness to IVSGround

We would like to clarify that RaV is not a single module but a complete pipeline for verifying the hypotheses of a frozen supervised grounder with a frozen VLM. It consists of orientation-based candidate generation, geometric viewpoint selection (VCF) that keeps both the candidate and its surroundings visible, and per-candidate verification with a conservative margin that lets the VLM override the grounder only when the evidence is clear. As shown in Tab. 4 of the main paper, each component is necessary, and none of them requires training. Since the pipeline modifies neither the grounder nor the VLM, it can be applied as is to any existing or future supervised grounder, which makes RaV a practical plug-in for 3D visual grounding.

IVSGround [1] is a zero-shot pipeline that replaces the supervised grounder. It selects captured frames for each candidate with a view selector obtained by fine-tuning a VLM on the ScanRefer training split, and compares the candidates with a frozen VLM in a pairwise tournament. In contrast, RaV keeps the supervised grounder, synthesizes the viewpoint of each candidate from geometric constraints, and verifies each candidate independently, without training any component or fine-tuning any VLM. These choices are reflected in the measurements. In ScanNet, a candidate box is visible in only 33% of its nearest captured frames, and 32% of the candidates are not visible in any of the four nearest frames. By placing the camera for each candidate, RaV makes the candidate visible in a single rendered view for 89.2% of the candidates, compared with 58.7% for a fixed canonical view. In addition, as shown in Tab. A8 of the main paper, independent Yes/No verification is 1.63 points higher in [email protected] than pairwise comparison. On the full ScanRefer validation split, IVSGround reports 56.2 / 49.7, while RaV reaches 62.20 [email protected] with PV-Ground two-stage and 54.93 [email protected] with EG-3DVG two-stage. We will add this comparison with IVSGround to the Related Work section.

W2. Per-dataset τ without a held-out procedure

We apologize for not describing the selection clearly. τ is a single scalar per dataset chosen from the coarse grid of Tab. A11 on the validation split, as is common for hyperparameters in vision tasks. Because RaV is training-free, we did not design a learnable or scene-adaptive margin. To address the concern, we re-selected τ with a held-out protocol. In this setting, the validation scenes are split into two halves, τ is chosen on one half and evaluated on the other, and the results of both directions are averaged. The results are shown in Tab. R1.

Table R1. Held-out selection of the margin τ (Overall [email protected]).

Configurationτ from the grid sweep (paper)Held-out selectionBase
ScanRefer, EG-3DVG61.6761.6157.07
Nr3D, PV-Ground53.2753.1351.12
Sr3D, PV-Ground56.3956.3956.18

The accuracies change by at most 0.14 points, so the reported gains do not depend on the selection procedure. The final version will report the held-out values and the procedure.

Sr3D is difficult for VLM-based grounding in general, since its templated spatial expressions carry little appearance information, and the recent zero-shot VLM methods (e.g., SeeGround [2], SPAZER [3], SeqVLM [4], TAB [5], and VLM-Grounder [6]) report ScanRefer and Nr3D only. To the best of our knowledge, RaV is the first VLM-based method to report Sr3D, and the conservative margin allows it to do so without degrading the grounder.

AuthorsOpenReview ↗

Response to Reviewer gD9A (2/3) - Artifact correctors (W3, Q1)

W3 and Q1. Difix3D+ and lighter other restoration models

We tried six lighter restoration models published from 2021 to 2026 while keeping everything else fixed, including the VCF viewpoints, the marker drawn after correction, the 7B verifier and the scoring rule. The results are shown in Tab. R2.

Table R2. Comparison of artifact correctors (TSP3D, ScanRefer, 7B verifier).

CorrectorParamsCost per image (A6000)Overall @0.25 / @0.5Δ vs. no corrector
None–059.29 / 48.67–
Real-ESRGAN (ICCVW'21) [7]16.7M1.05 s59.04 / 48.28−0.25 / −0.39
NAFNet (ECCV'22) [8]116M0.18 s58.97 / 48.43−0.32 / −0.24
PromptIR (NeurIPS'23) [9]36M0.31 s58.84 / 48.08−0.45 / −0.59
InstructIR (ECCV'24) [10]15.8M0.11 s59.21 / 48.42−0.08 / −0.25
MoCE-IR (CVPR'25) [11]25.4M0.72 s59.13 / 48.57−0.16 / −0.10
MIRAGE (ICLR'26) [12]9.7M0.31 s59.37 / 48.74+0.08 / +0.07
Difix3D+ (CVPR'25)≈1B0.80 s59.86 / 49.42+0.57 / +0.75

Lighter generic restoration models yield no meaningful improvement. Five of the six models reduce the accuracy, and the most recent model MIRAGE provides only a marginal gain of +0.08 / +0.07. Among these models, the drop in [email protected] also tends to grow with model size. In RaV, the verifier judges each candidate solely from what is visible in the rendered view, so a corrector is beneficial only when it removes the rendering artifacts. Generic restoration models are trained for photographic degradations and hallucinate texture into the holes and seams of mesh renders, which confuses the verifier. In contrast, Difix3D+ is trained to remove the artifacts of rendered 3D reconstructions, and it is the only tested corrector that achieves a clear improvement of +0.57 / +0.75. This indicates that the quality of the rendered evidence is itself a factor in VLM-based verification, and that a corrector matched to rendering artifacts is needed to improve it. To the best of our knowledge, RaV is the first 3D visual grounding method to identify this factor and to correct rendered views before VLM verification. As shown in Tab. R3, the contribution of the corrector also depends on the grounder. On PV-Ground two-stage, Difix3D+ raises the gain of RaV with the 7B verifier from +0.98 to +1.78 [email protected], which is 45% of the total gain. Regarding the latency, the corrector is a separable module. Without it, RaV still improves TSP3D by +2.84 / +1.96, so it can be omitted when latency is the priority. We will add Tables R2 and R3 and this discussion to the appendix.

Table R3. Effect of Difix3D+ across grounders on the ScanRefer dataset (Overall [email protected] / [email protected], 7B verifier).

GrounderBaseRaV 7B, no correctorRaV 7B, Difix3D+Gain from Difix3D+
TSP3D56.45 / 46.7159.29 / 48.67 (+2.84 / +1.96)59.86 / 49.42 (+3.41 / +2.71)+0.57 / +0.75
PV-Ground two-stage59.87 / 47.5660.85 / 47.91 (+0.98 / +0.35)61.65 / 48.47 (+1.78 / +0.91)+0.80 / +0.56
AuthorsOpenReview ↗

Response to Reviewer gD9A (3/3) - Inference speed (Q2)

Q2. Inference time or FPS for the baselines and other methods in Table 1

We measured the inference time of the baselines and compiled it together with the reported times of the other methods in Tab. R4, which places the supervised grounders, the VLM-based pipelines and RaV in one table. The FPS of the earlier supervised methods are those reported by TSP3D from their reproduction. Our own measurements use one A6000 at batch size 1 on the full validation split. VLM-pipeline times are as reported by each paper.

Table R4. Accuracy and inference speed on the ScanRefer dataset.

MethodTraining[email protected] / @0.5 (full val)[email protected] / @0.5 (250-sample subset)Time per queryFPS
Supervised grounders
ScanRefer (ECCV'20)supervised41.19 / 27.40–0.15 s6.72†
3D-SPS (CVPR'22)supervised47.65 / 36.43–0.19 s5.38†
BUTD-DETR (ECCV'22)supervised50.42 / 38.60–0.30 s3.33†
EDA (CVPR'23)supervised54.59 / 42.26–0.30 s3.34†
MCLN (ECCV'24)supervised57.17 / 45.53–0.32 s3.17†
TSP3D (CVPR'25)supervised, 128 GPU-h56.45 / 46.71–0.08 s13.2
PV-Ground two-stage (CVPR'26)supervised59.87 / 47.56–0.23 s4.3
EG-3DVG two-stage (CVPR'26)supervised58.50 / 52.40–0.27 s3.7
VLM-based zero-shot methods
VLM-Grounder (CoRL'24) [6]none–51.6 / 32.850.3 s0.02
SPAZER (NeurIPS'25) [3]none–57.2 / 48.823.5 s0.04
Ours
RaV 7B + EG-3DVG two-stage (ours)none61.24 / 54.3357.4 / 51.45.4 s0.19
RaV 72B + EG-3DVG two-stage (ours)none61.87 / 54.9359.0 / 51.020.2 s0.05

† reported in TSP3D, measured on one RTX 3090. Other supervised rows and all RaV rows are measured by us on A6000s and reported in GPU-seconds, which equal the wall-clock time for every row that runs on one GPU. The 72B verifier does not fit on one A6000 and is sharded over four GPUs, so the RaV 72B row counts all four GPUs and its wall-clock time is about 7 seconds. VLM-pipeline times and accuracies are from the respective papers, which report results on the 250-sample subset of VLM-Grounder.

RaV is designed to improve the accuracy of an existing grounder without training, not to reduce its inference time. Each supervised row requires a full training run per dataset. It takes 128 GPU-hours to train TSP3D on ScanRefer alone, and Nr3D and Sr3D require separate runs, whereas RaV adds 1.6 to 3.4 points on top of these grounders with the 7B verifier, no training and about 11 GPU-hours of rendering and verification for the whole validation split. On the 250-sample subset, RaV 7B with EG-3DVG two-stage outperforms VLM-Grounder and SPAZER in both [email protected] and [email protected] and runs 4 to 9 times faster, since it issues one small-VLM call per candidate on a single rendered view. RaV 72B with EG-3DVG two-stage also outperforms both methods at a time comparable to SPAZER. We will add Tab. R4 to the paper.

References

[1] Chiang et al., Where to look matters: Learning influential views for VLM-based 3D visual grounding. ECCV 2026.

[2] Li et al., SeeGround: See and ground for zero-shot open-vocabulary 3D visual grounding. CVPR 2025.

[3] Jin et al., SPAZER: Spatial-semantic progressive reasoning agent for zero-shot 3D visual grounding. NeurIPS 2025.

[4] Lin et al., SeqVLM: Proposal-guided multi-view sequences reasoning via VLM for zero-shot 3D visual grounding. ACM MM 2025.

[5] Wang et al., Think, act, build: An agentic framework with vision language models for zero-shot 3D visual grounding. EMNLP 2026.

[6] Xu et al., VLM-Grounder: A VLM agent for zero-shot 3D visual grounding. CoRL 2024.

[7] Wang et al., Real-ESRGAN: Training real-world blind super-resolution with pure synthetic data. ICCVW 2021.

[8] Chen et al., Simple baselines for image restoration. ECCV 2022.

[9] Potlapalli et al., PromptIR: Prompting for all-in-one blind image restoration. NeurIPS 2023.

[10] Conde et al., InstructIR: High-quality image restoration following human instructions. ECCV 2024.

[11] Zamfir et al., Complexity experts are task-discriminative learners for any image restoration. CVPR 2025.

[12] Ren et al., Efficient degradation-agnostic image restoration via channel-wise functional decomposition and manifold regularization. ICLR 2026.

Reviewer_NDkcOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

In this paper, the authors propose a training-free Render-and-Verify framework---RaV (Render-and-Verify). A frozen supervised 3D grounder for candidate generation and a VLM for candidate verification. Experiments on ScanRefer, Nr3D, and Sr3D demonstrate its state-of-the-art results across multiple settings.

Critical strengths

(1) RaV (Render-and-Verify) framework, a training-free plug-in that improves a frozen 3D grounder by verifying its candidate hypotheses with a frozen 2D vision-language model (VLM) (2) Visibility-constrained Canonical Framing (VCF) is a principled and geometric contribution, which selects a candidate-conditioned viewpoint that maintains target visibility while preserving the surrounding context required for relational verification

Critical weaknesses

(1)The "training-free plug-in" value proposition is undermined by inference cost that may exceed compute-matched alternatives. whether RaV beats a frozen grounder, but whether a compute-matched alternative would achieve comparable or better gains — including on the pool coverage bottleneck that RaV cannot address。 (2)the paper's own analysis (Table A6, Table A7) reveals that the dominant residual failure mode is not verification quality but pool coverage: only 31.3% of initially wrong queries contain the correct object among the candidates at all. RaV recovers only 10–32% of the available oracle headroom, and no improvement in VLM verification strength.

Questions for authors

(1) The paper positions RaV as a "training-free plug-in" and compares against frozen baselines. Have you compared against a compute-matched fine-tuning baseline (2) IVSGround (Chiang et al., 2026) is described as studying "candidate-specific viewpoint selection and comparative VLM grounding," which overlaps with RaV's core contributions (VCF + VLM verification). Could you provide a concrete technical comparison (3) Table A7 shows the dominant residual failure is pool coverage, Is there an identifiable pattern to these broken predictions?

3 comments
AuthorsOpenReview ↗

Response to Reviewer NDkc (1/2) - Compute-matched baseline and pool coverage (W1, Q1, W2)

We sincerely thank the reviewer for recognizing RaV as a training-free plug-in that improves a frozen 3D grounder and VCF as a principled geometric contribution. In brief, we add (i) a direct training-cost comparison and the compute-matched fine-tuning baseline, (ii) a pool-expansion study showing that RaV converts better coverage into accuracy, and (iii) an analysis of the broken and unaddressable predictions.

W1 and Q1. Inference cost and a compute-matched fine-tuning baseline

We first compare the costs directly. The supervised grounders are trained separately for each dataset. One run of TSP3D on ScanRefer with the authors' recipe takes 128 GPU-hours on our A6000s (66 epochs, 1,753 s per epoch on four GPUs), and Nr3D and Sr3D each require another run. RaV needs no training, and its additional inference cost on the ScanRefer validation split is 9,508 queries × 4.3 GPU-s, about 11 GPU-hours with the 7B verifier, for a gain of 1.6 to 3.4 points in [email protected] across the evaluated grounders.

We then take the compute-matched comparison literally and spend a slightly larger budget of 12 GPU-hours on continuing to train the released checkpoints with the authors' recipes, keeping the global batch size, the position in the learning-rate schedule and the recipe's rotation augmentation, on the same GPUs. The results are shown in Tab. R1.

Table R1. Compute-matched fine-tuning versus RaV on the ScanRefer dataset (Overall [email protected] / [email protected], 7B verifier).

GrounderReleased checkpointReleased checkpoint + fine-tuningReleased checkpoint + RaV 7B
TSP3D56.45 / 46.7156.53 / 46.6259.86 / 49.42 (+3.41 / +2.71)

The released checkpoints are already converged, and under the original architecture and training recipe, continued supervised training with the same additional compute does not improve the accuracy. It does not address the pool coverage bottleneck either, since the oracle accuracy of the 4-yaw candidate pool of the fine-tuned checkpoint increases by only 0.63 points.

W2. The pool-coverage bottleneck

We would like to clarify the numbers in Tables A6 and A7. The 10–32% in Tab. A6 is computed against the oracle over the whole validation split, which also counts the queries whose correct box is not in the pool and therefore cannot be fixed by any re-ranking method. When we restrict the count to the queries that contain the correct box, RaV recovers 61% of them (782 of 1,289 with the 72B verifier), and it fixes about two predictions for every one it breaks (782 vs. 377). The verifier also benefits from a stronger VLM on the same pool, since moving from 7B to 72B increases the fixed predictions from 732 to 782 and reduces the broken ones from 422 to 377.

Regarding the coverage of the pool, we examined whether RaV can take advantage of a better pool. Keeping the grounder (EG-3DVG two-stage), the VLM, the viewpoints and the scoring rule fixed, we changed only the candidate pool. The results are shown in Tab. R2.

Table R2. Effect of the candidate pool on the gain of RaV (EG-3DVG two-stage, ScanRefer, 7B verifier).

Candidate poolOracle @0.25HeadroomOverall @0.25 / @0.5Δ Overall
Top-1 (base)––58.50 / 52.40–
Native top-467.148.6459.99 / 53.36+1.49 / +0.96
Native top-869.0710.5760.12 / 53.48+1.62 / +1.08
Native top-4 + 3 yaw rotations (ours)75.3816.8861.24 / 54.33+2.74 / +1.93

The gain of RaV increases with the headroom of the pool. Our yaw-rotated candidate generation is the component that enlarges this headroom from 8.64 to 16.88. It is far more effective than taking more proposals from the grounder, since the additional proposals largely overlap with the top-4 boxes and add less new coverage. Going from top-4 to top-8 raises the oracle accuracy by only 1.93 points, whereas adding three yaw rotations to top-4 raises it by 8.24 points. This also means that a future grounder with higher recall would directly increase the gain of RaV without any retraining. We will add Tab. R2 and this discussion to Appendix A.1.5.

AuthorsOpenReview ↗

Response to Reviewer NDkc (2/2) - Comparison with IVSGround and failure analysis (Q2, Q3)

Q2. Technical comparison with IVSGround

IVSGround [1] is a zero-shot pipeline that replaces the supervised grounder. It obtains candidates by screening all objects in the scene. For each candidate, it selects captured frames with a view selector that is fine-tuned from Qwen3-VL-8B with LoRA on the ScanRefer training split. The labels for this fine-tuning are derived from the feedback of a reasoning VLM. The candidates are then compared with a frozen Qwen3-VL-32B in a pairwise tournament. In contrast, RaV requires no training at all, including no fine-tuning of any VLM. Both the grounder and the VLM are used as released, and the viewpoint of each candidate is synthesized from geometric constraints on an occupancy grid. Since RaV verifies the hypotheses of the supervised grounder, it also keeps the box precision of the grounder. RaV is therefore a practical framework that can be applied to any existing or future supervised grounder.

These design choices are reflected in the measurements. In ScanNet, a candidate box is visible in only 33% of its nearest captured frames, and 32% of the candidates are not visible in any of the four nearest frames. By placing the camera for each candidate, RaV makes the candidate visible in a single rendered view for 89.2% of the candidates, compared with 58.7% for a fixed canonical view. In addition, as shown in Tab. A8 of the main paper, verifying each candidate independently with Yes/No logits is 1.63 points higher in [email protected] than pairwise comparison, which is the closest to the tournament of IVSGround. On the full ScanRefer validation split, IVSGround reports 56.2 / 49.7, while RaV reaches 62.20 [email protected] with PV-Ground two-stage and 54.93 [email protected] with EG-3DVG two-stage. We will include this comparison in the Related Work section.

Q3. Patterns in broken and unaddressable predictions

We thank the reviewer for this insightful question. We analyzed the residual failures of TSP3D with the 72B verifier on ScanRefer. Most of them are queries whose correct box is not proposed by the grounder at all. These are failures of proposal recall rather than of verification. Whenever the correct box is proposed, RaV resolves the ambiguity among same-class candidates, which is why its gain is concentrated on the Multiple subset (51.04 to 56.03 in [email protected] for TSP3D). The remaining broken predictions are cases in which two candidates of the same class receive nearly the same verification score, and these are the cases controlled by the conservative margin, as described in Appendix A.3.2. We will add this analysis to the discussion section of the final version.

References

[1] Chiang et al., Where to look matters: Learning influential views for VLM-based 3D visual grounding. ECCV 2026.

AuthorsOpenReview ↗

Brief summary of our response

We would like to thank the reviewer again for the constructive feedback on our submission. In our response, we provided a compute-matched baseline and an analysis of the candidate pool coverage for W1, Q1 and W2, together with a comparison with IVSGround and a failure analysis for Q2 and Q3.

Since the discussion period will close in a few days, we would be happy to address any remaining questions or concerns.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary of the revised manuscript

We sincerely thank all reviewers for their constructive comments. We have uploaded the revised manuscript with the following major changes, which are marked in blue.

  • [NDkc, gD9A, kHWn] Clarified the distinction of RaV from candidate-based VLM grounders including IVSGround (Sec. 2.2)
  • [NDkc, gD9A, kHWn] Added SeeGround and IVSGround to the ScanRefer comparison together with a discussion (Sec. 4.2, Tab. 1)
  • [gD9A, kHWn] Clarified the selection procedure of the margin τ and added the held-out selection (Sec. 4.3, Tab. 7)
  • [gD9A, kHWn] Added the end-to-end latency against the base grounder on matched hardware and the inference speed of the compared methods (Appendix A.5.3, Tab. A16, Tab. A17)
  • [NDkc] Added the training cost and the compute-matched fine-tuning baseline (Appendix A.5.1, Tab. A13)
  • [NDkc] Added the analysis of the candidate pool and its effect on the gain of RaV (Appendix A.5.6, Tab. A19)
  • [NDkc] Added the analysis of the residual failures and the broken predictions (Appendix A.5.7)
  • [gD9A] Added the comparison of artifact correctors and the effect of Difix3D+ across grounders (Appendix A.5.2, Tab. A14, Tab. A15)
  • [kHWn] Added the controlled experiment with the same VLM under candidate-only, context-masked and full-scene inputs (Sec. 4.3, Tab. 6)
  • [kHWn] Added the analysis of the visibility of candidates and anchor objects in the VCF views (Appendix A.5.5)
  • [kHWn] Added the comparison with SPAZER, SeqVLM, TAB and VLM-Grounder including the 250-sample subset (Appendix A.5.4, Tab. A18)

The numbering of the existing sections and tables is unchanged. The new tables in the main text are Tab. 6 and Tab. 7, and the new analyses are collected in Appendix A.5.

We would like to express our sincere gratitude to all reviewers for the time and effort devoted to our work. We have conducted all the additional experiments requested by the reviewers, and the results are now included in the revised manuscript. Since only a few hours remain in the discussion period, we would greatly appreciate it if the reviewers could let us know whether these results resolve the concerns raised. We remain available to clarify any remaining points.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.