acceptodds
Under review as a conference paper at ICLR 2027

GeomFunnel: Coordinate-Anchored Sparse Voxel Funneling for Metric-Aware 3D-VLMs

Abstract

Modern 3D vision-language models increasingly compress rich 3D scene evidence into compact representations for efficient reasoning, inevitably losing some information in the process. Our design addresses the risk that semantic scene understanding and relative spatial relationships can remain accessible after compression while the fine-grained metric geometry required for precise physical measurement becomes harder to recover. To address this gap, we introduce GeomFunnel, which constructs a coordinate-grounded sparse physical field that anchors multi-view scene evidence in a shared metric 3D space and preserves it as an explicit physical substrate. Specifically, its representation-decoupled reasoning uses compact tokens for semantic and relative spatial reasoning while retaining direct access to physical geometry for queries that require precise metric quantities. Within the metric pathway, language-guided physical grounding connects the referred scene elements to their relevant physical supports, allowing the requested quantities to be computed from explicit geometry rather than recovered from compressed tokens. GeomFunnel achieves a new state-of-the-art average score on VSI-Bench while remaining competitive on general 3D vision-language benchmarks.

Primary area:
foundation or frontier models, including LLMs
Keywords:
3D Vision-Language Models, 3D Spatial Reasoning, Metric Spatial Reasoning

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_n3jwOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
3
Soundness
3
Read review
Paper summary

The paper presents GEOMFUNNEL, an architecture that encodes 3D information of the scene into both compressed dense tokens and geometry-aware 3D anchor tokens (the Coordinate-Grounded Sparse Physical Field). By leveraging expert models and filtering with multi-view agreement, the “Physical Field” is built. Questions are divided into two categories, where one queries dense tokens and the other queries the Physical Field. Experiments show that they achieve SOTA performance on VSI-Bench and remarkable performance on ScanQA and SQA3D.

Critical strengths
  1. The research problem the paper is working on is pressing: how to preserve 3D geometric information while catering to the need of compression.
  2. Experimental results are strong, showing the effectiveness of the proposed method.
  3. The paper is clearly written, and effectively conveys its ideas.
Critical weaknesses
  1. Lack of novelty. I am sorry for this comment, but as a reviewer I have to point out that this paper is incremental. It maintains the mainstream “compact representations” while adding the “Coordinate-Grounded Sparse Physical Field”. The latter is supposed to be the main innovation, but in fact it conveys little message that is newer than PointNet++(2017) [1]. I recommend the authors read this classical paper.
  2. Weak technical contribution. For instance, the whole construction of the representation is trivial. The reconstruction and metrics come from VGGT and UniDepthV2, the metric alignment is a simple average, the sparsification is filtering. I failed to discover any surprising or exciting designs that qualify this paper to match the level of ICLR.
  3. Figures seriously need polishing. The Figs. 1 and 2 have low information density. Fig.2 even does not contain the performance of the proposed method, which confused me for 5 minutes. The figures are likely to be drawn or designed by AI, and may not convey central or catchy information to the reader.

[1] Qi, Charles Ruizhongtai, et al. "Pointnet++: Deep hierarchical feature learning on point sets in a metric space." Advances in neural information processing systems 30 (2017).

Questions for authors
  1. Please summarize what is your major contribution in this paper except for the improved performance.
  2. Please compare the core of your representation with PointNet++, and explain the novelty. I apologize for the words I use in the review that may sound frank. Generally it is a solid paper, and I recommend submitting it to some easier conferences.
4 comments
AuthorsOpenReview ↗

Novelty of GeomFunnel in comparison with PointNet++

The key difference is that GeomFunnel addresses how a 3D vision-language model can answer metric questions and proposes to measure the answers on explicit geometry, while PointNet++ addresses how to learn features from point sets and proposes a hierarchical point encoder.

  • GeomFunnel takes a question in natural language and returns a quantity in physical units, whereas PointNet++ takes only a point set and returns a semantic label.
  • In PointNet++, the 3D coordinates are an input that the encoder turns into features. In the physical field of GeomFunnel, the coordinates are never encoded for metric questions, and they are the data on which the answer is measured.
  • Using PointNet++ as the scene encoder of a 3D vision-language model would not solve the problem that GeomFunnel addresses. The language model would still have to decode the metric answer from the encoded features, in which fine-grained geometry is hard to recover.

We now cite and discuss PointNet++ in the revised Related Work.

AuthorsOpenReview ↗

Clarification of our technical contribution beyond VGGT-Ω and UniDepthV2

Our technical contribution is the workflow that turns the raw geometry from VGGT-Ω and UniDepthV2 into measured answers, because a direct combination of these two models has three problems.

  • UniDepthV2 predicts metric depth for each frame independently, so its depth maps do not agree with one another in 3D, and using them directly would break the multi-view consistency of VGGT-Ω. GeomFunnel therefore keeps the reconstruction of VGGT-Ω unchanged and takes only one global scale from UniDepthV2.
  • The combined reconstruction still contains locations that are seen in one view but contradicted by other views, and a measurement that includes them is unreliable. GeomFunnel therefore keeps only the locations that pass checks on confidence, visibility, depth consistency, and multi-view agreement.
  • The combined geometry does not tell which locations belong to the objects in a question, so it cannot answer the question. GeomFunnel therefore attaches visual features to the locations and finds the locations of each object that the question mentions.

Table 5 and Appendix G compare GeomFunnel with simpler variants that are built from the same components.

AuthorsOpenReview ↗

Further explanation of Figure 1 and Figure 2

Figure 1 shows that existing compact 3D-VLMs decode metric answers from compact tokens, while GeomFunnel measures them on an explicit physical field. The purpose of Figure 1 is to present the central idea of the paper, which is that metric questions should have direct access to explicit geometry and only qualitative questions should rely on compact tokens.

Figure 2 shows that when existing 3D-VLMs keep fewer visual tokens, metric information becomes much harder to recover, while qualitative reasoning drops much less. The purpose of Figure 2 is to give the evidence for the problem that motivates this central idea, which is why Figure 2 reports only existing models.

AuthorsOpenReview ↗

Summary of our core contributions

Our core contribution is a new 3D vision-language framework (GeomFunnel) that performs metric reasoning directly on explicit physical geometry, addressing a fundamental limitation of existing approaches that generate metric answers from compressed visual tokens, where fine-grained geometric information is difficult to preserve and recover.

Our first three contributions depend on one another, and only together do they remove this limitation.

  • The Coordinate-Grounded Sparse Physical Field takes metric geometry out of the compression path, but explicit geometry alone does not produce an answer.
  • Representation-Decoupled Reasoning therefore moves the metric answer from text generation to computation on this field, but a computation needs to know where to act.
  • Language-Guided Physical Grounding therefore links the words of a question to regions of the field, which existing models resolve only inside the tokens, and this completes the path from a question to a measurement.
Reviewer_VWDXOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper argues that compact 3D-VLM tokens preserve semantic and relative spatial reasoning but lose the fine metric geometry needed for measurement. It proposes GeomFunnel, which keeps an explicit physical field. It leverages a frozen multi-view geometry model and metric depth model to reconstruct a voxel feature filed. A small router sends qualitative queries to a compact Perceiver–LLM path and metric queries to a separate path that grounds the referred objects in the field and computes distances, sizes, and room areas with deterministic operators. The language model parses the query and predicts the physical supports, but the final number comes from geometry. Experiments demonstrate that GeomFunnel achieves best average score on VSI-Bench, with the largest gains on the metric categories.

Critical strengths
  1. The problem is well identified: compact tokens lose metric detail, and the paper separates semantic reasoning from metric measurement in a clean way.

  2. The decoupled design is supported by the routing study, which reaches high accuracy and nearly matches oracle routing, and by the component ablations.

  3. The VSI-Bench results are strong on the metric categories while the compact path remains competitive on general 3D question answering and captioning.

Critical weaknesses
  1. Base-model confound. The table does not list the language model used by each method or the performance of the base model itself, so it is unclear whether the gain comes from the method or from a stronger base. It would be better to add the base model for each method and a base-only result.

  2. Metric errors. For the metric tasks, only threshold accuracy is reported. Since the method computes metric values deterministically, it would be better to report the actual metric error (for example, MAE or relative error in meters) to show the effect of the method.

  3. Robustness to the geometry foundation models. All experiments use the same frozen VGGT-omega and UniDepthV2. It would be better to add a robustness study with different geometry foundation models to show that the gain does not depend on this specific pair.

Questions for authors

Please see above weaknesses.

4 comments
AuthorsOpenReview ↗

Clarification of the base models and the base-only result

We have added the base model of every method and a base-only result to Table 1. GeomFunnel and GeoThinker both use Qwen3-VL-8B as the base model, and GeomFunnel is still better. The gain therefore does not come from a stronger base model.

The base-only row for Qwen3-VL-8B uses the scores reported by Cai et al. (2026). In addition, GeomFunnel uses only the language model of Qwen3-VL-8B and does not use its native vision encoder.

AuthorsOpenReview ↗

Additional results on metric errors in physical units

We now report the mean absolute error of the three metric tasks in Appendix K, and GeomFunnel has the lowest error on all three tasks.

The table below is Table 12 of the revised paper. All models are run by us under the same evaluation protocol, and Always Compact is the same GeomFunnel checkpoint with all metric questions answered through compact tokens.

ModelAbs. Dist. (m) ↓Obj. Size (m) ↓Room Size (m²) ↓
VLM-3R1.320.369.8
SpatioLM-8B0.990.257.0
Cambrian-S-7B1.210.246.6
GS-Reasoner0.860.298.1
GeoThinker0.740.215.8
GeomFunnel (Always Compact)0.940.318.5
GeomFunnel0.580.184.7

The same GeomFunnel checkpoint has a clearly higher error on every task in the Always Compact setting. The lower error of GeomFunnel therefore comes from measuring metric answers on explicit geometry.

AuthorsOpenReview ↗

Additional robustness study on the geometry foundation models

We have added this study in Appendix L, where one geometry foundation model is replaced at a time. Replacing either model lowers the Metric Avg. and the VSI Avg. by at most 1.2 points, and GeomFunnel with VGGT stays above GeoThinker, which uses the same VGGT. The gain therefore does not depend on the specific pair of geometry foundation models.

The table below is Table 13 of the revised paper.

ReconstructionMetric scaleRetrainedAbs. Dist.Obj. SizeRoom SizeMetric Avg.VSI Avg.
VGGT-Ω (default)UniDepthV2–69.478.476.174.675.3
VGGTUniDepthV2Yes67.277.675.373.474.8
VGGT-ΩMetric3D v2No68.178.075.473.875.0
GeoThinker (VGGT), reported62.876.974.971.572.6

I thank the authors for their rebuttal, which has addressed my concerns. I would like to maintain my score of weak accept.

Reviewer_btmgOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

GeomFunnel separates compact language reasoning from direct metric computation. Frozen VGGT-Omega reconstructs geometry; UniDepthV2 anchors supply a metric scale. A sparse voxel field retains coordinates and fused features. A learned selector sends qualitative questions through a 256-token bridge, while metric questions use language-conditioned grounding and deterministic distance, extent, or floor-area operators. Training adapts the bridge, grounding decoder, selector, and Qwen LoRA parameters. The paper reports improvements on VSI-Bench and competitive results on ScanQA, SQA3D, and Scan2Cap.

Critical strengths
  1. The metric pathway makes a concrete, useful distinction between identifying an object and measuring its physical support. Equation 1 preserves multi-view consistency through one global scale; Equations 4-5 compute quantities from retained geometry. In particular, the distance operator measures closest voxel-support separation rather than substituting centroid distance. Appendix D explains the support refinement and robust extent construction. This is a coherent mechanism, rather than an additional learned answer regressor.
  2. The reported results support the combined design. Table 1 gives 75.3 average versus GeoThinker's 72.6, with a substantial absolute-distance gain of 69.4 versus 62.8. Table 5 improves Metric Avg. from 70.6 without decoupled reasoning to 74.6. Appendix G also provides a useful routing control: the learned selector reaches 75.3 overall versus 75.4 with oracle routing, while Always Compact reaches 73.6. These findings support explicit metric access without requiring oracle task labels at inference.
Critical weaknesses
  1. Figure 2 cannot currently support an empirical conclusion about compression-induced metric information loss. Appendix J explicitly states that the supplied values lack full-budget scores, per-example predictions, and a documented raw-score calculation; the MSR target/readout and token-retention protocol are also unspecified. The whiskers are predicted intervals, as the caption acknowledges. I appreciate this disclosure, but verifying marker-to-bar arithmetic does not establish the diagnostic's empirical basis. This limits the explanatory contribution of the motivation experiment; it does not demonstrate that Tables 1-5 are incorrect.
  2. The training and comparison protocol remains insufficiently specified to assess the source of the gains. Sections 3.5 and 4.1 and Appendices A/C explain optimization and support labels, but do not identify the language-model training examples, construction of structured interpretation targets, or exact scene partitions used across training and evaluation. The statement about official evaluation protocols is helpful, but does not resolve those training choices. Table 5 also describes the grounding ablation as replacing learned correspondence without saying what replaces it. These are material clarification needs for interpreting the results, rather than evidence of overlap or a failed method.
Questions for authors
  1. What measurements and experimental protocol generated Figure 2's model-specific values, including the MSR readout? If these cannot be established from existing records, can its status be restricted to an illustrative diagnostic?
  2. Please specify the scene partitions and language-model supervision used in each stage, including the source and construction of structured interpretation targets and whether benchmark question-answer training sets were used.
  3. What precisely replaces the physical field and language-guided grounding in the corresponding Table 5 variants? Which components and supervision are held fixed?
3 comments
AuthorsOpenReview ↗

Measurements, protocol, and empirical basis of Figure 2

We thank the reviewer for the careful reading of Appendix J. All values in Figure 2 are measured results.

  1. The values in Figure 2 are now supported by full-budget scores, per-example predictions, and a documented calculation. Appendix J lists the full-budget score of every model. Every score is computed from the per-question predictions of our own runs with the official VSI-Bench scoring. Each value in Figure 2 is the drop from the full-budget score of the same model, divided by this score.
  2. The token-retention protocol randomly retains 75%, 50%, 25%, or 12.5% of the visual tokens that each model feeds into its language model. All model weights and all other evaluation settings stay fixed.
  3. The MSR target is the metric 3D positions of scene points, and the MSR readout is a lightweight probe that is trained on the frozen retained tokens to predict these positions. The MSR score is the accuracy of the distances between the predicted points.
  4. The empirical basis of Figure 2 is the measurements themselves, and not the marker-to-bar arithmetic. Each marker is the measured result of one model, and each bar is the mean of its five markers. The whiskers are 95% bootstrap confidence intervals over scenes, computed from the same measured scores.

The complete protocol and the details of the probe are given in Appendix J.

AuthorsOpenReview ↗

Language-model training data and structured-interpretation targets

We thank the reviewer for pointing out these missing details, which are now stated in Sec. 4.1 and Appendix C.

  1. The scene partitions are the official splits. Training uses only the official training splits of ScanNet, ScanNet++, and ARKitScenes, and evaluation uses their validation splits. No evaluation scene appears in the training data.
  2. Both training stages use the same language-model supervision, which is the language-modeling loss on the same examples. These examples are the training sets of ScanQA, SQA3D, Scan2Cap, and Sr3D, together with question-answer pairs from GCoT, the dataset released with GS-Reasoner. From GCoT we use the training questions of six types, namely absolute distance, object size, room size, relative distance, relative direction, and route planning. Stage 1 keeps the language model frozen, and Stage 2 adapts it with LoRA.
  3. The structured-interpretation targets are constructed from the annotations that GCoT already provides, without additional manual labeling. Each GCoT question comes with its question type, the objects that the question refers to, and the 3D bounding boxes of these objects. The question type sets the track and the operation, the referred objects give the entities, and the 3D bounding boxes give the spatial support of each entity. The language model is trained to output this interpretation with the language-modeling loss.
  4. Benchmark question-answer training sets are used for ScanQA, SQA3D, and Scan2Cap, and these benchmarks are evaluated on their validation splits.
AuthorsOpenReview ↗

Settings of the Table 5 ablation variants

Each variant in Table 5 replaces exactly one component and is retrained from the same initialization on the same data.

  1. The physical field is replaced by a metric point memory. The point memory uses the same reconstruction, scale alignment, and filtering as GeomFunnel, but each point keeps only the features of its source view. This variant therefore has no sparse voxel field and no cross-view feature fusion, while grounding and the geometric operators still run on the points.
  2. Language-guided grounding is replaced by similarity matching. This variant keeps the same query interpretation and entity embeddings. It scores each voxel by the dot product between entity features and voxel features instead of using the grounding decoder.
  3. All other components and all supervision are held fixed. The fixed components are the reconstruction, the scale alignment, the filtering, the compact-token budget, and the geometric operators. The fixed supervision is the training scenes, questions, and splits, the grounding labels and loss, and the interpretation targets. In the point memory, the grounding supervision is applied to points instead of voxels.

Table 5 now names each variant after its replacement, and Appendix G gives the full settings.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.