- Rating
- 1
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
The paper presents GEOMFUNNEL, an architecture that encodes 3D information of the scene into both compressed dense tokens and geometry-aware 3D anchor tokens (the Coordinate-Grounded Sparse Physical Field). By leveraging expert models and filtering with multi-view agreement, the “Physical Field” is built. Questions are divided into two categories, where one queries dense tokens and the other queries the Physical Field. Experiments show that they achieve SOTA performance on VSI-Bench and remarkable performance on ScanQA and SQA3D.
Critical strengths
- The research problem the paper is working on is pressing: how to preserve 3D geometric information while catering to the need of compression.
- Experimental results are strong, showing the effectiveness of the proposed method.
- The paper is clearly written, and effectively conveys its ideas.
Critical weaknesses
- Lack of novelty. I am sorry for this comment, but as a reviewer I have to point out that this paper is incremental. It maintains the mainstream “compact representations” while adding the “Coordinate-Grounded Sparse Physical Field”. The latter is supposed to be the main innovation, but in fact it conveys little message that is newer than PointNet++(2017) [1]. I recommend the authors read this classical paper.
- Weak technical contribution. For instance, the whole construction of the representation is trivial. The reconstruction and metrics come from VGGT and UniDepthV2, the metric alignment is a simple average, the sparsification is filtering. I failed to discover any surprising or exciting designs that qualify this paper to match the level of ICLR.
- Figures seriously need polishing. The Figs. 1 and 2 have low information density. Fig.2 even does not contain the performance of the proposed method, which confused me for 5 minutes. The figures are likely to be drawn or designed by AI, and may not convey central or catchy information to the reader.
[1] Qi, Charles Ruizhongtai, et al. "Pointnet++: Deep hierarchical feature learning on point sets in a metric space." Advances in neural information processing systems 30 (2017).
Questions for authors
- Please summarize what is your major contribution in this paper except for the improved performance.
- Please compare the core of your representation with PointNet++, and explain the novelty. I apologize for the words I use in the review that may sound frank. Generally it is a solid paper, and I recommend submitting it to some easier conferences.
4 comments
Clarification of our technical contribution beyond VGGT-Ω and UniDepthV2
Our technical contribution is the workflow that turns the raw geometry from VGGT-Ω and UniDepthV2 into measured answers, because a direct combination of these two models has three problems.
- UniDepthV2 predicts metric depth for each frame independently, so its depth maps do not agree with one another in 3D, and using them directly would break the multi-view consistency of VGGT-Ω. GeomFunnel therefore keeps the reconstruction of VGGT-Ω unchanged and takes only one global scale from UniDepthV2.
- The combined reconstruction still contains locations that are seen in one view but contradicted by other views, and a measurement that includes them is unreliable. GeomFunnel therefore keeps only the locations that pass checks on confidence, visibility, depth consistency, and multi-view agreement.
- The combined geometry does not tell which locations belong to the objects in a question, so it cannot answer the question. GeomFunnel therefore attaches visual features to the locations and finds the locations of each object that the question mentions.
Table 5 and Appendix G compare GeomFunnel with simpler variants that are built from the same components.
Further explanation of Figure 1 and Figure 2
Figure 1 shows that existing compact 3D-VLMs decode metric answers from compact tokens, while GeomFunnel measures them on an explicit physical field. The purpose of Figure 1 is to present the central idea of the paper, which is that metric questions should have direct access to explicit geometry and only qualitative questions should rely on compact tokens.
Figure 2 shows that when existing 3D-VLMs keep fewer visual tokens, metric information becomes much harder to recover, while qualitative reasoning drops much less. The purpose of Figure 2 is to give the evidence for the problem that motivates this central idea, which is why Figure 2 reports only existing models.
Summary of our core contributions
Our core contribution is a new 3D vision-language framework (GeomFunnel) that performs metric reasoning directly on explicit physical geometry, addressing a fundamental limitation of existing approaches that generate metric answers from compressed visual tokens, where fine-grained geometric information is difficult to preserve and recover.
Our first three contributions depend on one another, and only together do they remove this limitation.
- The Coordinate-Grounded Sparse Physical Field takes metric geometry out of the compression path, but explicit geometry alone does not produce an answer.
- Representation-Decoupled Reasoning therefore moves the metric answer from text generation to computation on this field, but a computation needs to know where to act.
- Language-Guided Physical Grounding therefore links the words of a question to regions of the field, which existing models resolve only inside the tokens, and this completes the path from a question to a measurement.
Novelty of GeomFunnel in comparison with PointNet++
The key difference is that GeomFunnel addresses how a 3D vision-language model can answer metric questions and proposes to measure the answers on explicit geometry, while PointNet++ addresses how to learn features from point sets and proposes a hierarchical point encoder.
We now cite and discuss PointNet++ in the revised Related Work.