acceptodds
Withdrawn from ICLR 2027

GAUGE3D: COORDINATE-GAUGE CONSISTENCY FOR DENSE 3D VISION–LANGUAGE GROUNDING

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Geometry-aware vision–language models ground language in 3D scenes, yet high accuracy in one coordinate frame can conceal sensitivity to arbitrary coordinate choices. Changing the global origin or orientation preserves the observed scene but requires dense masks, grounded text spans, and language answers to follow consistent transformation rules. We introduce , a framework for evaluating and learning coordinate consistency in dense 3D vision–language grounding. The key idea is to turn coordinate re-expression into paired supervision of the complete output interface. A geometry-only compiler transforms camera poses, world-space coordinates, and geometric targets while preserving the underlying color and depth observations. A joint objective constrains mask equivariance on a common source support, text-span grounding consistency, and answer invariance for frame-independent queries, with transformed targets for frame-dependent queries. A consistency audit activates paired training when repeatable discrepancies exceed the identity and compiler noise floor, linking diagnosis to targeted supervision without adding a new 3D encoder. Evaluations on ScanRefer, ScanQA, and SQA3D demonstrate that improves both dense grounding and spatial question answering over the underlying 3D vision–language model.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
3D vision-language grounding, coordinate consistency, geometric equivariance, spatial reasoning, dense grounding, multimodal learning

Reviews

Among:
Shape:
1234this paper, Bottom 1%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.00 over 3 reviews, above 0% of the others.
Reviewer_GbX3OpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

This paper presents GAUGE3D, which is a framework designed to assess and enhance the "coordinate-gauge consistency" in dense 3D visual-language grounding. The core observation is that a model's excellent performance in a single world coordinate system does not necessarily mean that it will maintain consistent performance when using equivalent coordinate transformation representations for the same RGB-D scene. Therefore, this paper defines corresponding transformation rules for different output modalities: geometric masks should be equivariant; the area pointed to by language localization should always be associated with the same physical entity; for queries independent of the coordinate system, the answers should remain unchanged; and for queries related to the coordinate system, the answers should be adjusted accordingly based on the corresponding coordinate transformation.

Critical strengths

None

Critical weaknesses

1.The paper demonstrates a very weak understanding and analysis of the problem, and the authors appear to lack even the most basic academic writing skills. As a result, the manuscript suffers from pervasive and serious issues in presentation and exposition, including, but not limited to, poorly defined mathematical formulations, inconsistent or inappropriate notation, and problematic formatting throughout the paper.

2.Moreover, the manuscript does not comply with the ICLR submission requirements. In particular, the paper exceeds the page limit specified by the official template. Given these substantial issues with both the technical understanding and the submission format, I recommend a direct desk rejection.

Questions for authors

see above

Reviewer_thNjOpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

This paper proposes Gauge3D, a 3D VLM training scheme designed with the central idea of coordinate transformation equivariance. The answers are changed according to a sampled coordinate change, and the consistencies at language and segmentation mask output are leveraged as additional training objective. The model is based on Qwen-3D and achieves around 1 point of improvement on ScanRefer, ScanQA, and SQA3D. However, the paper is poorly-written and should see a complete overhaul. It is not recommended for publication in the current edition (see W1&W2).

Critical strengths
  1. The idea of coordinate change equivariance on 3D VLMs is an interesting direction, upon which a great pack of manuscripts of equivariant point cloud representations may see a fresh application in the field of 3D VLMs.
Critical weaknesses
  1. Method section is poorly written and needs a complete re-write. Take Section 3.3 for an example: the style is too messy, many definitions are missing or unexplained: e.g., missing definition of τ(⋅)\tau(\cdot) in L164, reference of ‘IoU’ before definition in the same sentence (L167), unknown concept of ‘strata’ (L171), unexplained normalization and canonicalization rules (L171-172) (why is there such a word choice difference of ‘normalize’ and ‘canonicalize’, what is their difference), undefined ‘top-mask overlap’, ‘top-span grounding’, ‘answer correctness (do you mean accuracy?)’, unexplained symbol ‘⊮’ (L180), over-lengthy definition of ‘BothCorrect & Consistency’ (L192-195), etc, the list goes on and on. The most important one would be the loss of definition for τ(⋅)\tau(\cdot): it should act as a way of altering ground-truth answers to reflect the altered global reference frame; However, such explanation is not provided. Most of these items are not explained in the appendix either.

    This section is likely AI-generated since the logic is so compact, yet the word choices are deliberately diversified, the writing flaws are so frequent that it is unlikely directly written by a human. Whether AI-generated or not, this section’s poor readability detriments any attempt to understand the paper content, which is reflected as a great negative influence in my final rating. To make it clear, I am not against the use of AI; however, this section leaves lots of room for improvement possibly due to a suboptimal way of using AI. A more reasonable layout would be to fit in a proper level of detail allowed in the section length budget, and refer readers to a more detailed explanation in the appendix, rather than cramming everything in by super-condensing it.

    Finally, I would like to share some of my personal thoughts. Conference papers are meant to be written in a self-contained manner and maintain clear readability; This paper (especially Section 3.3-3.4) clearly did not live up to that standard. Such writing style is irresponsible and causes unnecessary waste of the valuable time and patience of the reviewing team and the ACs, a waste which the authors could have avoided by at least as reading the content through once.

  2. Continuing W1, the same level of poor writing quality is also observed in Section 4 experiments. Overall, the introduction, Section 3.1-3.2, and the conclusions section are still clearly readable; the rest sections all suffer from similar problems as listed in W1.

Questions for authors

See weakness.

Reviewer_QBsjOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper investigates the sensitivity of existing 3D vision–language models to changes in the global coordinate frame. It proposes GAUGE3D, a framework that pairs different coordinate representations of the same scene while keeping observations unchanged. Through paired consistency supervision, the framework encourages masks, text-to-object associations, and answers to remain consistent under the appropriate coordinate transformation rules.

Critical strengths

The paper addresses a novel problem with a clear motivation and a well-justified methodological approach.

Critical weaknesses

1. The three proposed loss terms are not defined in sufficient detail. Although their intended roles are described, the paper does not provide explicit mathematical formulations or explain how each term is computed. These details are necessary to understand and reproduce the training procedure.

2. Section 4.2 does not provide sufficient experimental details to substantiate H1. In particular, the confidence intervals and statistical testing details needed to assess the reported consistency gaps and equivalence claim are deferred to Appendix A, which is absent from the provided submission.

3. The motivation for Section 4.3 is unclear. Why is this particular text-only evaluation necessary, and what conclusion does it establish about the proposed method? The authors should clarify how the results support their central claims, beyond illustrating that language priors can contribute to performance on SQA3D.

4. The conclusions of Section 4.4 are difficult to interpret. The diagnostics show that translation affects different internal representations and voxel groupings, but their implications for end-to-end grounding performance remain unclear. In particular, which component is most sensitive to coordinate re-expression, and which contributes most to the observed output inconsistency? This attribution is the main insight I would expect this section to provide, yet the current analysis does not establish it.

5. The comparisons in Section 4.5 leave open whether standard coordinate augmentation, as implemented by Orbit Replay, is already sufficient to address much of the coordinate sensitivity motivating the paper. In Figure 4, Orbit Replay and Gauge3D exhibit very similar coverage curves across the tested transformations. Although Table 4 reports additional gains from Gauge3D, the paper does not adequately reconcile these findings or explain when explicit paired consistency supervision provides a meaningful advantage over augmentation alone. A more detailed analysis of Figure 4 and the differences between the evaluation metrics would help establish the necessity of the proposed losses.

Questions for authors

The submission appears incomplete, particularly in the experimental section, where several analyses lack the explanations needed to interpret their results. Table 4 is not explicitly referenced in the main text, and the content on the final page extends into the footer and overlaps the page number. I appreciate the paper’s motivation and believe the problem deserves further investigation. However, I encourage the authors to complete the experimental analysis, clarify the presentation, and address the formatting issues before resubmitting a more complete version.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.