acceptodds
Under review as a conference paper at ICLR 2027

3D-GIA: Geometry-Aware Global Identity Learning for Open-Vocabulary 3D Gaussian Segmentation

Abstract

Open-vocabulary 3D segmentation localizes text-specified objects from novel views using a queryable 3D representation. Existing 3D Gaussian Splatting methods often distill view-dependent features and supervise Gaussians with independently generated or tracked masks, causing inconsistent view-local identities and semantic leakage across object boundaries. We propose 3D-GIA, which treats hierarchical masks as unordered observations and associates them in a scene-global identity space. Complementary 3D support, bidirectional reprojection, and appearance propose cross-view matches, while cannot-links provide explicit incompatibility evidence. Confidence-weighted voting projects global clusters to Gaussians as soft identity targets that preserve ambiguous mask-to-3D support. These targets supervise identity and language fields; detached identity predictions guide geometry. 3D-GIA preserves object-part relations, enabling consistent whole-object and part-level queries across views. It achieves 88.0% mIoU and 82.1% mBIoU on LERF-Mask, 96.2% mean mIoU on 3D-OVS, and more consistent cross-view identities than DEVA tracking under a controlled common-mask protocol. The learned identities also support text-guided removal, recoloring, and resizing.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
3D Gaussian Splatting, Open-Vocabulary 3D Segmentation, Scene-Global Identity, Multi-View Mask Association, 3D Scene Understanding, Language-Guided Segmentation, Geometry-Aware Learning

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_SghbOpenReview ↗
Rating
2
Confidence
4
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

The paper introduces 3D-GIA, a framework for open-vocabulary 3D segmentation with 3D Gaussian Splatting. It proposes a conflict-constrained association framework that generates persistent scene-global identities from unordered masks, together with a hierarchy-aware design that separates whole objects from parts. The resulting identities are used to supervise identity and language learning and to guide geometry refinement, aiming to improve cross-view mask consistency and fine-grained part segmentation.

Critical strengths

The paper attributes the main bottleneck of 3DGS-based segmentation to the lack of a persistent cross-view unit of correspondence, and proposes a conflict-constrained association framework to generate persistent scene-global identities from unordered SAM2 masks. This design is reasonable.

Critical weaknesses
  1. The proposed pipeline is quite complicated and relies on a large number of hand-crafted rules and hyper-parameters, the sensitivity analysis of the hyper-parameter selection is limited. And according to Appendix A.7 some parameters need to be tuned on held-out training views. Therefore, it is unclear if these settings and parameter selections can be transfer to new datasets or scenes without re-tuning.
  2. The proposed method includes hierarchy-aware designs and claims better handling of fine-grained parts. However, the evaluated benchmarks, i.e. LERF-Mask, 3DOVS and ScanNet, contain almost no object part-level queries. It would be better to evaluate the proposed method on LERF-OVS dataset (used by LangSplat), which contains more object part-level queries, and report whole-object and part queries separately.
  3. In ablation study table 5, w/o hierarchy module cause the mIoU performance on LERF-Mask drops to 68.8, which is a bit surprising because LERF-Mask almost contains no part queries. The queries that could possibly be affected could be eggs/pork belly/wavy noodles inside the ramen. It would be better if the authors can provide a per-query breakdown of this ablation. Moreover, 68.8 is lower than gaussian-grouping (72.8), which does not use any hierarchy and simply rely on DEVA to do the association, without conflict-constrained association. This post question on the effectiveness of the proposed conflict-constrained association, as gaussian-grouping (no hierarchy + DEVA) performs better than the ablated variant (no hierarchy + conflict-constrained association). A controlled comparison using the same masks for both association strategies would help clarify this.
Questions for authors
  1. See weakness 2. Can the authors provide comparison on LERF-OVS benchmark and report whole-object and part queries separately?
  2. See weakness 3. Can the authors provide a controlled comparison using the same masks (i.e. the one used by gaussian-grouping) and compare different association strategies.
5 comments
AuthorsOpenReview ↗

Part 1: Responses to Questions for Authors

Thank you for recognizing the motivation for persistent cross-view correspondence. We report new whole-object/part evaluation results and clarify the hierarchy and parameter-development protocol.

Response to Q1

We evaluated 3D-GIA on all four LERF-OVS scenes (67 scene-query entries; 208 annotated query-view pairs). For this additional evaluation, we freeze the trained Gaussian and identity fields and fuse text-relevance maps from the learned language field, training-view CLIP ViT-B/16 identity prototypes, and training-view CLIP ViT-L/14@336 identity prototypes. The three maps are min-max normalized individually and averaged equally. We aggregate 100 SAM candidates per query-view, each prompted with three positive points, using SAM-predicted IoU weights and Otsu binarization. This text-guided semantic-fusion/SAM inference rule is shared across all scenes and queries, without Grounded-SAM identity selection or optional hierarchy expansion.

Since the dataset does not provide whole-object/part labels, we manually annotate 55 whole-object and 12 part queries as a supplementary split, shared by all four methods. Parts include named components, attached accessories, and dish ingredients.

MethodOverall mIoU (%)Whole-object mIoU (%, 4 scenes)Part mIoU (%, 3 scenes)
LangSplat (CVPR 2024)48.5353.6519.80
LangSplatV2 (NeurIPS 2025)59.8865.2722.52
GAGS (AAAI 2026)53.2756.9818.65
3D-GIA65.0067.5053.81

Scores average query-view IoUs within each scene/group, then average eligible scenes equally. Overall and whole-object results cover all four scenes; part results cover ramen, figurines, and teatime. Waldo Kitchen has no part annotations and is not assigned a zero part score. These results are included in Sec. 4.6/Table 7, with inference and grouping details in Appendix G.3.

Response to Q2

We ran the requested control using shared pre-association SAM observations from GG's native pipeline. Both branches share the 8K Gaussian initialization, applicable downstream losses, unknown-pixel policy, training schedule, and frozen evaluation; DEVA retains its native sequence processing. Unobserved pixels are ignored rather than supervised as background, so sparse detections do not become artificial negative supervision. Single-level masks provide no parent-child relations, so neither branch uses hierarchy-specific association or supervision.

AssociationramenfigurinesteatimeMean (mIoU/mBIoU, %)
DEVA49.93 / 41.3569.46 / 66.7081.67 / 77.7867.02 / 61.94
3D-GIA46.65 / 38.9577.98 / 75.3484.07 / 80.7269.57 / 65.00

This common-input control tests association without hierarchical inputs. We report it in Sec. 4.7/Table 8(b), with the protocol in Appendix G.2.

AuthorsOpenReview ↗

Part 2: Responses to Critical Weaknesses

Response to W1

Preset development and cross-scene reuse are separate stages. The LERF-Mask SAM2 preset is developed on figurines training views and then reused for ramen and teatime. 3D-OVS uses another preset shared across its five scenes. This reuse is clarified in Sec. 4.1 (Implementation Details). Appendix A.8 describes the training-view development protocol and candidate settings; Appendix A.9/Table 10 lists the final coarse/fine SAM2 parameters. Once selected, the presets and association settings are fixed across scenes within each benchmark.

Across seven ScanNet scenes, we vary each of four association thresholds while fixing the other three. B3-F1 changes by only -0.48 to +0.26 points from the fixed setting's 76.94% (Appendix C.1/Table 12). This checks threshold sensitivity without changing masks or the other association settings.

Camera extent and neighborhood statistics provide scale and bandwidth adaptation (Appendix A.6), allowing common settings to accommodate different scene scales. These results support fixed-setting reuse across scenes and stability within the tested threshold ranges.

Response to W2 See Response to Q1 on LERF-OVS whole-object/part evaluation.

Response to W3

Hierarchy improves observations, association, and learning before a query is issued, so whole-object queries also benefit. Secs. 3.2–3.4 use fine regions and parent residuals to learn distinct, consistent identities for whole-object coverage and separation from neighboring instances; this does not require the query to name a part. The original w/o hierarchy module replaces hierarchical SAM2 inputs with single-level masks and removes hierarchy processing and coarse/fine supervision together. Thus, the drop from 88.0 to 68.8 measures a joint change to the hierarchy pipeline, rather than removal of a part-query function alone.

Following suggestions from the other reviewers, we have added these finer-grained ablations to Sec. 4.4/Table 5, with scene-wise results and factor definitions in Appendix G.1/Table 15. Hierarchical-input rows retain the same raw SAM2 coarse/fine masks while changing pre-association processing, parent-child association, or the whole hierarchy loss:

SettingMasksramenfigurinesteatimeMean (mIoU/mBIoU, %)
Full modelHierarchical88.8/78.589.4/87.185.9/80.788.0/82.1
w/o decompositionHierarchical88.3/78.586.1/84.080.8/75.885.1/79.4
w/o parent-child constraintsHierarchical86.7/77.172.6/70.874.3/68.577.9/72.1
w/o hierarchy lossesHierarchical87.3/77.186.1/84.081.1/75.784.8/79.0
w/o hierarchy module (Table 5)Single-level79.5/69.059.3/56.867.6/60.068.8/61.9

Decomposition separates parent residuals from child observations; parent-child association uses the retained relations to guide matches and prevent incompatible merges. The loss ablation removes the whole L_hier. Each single-factor ablation lowers mean performance, showing benefits from hierarchy processing and supervision beyond mask inputs.

The complete per-query comparison is below and in Appendix E/Table 14.

SceneQueryFull IoU (%)w/o hierarchy module IoU (%)
ramenchopsticks77.1661.52
ramenegg95.9490.87
ramenglass of water83.3342.89
ramenpork belly95.1496.83
ramenwavy noodles in bowl85.7891.01
ramenyellow bowl95.2694.09
figurinesgreen apple91.9995.31
figurinesgreen toy chair91.320.54
figurinesold camera84.8471.33
figurinesporcelain hand84.4258.61
figurinesred apple90.6894.88
figurinesred toy chair89.640.10
figurinesrubber duck with red hat93.0293.98
teatimeapple90.9794.07
teatimebag of cookies94.5887.86
teatimecoffee mug77.1572.34
teatimecookies on a plate89.6951.23
teatimepaper napkin91.4264.89
teatimeplate89.8189.14
teatimesheep88.8072.83
teatimespoon handle53.350.00
teatimestuffed bear90.4660.54
teatimetea in a glass93.6883.02

The per-query results show that the hierarchy ablation affects whole-object queries as well as parts. Together with the fixed-mask controls, this supports the hierarchy's role in whole-object coverage and identity separation, explaining why its removal can cause a substantial drop even on a benchmark with few part queries.

For the same-mask association comparison, see Response to Q2.

Dear Reviewer Sghb,

We hope the above clarifications and the additional experiments in the revised draft sufficiently addressed your concerns. If you are satisfied, we kindly request you to consider updating the score to reflect the newly added results and discussion. We remain committed to addressing any remaining points you may have during the discussion phase.

Regards from the authors.

Reviewer_TXkDOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper addresses inconsistent object identities and semantic leakage in open-vocabulary Gaussian segmentation. It associates hierarchical masks across views using geometric support, appearance, reprojection, and cannot-link constraints, then transfers these associations into soft Gaussian identity targets. The resulting identity field supports language-feature learning and geometry refinement, with strong results on LERF-Mask and additional evaluation on 3D-OVS and ScanNet.

Critical strengths
  • The shared identity representation connects mask association, semantic learning, and geometry refinement coherently.
  • LERF-Mask improvements are substantial, including gains in boundary accuracy.
  • Soft-target and cannot-link ablations provide useful evidence for individual design choices.
  • The appendix documents implementation details, repeated runs, and association statistics carefully.
Critical weaknesses
  • The association comparison does not establish an advantage over existing geometric methods. Gaga[1] already associates masks through 3D support, whereas Table 6 mainly compares against RGB-based DEVA. A comparison using identical observations, geometry, and downstream losses is needed.
  • The hierarchy ablation conflates better observations with better learning. Removing the module changes both mask proposals and supervision, causing the largest reported drop. Retain the same masks and separately remove decomposition, hierarchy losses, and hierarchical querying.
  • The language-field contribution remains unclear. LERF-Mask uses Grounded-SAM masks to select identities, while the text-based 3D-OVS improvement over SAGA is only 0.2 points. A text-only retrieval diagnostic and repeated 3D-OVS runs would clarify what semantic learning contributes.

[1] Gaga: Group Any Gaussians via 3D-aware Memory Bank

Questions for authors
  • How much boundary improvement comes from identity guidance rather than Gaussian flattening?
  • Were baseline preprocessing and retrieval parameters tuned using equivalent validation information?
3 comments
AuthorsOpenReview ↗

Part 1: Responses to Questions for Authors

Thank you for the review.

Response to Q1

In Sec. 3.5, identity-guided geometry uses detached identity predictions for within-identity continuity and cross-identity separation. Identity-independent shape constraints comprise Gaussian flattening (thickness), scale and footprint penalties. We ablate each block as a whole, retaining the other block and all remaining training terms. We measure boundary quality with mBIoU alongside mIoU.

Geometry regularizationmIoU (%)mBIoU (%)
Full (Table 5)88.082.1
w/o shape constraints87.381.6
w/o identity-guided geometry86.780.8
w/o geometry regularization (Table 5)83.277.2

With shape constraints fixed, identity guidance adds 1.3 mBIoU points; with identity guidance fixed, shape constraints add 0.5 points. Identity guidance thus provides the larger additional boundary gain, beyond flattening alone. These block-wise ablations are now in Sec. 4.4, Table 5.

Response to Q2

Table 1 retains seven published LERF-Mask baselines, preserving the results established under each method's published protocol. We reproduced ObjectGS (†) because its discrete object identities couple reconstruction and scene understanding, closely relating to our use of identity to connect semantics and geometry. We use its released 30K training preset and README-default corr initialization.

Regarding validation information, Table 1 follows published/released settings rather than re-tuning all methods on a common validation set. The new controlled comparison in Response to W1 directly matches the information and settings available to both association branches: preprocessing and downstream training are frozen, and each scene uses the same evaluation views, queries and retrieval parameters, without branch-specific tuning. The setting sources are now specified in Sec. 4.1; Appendix A.8 documents preset development, and Appendix G.2 gives the matched-control protocol.

AuthorsOpenReview ↗

Part 2: Responses to Critical Weaknesses

Response to W1

We replace only the association module with Gaga, sharing the same mask observations and 8K Gaussian geometry. Gaga's associated identities enter the same supervision interface, including soft identity targets, training pairs and hierarchy information. Downstream losses, 24K training and retrieval/evaluation settings remain fixed, so the comparison isolates the association strategy within the same complete framework. We have added this comparison in Sec. 4.7, Table 8(a), with its protocol in Appendix G.2.

Association in the 3D-GIA frameworkramenfigurinesteatimeMean (mIoU/mBIoU, %)
3D-GIA88.8 / 78.589.4 / 87.185.9 / 80.788.0 / 82.1
Gaga90.6 / 82.886.1 / 84.378.6 / 74.285.1 / 80.4

Our association improves the mean by 2.9/1.7 points. Gaga association in our framework reaches 85.1/80.4, versus 78.5/74.2 for published Gaga, supporting the value of the complete framework.

On ScanNet, 3D-GIA improves identity continuity over the Gaga adapter, reducing switches from 408 to 326 and fragmentation from 1.39 to 1.15.

Response to W2

Training uses parent/child targets and hierarchy losses to distinguish residual and part identities, without query expansion. Inference directly selects matching identities to preserve the query's object/part extent. Training with hierarchy does not require merging these identities at inference. Sec. 3.2 makes expansion optional; published benchmarks use direct selection.

The hierarchical-input variants retain the same original SAM2 coarse/fine mask bank.

SettingMasksramenfigurinesteatimeMean (mIoU/mBIoU, %)
Full modelHierarchical88.8 / 78.589.4 / 87.185.9 / 80.788.0 / 82.1
w/o decompositionHierarchical88.3 / 78.586.1 / 84.080.8 / 75.885.1 / 79.4
w/o parent-child constraintsHierarchical86.7 / 77.172.6 / 70.874.3 / 68.577.9 / 72.1
w/o hierarchy lossesHierarchical87.3 / 77.186.1 / 84.081.1 / 75.784.8 / 79.0
w/o hierarchy moduleSingle-level79.5 / 69.059.3 / 56.867.6 / 60.068.8 / 61.9

The single-factor ablation of parent-child constraints was added following another reviewer's suggestion. Decomposition subtracts child regions before association; its ablation changes this processing without regenerating masks and retains parent-child association and hierarchy losses. The original module ablation jointly changes masks and supervision. These ablations are now in Sec. 4.4, Table 5, and Appendix G.1, Table 15.

Hierarchy-aware querying controls selection granularity (Sec. 3.2): parent selections can include descendants for whole objects, while part selections stay child-specific. We provide hierarchy-query visualizations and quantitative comparisons in Appendix F, Fig. 8.

Response to W3

Thank you for pointing out an error in our inference description. The Grounded-SAM setting was intended to localize targets and evaluate cross-view identity propagation and segmentation by the 3D model. We mistakenly described this auxiliary setting as the main LERF-Mask evaluation. The Table 1 results do not use Grounded-SAM query masks: identities are selected by matching text queries to our model's decoded language features, with multiview consistency and semantic mask refinement. We have corrected this description in Sec. 3.6 (Training and Open-Vocabulary Inference).

Sec. 3.4 transfers identity agreement to semantic prototypes and positive/cannot-link language pairs, encouraging consistency within identities and separation between incompatible identities. Sec. 4.4, Table 5 isolates this training contribution: w/o identity-semantic transfer, mIoU/mBIoU drops from 88.0/82.1 to 82.1/76.0.

We have added the 3D-OVS identity-semantic transfer ablation in Appendix G.4, Table 16:

SettingbedbenchroomsofalawnMean mIoU (%)
Full model (Table 2)94.995.897.795.597.196.2
w/o identity-semantic transfer94.188.289.189.293.090.7

The decrease across all five scenes supports the contribution of identity-semantic transfer to text-based retrieval.

We completed three fresh-seed 3D-OVS runs under the paper's training and evaluation settings (Supplementary Sec. G.5, Table 17). Each entry is the equal-weight average over five scenes:

RunmIoU (%)
195.19
296.15
395.94
Mean ± sample SD95.76 ± 0.50

Dear Reviewer TXkD,

As the discussion phase is approaching its end, we kindly request the reviewer to let us know if the above clarifications and the previously added experiments have addressed the remaining questions. We would be happy to address any additional points the reviewer may have during the remaining time of the discussion phase. We thank the reviewer for engaging with us in the discussion.

Regards from the authors.

Reviewer_9C5oOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

3D-GIA proposes a framework for open-vocabulary 3D Gaussian segmentation that associates hierarchical multi-view masks into scene-global identities using 3D support, reprojection, appearance, and cannot-link constraints. The resulting soft identity targets supervise identity and language learning, while identity predictions guide geometry regularization. The framework also preserves object–part relations for structured querying. Evaluation on LERF-Mask and 3D-OVS reports improved segmentation performance, complemented by controlled cross-view association experiments on ScanNet, component ablations, repeated training runs on LERF-Mask, and qualitative demonstrations of text-guided scene editing.

Critical strengths
  • Coherent integration through scene-global identities. The shared identity representation connects cross-view association, language learning, and geometry regularization while preserving object–part relations. This provides a technically coherent basis for consistent segmentation and structured querying, supporting the value of the overall system.

  • Meaningful empirical gains with supporting validation. On LERF-Mask, the method improves mIoU/mBIoU by 4.0/3.3 percentage points over the strongest reported baseline. Repeated training runs and controlled association experiments with identical input masks strengthen confidence in its effectiveness.

Critical weaknesses

Limited novelty

The pipeline is coherent, but most components have close precedents, such as hierarchical masks in LangSplat and persistent cross-view identities in Gaussian Grouping. The paper does not clearly explain why their combination constitutes a new algorithmic contribution rather than an engineering integration. The authors should either better distinguish the proposed mechanisms from prior work or narrow the contribution claims.

Experiments

  1. If I understand the protocol correctly, in the ScanNet experiment, 3D-GIA uses depth, camera poses, intrinsics, and depth-to-color calibration, while DEVA uses only RGB and the same masks. Thus, the reported gain may partly come from additional geometric information rather than global clustering or cannot-link constraints. The authors should provide a geometry-matched baseline or evaluate 3D-GIA under RGB-only/estimated-depth settings.

  2. The “w/o hierarchy” ablation changes both the mask generation process and the training objectives: hierarchical masks and coarse/fine supervision are removed simultaneously. Hence, the large performance drop does not reveal whether the gain comes from better masks, parent–child constraints, or hierarchy losses. Single-factor ablations with a fixed mask bank are needed.

Questions for authors
  1. Novelty. How does the proposed method differ fundamentally from prior approaches such as LangSplat and Gaussian Grouping? Which components are genuinely new, and should the contribution claims be narrowed to emphasize the proposed integration?

  2. Fairness of the ScanNet comparison. In the controlled association experiment, 3D-GIA uses depth, camera calibration, and registered poses, whereas DEVA uses only RGB and the same masks. How much of the performance gap is attributable to the additional geometric information? Can the authors provide a geometry-matched baseline or an RGB-only/estimated-depth version of 3D-GIA?

5 comments
AuthorsOpenReview ↗

Part 1: Responses to Questions for Authors

Thank you for the review. We clarify the algorithmic contribution and the role of geometry below.

Response to Q1

3D-GIA introduces mechanisms for constructing cross-view identities and turning them into supervision (Secs. 3.2-3.5). First, conflict-constrained, hierarchy-aware association: LangSplat learns granularity-specific language features, while Gaussian Grouping uses tracked identities. We replace parent masks with child-excluded residuals and cluster these alongside child masks, preserving their distinct identities and relations. Geometry and appearance propose matches, but any cannot-link between members of two components vetoes their merger, including an indirect merger through other observations. This makes conflict handling a constraint on global identity construction. Second, uncertainty-aware semantic transfer and identity-guided geometry: confidence-weighted votes produce soft Gaussian identity targets. Agreement between these targets and detached identity predictions determines semantic prototypes and transfer weights, so uncertain association contributes less to language supervision. Gaussian Grouping smooths identity features; our detached identity differences guide Gaussian positions, scales, and rotations toward continuity within identities and separation across boundaries. These mechanisms change both supervision and optimization. Table 5 supports their individual effects: removing cannot-links or identity-semantic transfer reduces mIoU by 4.0 or 5.9 points; replacing soft targets with one-hot targets reduces it by 2.4. We have revised the Introduction, including the contribution list, and Related Work to state the contribution more precisely as hierarchy-aware association and identity-guided learning, explicitly crediting existing multi-granularity masks, tracking, and pretrained features as foundations; Secs. 3.2-3.5 detail the mechanisms.

Response to Q2

3D-GIA requires 3D support and reprojection geometry for association, but does not require externally measured depth. In the main posed-RGB experiments, this geometry comes from reconstruction. In the original ScanNet diagnostic, we used official depth, intrinsics, poses, and depth-to-color calibration to fix the geometry and avoid conflating RGB reconstruction quality with association performance. DEVA uses RGB and the shared masks. Thus, the original comparison evaluates the complete geometry-based association pipeline; the fixed-geometry variants test its association design.

To address your requested RGB-only setting, we estimate cameras and sparse geometry with COLMAP, perform an 8K RGB Gaussian warm-up, and construct 3D support from rendered depth. This retains geometry-based association while obtaining its geometry from RGB. No official ScanNet depth, intrinsics, poses, or depth-to-color calibration is provided to reconstruction or association in this branch. We have added 3D-GIA RGB-only to Table 6 in Sec. 4.5 and documented the RGB reconstruction protocol in Supplementary Sec. C.3; the original rows are unchanged.

MethodAssA ↑AQ ↑IDF1 ↑B³-F1 ↑Sw. ↓Fr. ↓Cov. ↑
DEVA tracking55.3349.0168.5169.318483.0392.30
3D-GIA RGB-only56.0450.0964.9873.243421.3797.63
3D-only39.8144.2646.4059.53(213)(0.80)98.46
w/o C-link56.2054.6964.2073.45(277)(0.96)98.46
3D-GIA assoc.60.6556.5870.8376.943261.1598.46

Block 1 (DEVA tracking, 3D-GIA RGB-only) uses RGB and supplied masks without official geometry. Block 2 (3D-only, w/o C-link, 3D-GIA assoc.) uses official depth and camera calibration/poses. Both evaluate the same 639 masked views. RGB reconstruction uses 2,835 registered frames. Sw. follows the original filename order.

RGB-only 3D-GIA outperforms DEVA on most metrics, showing that its association benefits persist without official geometry. The stronger official-geometry results demonstrate the value of reliable geometric support. Within that fixed-geometry setting, the w/o C-link comparison establishes an additional benefit from conflict constraints. Together, these controls show that the gains involve both geometric support and the association mechanism.

AuthorsOpenReview ↗

Part 2: Responses to Critical Weaknesses

Thank you for identifying the concerns about novelty, comparison inputs, and the hierarchy ablation.

Response to W1

Response to Q1 explains how global conflict constraints, hierarchy-aware identity construction, and uncertainty-aware transfer change supervision and optimization. We have clarified this scope in the Introduction and Related Work, with the mechanisms detailed in Secs. 3.2-3.5.

Response to W2

Response to Q2 provides the requested RGB-only ScanNet results and distinguishes the RGB-reconstructed and official-geometry settings (Sec. 4.5, Table 6; Supplementary Sec. C.3). The new RGB-only branch retains the association pipeline without supplying official geometry; the original fixed-geometry ablations separately test the association rules.

Response to W3

The original w/o hierarchy module variant measures the combined change in mask inputs and hierarchical supervision. To separate these factors, we have added single-factor ablations that retain the same original SAM2 coarse/fine masks:

SettingMasksramenfigurinesteatimeMean (mIoU/mBIoU, %)
Full modelHierarchical88.8/78.589.4/87.185.9/80.788.0/82.1
w/o decompositionHierarchical88.3/78.586.1/84.080.8/75.885.1/79.4
w/o parent-child constraintsHierarchical86.7/77.172.6/70.874.3/68.577.9/72.1
w/o hierarchy lossesHierarchical87.3/77.186.1/84.081.1/75.784.8/79.0
w/o hierarchy module (Table 5)Single-level79.5/69.059.3/56.867.6/60.068.8/61.9

The pre-association decomposition ablation was added following another reviewer's suggestion. All hierarchical-input variants start from the same coarse/fine mask proposals. Decomposition subtracts child regions from a parent before association, keeping parent residuals and parts as separate observations. Its ablation retains the original proposals and removes this subtraction. The w/o parent-child constraints variant removes the parent-derived association mechanism as a whole while retaining decomposition and hierarchy losses. The w/o hierarchy losses variant retains the association pipeline and removes the complete hierarchy-loss objective. Each single-factor variant therefore changes the indicated operation without regenerating the masks. The full model and original single-level reference reuse Table 5.

Each single-factor removal lowers mean performance with the mask inputs fixed, showing that the hierarchy design contributes beyond the input masks. The original 19.2-point mIoU drop measures the complete hierarchy pipeline, rather than the independent effect of one hierarchy-loss term. These finer-grained ablations are now included in Sec. 4.4, Table 5, with scene-wise results and factor definitions in Supplementary Sec. G.1, Table 15.

Dear Reviewer 9C5o, As the discussion phase is approaching its end, we kindly request the reviewer to let us know if the above clarifications and the previously added experiments have addressed the remaining questions. We would be happy to address any additional points the reviewer may have during the remaining time of the discussion phase. We thank the reviewer for engaging with us in the discussion. Regards from the authors.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.