acceptodds
Under review as a conference paper at ICLR 2027

Geometry-Aware Feature Fusion for Open-Vocabulary 3D Segmentation

Abstract

Open-vocabulary 3D segmentation relies on integration of multi-view language features into a spatially coherent scene representation. Although existing language and instance representations provide semantic and object-level information, their contributions to individual 3D elements may become ambiguous, since region-level observations can span neighboring objects or cover only parts of a target. To address this issue, this work presents Geometry-Aware Feature Fusion (GAFF), a framework that uses geometric relationships and instance structure to guide the integration of language evidence into a 3D representation, without requiring additional gradient-based optimization. Geometrically admissible observations are weighted according to depth consistency, local region agreement, and instance compatibility, then combined with the existing scene semantics. This formulation grounds semantic aggregation in both local geometry and object-level structure. Although this fusion refines scene semantics, language relevance alone does not determine the correct spatial extent of a queried object. We therefore further use object-guided mask refinement to select among multilevel language masks and adjust their extent using object candidates. GAFF achieves 68.45% mIoU on LERF-OVS and 94.88% on 3D-OVS. On a ten-scene subset of ScanNetV2, it achieves 38.28% mIoU for 19-class point-level semantic segmentation. Controlled fusion experiments demonstrate improved semantic prediction without mask editing, while component studies show complementary benefits from feature fusion and mask refinement. These results highlight the value of using geometry and instance structure to organize multi-view semantic evidence for open-vocabulary 3D segmentation.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Open-Vocabulary 3D Segmentation, Geometry-Aware Feature Fusion, Multi-View Semantic Aggregation, Instance-Aware Scene Understanding, Object-Guided Mask Refinement

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_uau6OpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

GAFF refines existing language representations by weighting region observations according to geometry, local region agreement, and instance compatibility. An anchored update combines these observations with the original features, after which object-guided mask refinement selects and edits query masks. Experiments cover rendered segmentation and point classification, including mass-matched controls intended to isolate relative evidence allocation.

Critical strengths
  • The mass-matched uniform control addresses an important confound in weighted aggregation.
  • Fusion and mask editing are evaluated separately as well as together.
  • Scene-level diagnostics include failures and harmful prediction changes.
  • Refinement reuses fixed representations without additional gradient-based optimization.
Critical weaknesses
  • The contribution of the geometric weighting factors is uncertain. Beyond COS3D’s instance–language integration, matched weighting adds 0.95 ScanNet mIoU, but instance weighting alone essentially matches the full product in Table S4. Mass-matched factor ablations are needed to establish what depth and local agreement add.
  • Most rendered-mask gains depend on an elaborate editor whose transfer is untested. OMR alone improves LERF mIoU by 6.60 points, compared with 1.23 for fusion alone. Document how its dataset-specific gates were selected and test frozen rules on held-out scenes.
  • External comparisons retain incompatible evaluation conditions. Appendix F acknowledges different ScanNet point-filtering domains and different cross-view prediction constraints on LERF. Reevaluate principal baselines under shared conditions and measure cross-view consistency explicitly.

[1] COS3D: Collaborative Open-Vocabulary 3D Segmentation

Questions for authors
  • What development data and selection procedure determined the OMR thresholds?
  • Can the weighting reject mixed-object observations when instance compatibility measures coverage rather than region purity?
5 comments
AuthorsOpenReview ↗

Response to Reviewer uau6 Part1

W1: Contributions of depth and local agreement

We thank you for recognizing the mass-matched control, separate and combined component evaluation, and refinement of fixed representations without additional gradient-based optimization. Your suggestions focus on identifying the effective weighting cues, validating the editor, and making comparisons more informative. We completed the factor ablations, parameter sensitivity, shared scoring of local baselines, cross-view measurements, and mixed-region analysis. We have incorporated the corresponding clarifications and additional results into the revised manuscript PDF, with the revisions marked in blue text for ease of review.

“Mass-matched factor ablations are needed to establish what depth and local agreement add.”

The new ablation shows that coverage provides the main gain, depth adds a smaller improvement, and local agreement makes a modest adjustment to incoming evidence. We completed an eight-configuration matched-mass ablation on ten ScanNet scenes, 1,158 views, 1,569,975 points, and 18,204,562 accepted associations (Appendix J.1, Tables S14–S15). For a factor subset SS, we use

\qquad M_p=\sum_o d_{po}b_{po}h_{po},$$ where $q^{(S)}$ is the factor product, or 1 for Uniform. Accepted observations, initial fields, total mass, effective anchor strength, and cosine readout are fixed; OMR and optimized readout components are disabled. | Factors | 19-class mIoU (%) | 15-class mIoU (%) | 10-class mIoU (%) | | --- | ---: | ---: | ---: | | Uniform | 36.7398 | 36.0969 | 44.3463 | | $d$ | 36.8304 | 36.2013 | 44.5041 | | $b$ | 36.7453 | 36.0942 | 44.3585 | | $h$ | 37.6827 | 37.0532 | 45.6116 | | $dh$ | **37.7081** | **37.1001** | **45.6808** | | $bh$ | 37.6666 | 37.0375 | 45.5923 | | $dbh$ | 37.6899 | 37.0852 | 45.6601 | | $db$ | 36.8237 | 36.1879 | 44.4944 | **The full product preserves the principal allocation gain, with a very small difference from $dh$.** Coverage contributes **0.9429/0.9564/1.2654 pp** over matched Uniform, and adding depth gives **0.0254/0.0469/0.0691 pp**. The advantage of $dh$ over $dbh$ is **0.0182/0.0149/0.0206 pp**, approximately 0.02 pp at most. Full $dbh$ improves over matched Uniform by **0.9501/0.9883/1.3139 pp**. **Why retain $b$?** It supplies local information complementary to depth and instance coverage. From its definition, $b=1$ when all sampled neighbors agree with the projected pixel’s region label; it decreases toward its floor as agreement falls. Thus, even when depth and instance coverage are similar, the weight can respond to local region-boundary ambiguity. This is the design reason for including $b$ in the joint allocation rule. The completed mixed-region diagnostic provides an observation-level check. Across low, medium, and high semantic-purity groups, weighted region-label correctness increases from **18.7310/36.5651/59.2220% with $dh$ to 18.7614/36.5952/59.2552% with $dbh$**. The mean $b$ values of **0.9752/0.9831/0.9853** are consistent with a modest local adjustment. We therefore retain $b$ to encode local association stability alongside surface compatibility and instance coverage, while preserving the full eight-configuration comparison in Appendix J.1. The final point labels come from the argmax after multiview accumulation and anchoring, explaining why observation-level changes need not produce identical changes in final accuracy.
AuthorsOpenReview ↗

Response to Reviewer uau6 Part2

W2 & Q1: Development data, thresholds, and frozen-rule evaluation

“Document how its dataset-specific gates were selected and test frozen rules on held-out scenes.”

“What development data and selection procedure determined the OMR thresholds?”

Selection protocol. The reported benchmark scenes served as development data; a separate held-out selection set was not used. Fusion parameters and the scoring coefficients within each route are shared, while the OMR controllers also include dataset-, scene-, and query-dependent settings. Table S12 (Appendix I.3) lists the principal values, and Appendix D describes the controllers. RGB refinement was selected through two search rounds totaling 18 configurations on five 3D-OVS scenes. We then fixed the selected configurations for the controlled comparisons.

To test the practical concerns behind the question—whether the editor adds value and whether that value depends on precise parameter choices—we completed two complementary experiments. Fixed-seed/candidate comparisons show that complete editing exceeds the strongest tested simple alternative by 3.1349/11.3536 pp on 3D-OVS/LERF, excluding RGB refinement. Perturbing editing parameters gives 94.5112–94.6412% / 67.8466–68.4450%, with maximum absolute macro changes of 0.1260/0.5983 pp. Appendices I.2–I.4 provide the parameter settings, perturbations, and complete-controller ablation. These results quantify the complete editor’s benefit and its sensitivity around the selected settings.

Regarding the requested held-out-scene test: we appreciate its value for assessing transfer to new scenes. A complete evaluation with frozen rules cannot be completed within the rebuttal period. We have prioritized the above editing comparisons and sensitivity experiments (Tables S11–S13) to provide concrete evidence within the available time. They establish the editor’s measured benefit and local parameter sensitivity; the requested held-out transfer test remains a separate evaluation.

AuthorsOpenReview ↗

Response to Reviewer uau6 Part3

W3: Shared scoring of principal baselines and cross-view consistency

“External comparisons retain incompatible evaluation conditions.”

“Reevaluate principal baselines under shared conditions and measure cross-view consistency explicitly.”

Shared scoring of principal baselines. Following your suggestion, we rescored the available DrSplat, COS3D, and GAFF point predictions under a common protocol (Appendix J.3, Tables S18–S19). ScanNet uses the same valid GT points, class mapping, shared text assets, and scene-macro aggregation; unknown predictions count as errors.

Method19-class mIoU (%)15-class mIoU (%)10-class mIoU (%)
DrSplat (local, rescored; shared text)36.293436.810844.7791
COS3D (local, rescored; shared text)33.467532.859141.1075
GAFF with cosine readout (dbhdbh)37.689937.085245.6601
Matched Uniform36.739836.096944.3463

GAFF with cosine readout leads the listed local comparisons across all three class settings. The appendix additionally evaluates every method on the same opacity-filtered domain. DrSplat uses its fixed 4NN interpolation adapter, and GAFF uses the cosine-readout configuration of Appendix J.1.

For LERF, the same units, GT rasterization, grid resizing, and aggregation give 53.7917% for locally rescored LangSplatV2 and 68.4450% for OMR across four scenes. On figurines, locally rescored COS3D obtains 56.8657%, versus 63.8688% for OMR. Table S20 (Appendix J.3) reports both domains explicitly.

These comparisons show GAFF’s prediction benefit when the scoring domain and aggregation are shared, directly strengthening the benchmark evidence.

Cross-view consistency.

We completed the explicit consistency measurement you requested on native seed, morphology, and OMR outputs (Appendix J.2, Tables S16–S17), using reciprocal COLMAP tracks with fixed geometric filters.

OMR achieves 68.4450% full-set 2D mIoU, compared with 54.7334% for the seed, and 76.2115% paired foreground consistency on 40 queries and 203 query–view pairs (Table S17). The paired seed consistency is 78.7467%. Among the 40 queries, 26 retain agreement and four improve; sampled-label disagreement is 1.9401%. Appendix J.2 provides the complete seed/morphology/OMR comparison and the metric definitions. The diagnostic measures sparse tracked surfaces, with 0.00990% mean sampled-pixel coverage.

The results quantify substantial foreground agreement together with more accurate query-mask extent. Fusion’s representation-level gains are established separately through the matched-mass point-prediction controls. This gives a direct evaluation of both the semantic allocation and output refinement contributions.

Reviewer_BtsBOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes GAFF, a framework for refining open-vocabulary 3D segmentation without additional gradient-based optimization. It fuses multi-view region features using geometric admissibility, depth consistency, local-region agreement, and instance compatibility, while retaining initial semantics through an anchored update. An object-guided mask refinement module subsequently selects and edits rendered query masks. Evaluation on LERF-OVS, 3D-OVS, and ten ScanNetV2 scenes includes component ablations and mass-matched fusion controls, reporting improvements in query-mask segmentation and point-level semantic prediction.

Critical strengths
  1. Well-controlled validation of feature fusion. The mass-matched uniform baseline preserves accepted observations, total observation mass, and effective anchor strength. GAFF improves mIoU by approximately 0.95–1.31 percentage points across the three ScanNet class settings, providing credible evidence that relative evidence allocation itself benefits semantic prediction.

  2. Complementary components with measurable practical benefits. Under matched settings, the complete framework achieves 68.45% mIoU on LERF-OVS, compared with 59.13% for fusion alone and 64.50% for OMR alone. This supports the value of combining semantic feature refinement with object-guided mask editing.

Critical weaknesses

1. Novelty and significance: The contribution of the main performance gains to 3D segmentation remains unclear.

The experiments suggest that the full system’s substantial gains depend heavily on OMR, which incorporates numerous empirical rules for candidate selection, gating, and mask editing. In particular, OMR directly refines view-specific 2D masks after rendering. These edits are not propagated back to the Gaussian representation, and the final masks across views are not guaranteed to correspond to a shared selection of 3D Gaussians. Consequently, OMR’s gains primarily demonstrate improvements in final 2D query-mask quality and do not, on their own, establish improvements in 3D object segmentation or cross-view consistency. I'm not sure if my understanding is incorrect. If so, please kindly correct me.

2. Experimental validation: OMR’s parameter selection and generalizability are insufficiently documented.

OMR substantially improves performance, but its scoring weights, acceptance thresholds, and various fallback rules rely on empirical configurations. The paper does not sufficiently explain which data were used to select these parameters, whether they are shared across scenes and queries, or whether they remain effective once frozen. Holding parameters fixed within each ablation supports component comparisons, but does not sufficiently establish the independence of parameter selection from evaluation data or the transferability of the rules. Please clarify the configuration-selection protocol and provide a limited analysis of parameter transfer or sensitivity.

Questions for authors

See Weaknesses.

My assessment and score may change based on the authors’ clarifications and supporting evidence.

6 comments
AuthorsOpenReview ↗

Response to Reviewer BtsB Part1

W1 & corresponding question: Contribution to 3D segmentation

We thank you for recognizing the well-controlled fusion validation and the measurable benefits of the complementary components. Your central question is what GAFF contributes to 3D segmentation beyond the improvement in rendered masks. We address that distinction first, followed by the requested parameter-selection and sensitivity evidence. We have incorporated the corresponding clarifications and additional results into the revised manuscript PDF, with the revisions marked in blue text for ease of review.

“The contribution of the main performance gains to 3D segmentation remains unclear.”

“OMR directly refines view-specific 2D masks after rendering. These edits are not propagated back to the Gaussian representation, and the final masks across views are not guaranteed to correspond to a shared selection of 3D Gaussians.”

You are correct that OMR edits rendered masks. The 3D representation contribution comes from fusion. Fusion allocates the contributions of region observations to each 3D element and combines them with an initial semantic anchor, without further gradient-based optimization. As you noted in your strengths, the matched-mass control already provides credible evidence for this contribution: Table 4 (§4.3) reports gains of 0.9501/0.9883/1.3139 pp on ten ScanNet scenes across 19/15/10 classes. The accepted observations, total mass, effective anchor strength, and cosine readout are identical, with OMR and optimized readout disabled. The improvement therefore measures how the same evidence is allocated to the 3D representation.

The new experiments explain the mechanism and test the design beyond this original control. The eight-configuration matched-mass ablation identifies visible-instance coverage as the dominant soft-weighting cue (Appendix J.1, Tables S14–S15). Source-replacement experiments further show that both fusion and object-guided editing can use OpenSplat3D inputs (Appendix I.1, Tables S8–S10). Together, these results support GAFF as a way to refine existing semantics using geometry and instance structure, with the role of each cue made explicit.

OMR addresses the complementary task of query-mask extent. Its edits operate on the rendered masks. With native inputs fixed, complete editing exceeds the strongest tested simple editor by 3.1349/11.3536 pp on 3D-OVS/LERF (Appendix I.2). Thus the paper supplies both point-level evidence for semantic fusion and mask-level evidence for object-guided refinement.

To make the cross-view behavior explicit, we added reciprocal-COLMAP-track measurements (Appendix J.2, Tables S16–S17). OMR achieves 68.4450% full-set 2D mIoU, compared with 54.7334% for the seed, and 76.2115% paired foreground consistency in the paired evaluation in Table S17, covering 40 queries and 203 query–view pairs; the paired seed score is 78.7467%. Of the 40 queries, 26 retain their consistency and four improve. The diagnostic evaluates sparse tracked correspondences with 0.00990% mean sampled-pixel coverage. These measurements quantify substantial foreground agreement alongside the improvement in query-mask accuracy.

The contribution is therefore supported at both output levels: controlled point prediction validates evidence allocation in the 3D representation, and fixed-input mask editing validates object-guided extent correction. Sections 1, 3.3, 4, and 5 now make this relationship explicit.

AuthorsOpenReview ↗

Response to Reviewer BtsB Part2

W2 & corresponding question: Parameter provenance and generalizability

“Please clarify the configuration-selection protocol and provide limited analysis of parameter transfer or sensitivity.”

Thank you for suggesting parameter transfer or sensitivity as a concrete way to strengthen the evidence. We completed the sensitivity analysis and clarified parameter provenance (Appendices I.3–I.4, Tables S12–S13). The reported benchmark scenes served as development data; a separate held-out selection set was not used. Fusion parameters and scoring coefficients within each route are shared, alongside dataset-, scene-, and query-dependent OMR settings described in Appendix D. RGB refinement was selected through two search rounds totaling 18 configurations on five 3D-OVS scenes.

The sensitivity study covers nine scenes and 363 units, with native seeds/candidates fixed and RGB refinement excluded. Seven 3D-OVS scalars vary individually by ±10%, kernel caps by ±2; LERF tests four joint ±10% gate corners and individual ±10% changes to two normalized score weights.

DatasetmIoU with RGB refinement disabled (%)Perturbed range (%)Maximum absolute macro change (pp)
3D-OVS94.637294.5112–94.64120.1260
LERF-OVS68.445067.8466–68.44500.5983

The small macro-average changes show stable aggregate accuracy around the selected settings. Fixed-input comparisons establish that complete editing improves on simple alternatives, while Appendix I.4 reports the full perturbation results and a complementary complete-controller ablation.

A complete frozen-rule evaluation on held-out scenes cannot be completed within the rebuttal period. We have therefore addressed the sensitivity analysis you proposed, with explicit parameter settings and perturbation ranges. Together with Table S11, this provides evidence for both the editor’s benefit over simpler alternatives and its stable aggregate performance around the selected settings.

We respectfully ask the reviewer to reconsider the assessment in light of the independently validated 3D contribution, the explicit distinction between output settings, and the completed provenance/sensitivity analysis.

Thank you for the detailed rebuttal and additional experiments. I appreciate the authors' efforts to address my concerns.

However, my primary concern remains the 3D nature of the proposed framework. One of the key advantages of 3D semantic Gaussian representations is their ability to support efficient and cross-view consistent semantic rendering. In contrast, a substantial part of GAFF relies on view-specific 2D mask refinement through OMR, without updating the underlying 3D semantic representation.

Although the authors demonstrate improvements in 3D point-level semantic prediction, the major gains in the final segmentation results still rely heavily on 2D post-processing.

Therefore, I remain unconvinced that the improvements in rendered 2D segmentation masks sufficiently demonstrate advances in coherent 3D segmentation.

At this stage, I lean toward maintaining my original rating, although I would not strongly oppose acceptance if the paper receives sufficient support from other reviewers.

Reviewer_1i4yOpenReview ↗
Rating
2(+1)
Confidence
2
Informativeness
1
Presentation
1
Soundness
2
Read review
Paper summary

This paper proposes GAFF, a framework for open-vocabulary 3D segmentation. The authors argue that lifting 2D language features to 3D is inherently ambiguous, since 2D regions may span neighboring objects or cover only part of a target. To address this, they propose two components:

  • GAFF fusion: determines which observations contribute to each 3D element, and how much, using depth consistency, local region agreement, and instance compatibility.
  • Object-Guided Mask Refinement (OMR): corrects query-mask extent, which can be inaccurate even when semantics are correct, using object candidates derived from COS3D instances.
Critical strengths

S1. The paper identifies a reasonable limitation of prior work: ambiguous region-to-3D feature lifting.

S2. The method achieves strong performance on three benchmarks: LERF-OVS, 3D-OVS, and ScanNetV2.

Critical weaknesses

W1. The stated limitation and the method are not well connected. It is unclear how each component of the method resolves the 2D ambiguity identified as the core limitation.

  • Expressions describing the limitation, such as “region-to-element reliability” (L51), are hard to follow.
  • Figure 1 is overly complex and does not convey the core idea.
  • The notation in Eq. 2 is defined across several places, and subscripts are used inconsistently (po vs. pv).

W2. Reliance on COS3D. Both the fusion stage and OMR depend on COS3D. The authors should verify whether the method works with other instance representations. A description of COS3D on preliminaries would be beneficial.

W3. OMR seems to be overly engineered. OMR relies on many ordered heuristic routes (e.g., L1 rescue, background rescue, replacement, patching, fallback) with numerous thresholds and weights. Its editing pipeline also differs between 3D-OVS and LERF-OVS, and the hyperparameters are only described as “empirically selected.”

Questions for authors
  • How were the OMR hyperparameters selected? Was a held-out set used, and how sensitive is performance to these choices?
  • What is the runtime and memory overhead of fusion and OMR per query?
7 comments
AuthorsOpenReview ↗

Response to Reviewer 1i4y Part1

W1: Connecting the limitation, components, figure, and notation

We thank you for recognizing the importance of ambiguous region-to-3D feature lifting and the strong performance on three benchmarks. Your main concerns are the connection between this problem and our design, dependence on COS3D, and the complexity and practical cost of OMR. We address these in order below. GAFF improves which region observations contribute to a 3D element and how much; OMR then uses object structure to correct query-mask extent. We have made this distinction explicit in Figure 1 and Section 3 and added experiments that test each contribution. We have incorporated the corresponding clarifications and additional results into the revised manuscript PDF, with the revisions marked in blue text for ease of review.

“It is unclear how each component of the method resolves the 2D ambiguity identified as the core limitation.”

“Figure 1 is overly complex and does not convey the core idea.”

“The notation in Eq. 2 is defined across several places, and subscripts are used inconsistently (po vs. pv).”

Thank you for pointing out where our explanation was difficult to follow. By “region-to-element reliability,” we mean how appropriate a particular 2D region observation is for updating a particular 3D element. Sections 1 and 3 now explain this through four concrete decisions:

Ambiguity in the inputGAFF's corresponding decision
A projected region may be geometrically incompatible with an element.Admissibility: accept valid projections consistent with the local reference depth.
Accepted observations need not be equally informative for that element.Allocation: use depth agreement, local region-label stability, and visible-instance coverage to assign observation-specific weights.
A region descriptor may be mixed or incomplete.Anchoring: combine weighted observations with the initial semantic vector, retaining its contribution.
Strong language relevance need not give the correct object extent.OMR: use object candidates for multilevel seed selection and subsequent retention, addition, removal, or replacement.

The method reduces the influence of less suitable observations, retains the initial semantic anchor, and uses object structure to correct the output extent. These decisions directly address evidence ambiguity and incomplete object coverage.

The revised Figure 1 makes the connection between the identified ambiguity and our design explicit. Compared with Figure 1 in the original submission, the revised figure clarifies the data flow between inputs, fusion, and OMR; replaces dense weight heatmaps with mechanism schematics; connects geometric admissibility, observation weighting, and semantic anchoring; and shows how language masks and object candidates support seed selection and extent editing. The enlarged example illustrates the resulting extent correction, with IoU improving from 49.34% to 94.08%.

The indices refer to different, explicitly linked objects. We now define rpv=Rv(apv)r_{pv}=R_v(\mathbf a_{pv}) and o=(v,rpv)o=(v,r_{pv}): pvpv indexes element–view projection geometry, whereas popo indexes the corresponding element–region observation. Eqs. 2–3 consistently use popo for the weighting factors wpo,dpo,bpo,hpow_{po},d_{po},b_{po},h_{po}; the camera, depth, count, and neighborhood quantities are defined together in Section 3.2. Appendix C maps the implementation-oriented notation to these definitions. This correspondence makes the notation consistent between the main text and Appendix C. The depth difference is denoted by Δd\Delta d in Figure 1 and Δdpv\Delta d_{pv} in the equations.

The original matched-mass control in Table 4 (§4.3) already isolates the key design claim: with accepted observations, total mass, effective anchor strength, and cosine readout fixed, full allocation improves ScanNet mIoU by 0.9501/0.9883/1.3139 pp across 19/15/10 classes, without OMR. The revised explanation therefore connects the design to a measured representation-level benefit, independently of mask postprocessing.

AuthorsOpenReview ↗

Response to Reviewer 1i4y Part2

W2: Dependence on COS3D and alternative instance sources

“Both the fusion stage and OMR depend on COS3D. The authors should verify whether the method works with other instance representations. A description of COS3D on preliminaries would be beneficial.”

We have added the requested COS3D preliminaries in Section 3.1 and completed source-replacement controls in Appendix I.1, Tables S8–S10. The refinement operates on defined interfaces: fusion uses aligned instance labels for coverage counts, while OMR uses object supports and candidate extents. COS3D supplies these cues through its instance features and instance–language associations; all upstream representations remain frozen during refinement.

For fusion, replacing only the instance source and resulting hh, while fixing geometry, language, admissibility, d,bd,b, anchoring, and readout, gives:

Instance-source comparison: mIoU (%)COS3D partitionOpenSplat3D partition
3D-OVS84.362984.3731
LERF-OVS59.069559.1284

The two instance sources yield closely matching fusion scores on both datasets. This supports our hypothesis that the allocation mechanism uses instance structure through a common interface, rather than requiring COS3D-specific instance features.

For OMR on room and bed, 60 query–frame units, replacing object candidates while keeping the same seed masks across both instance sources improves 87.4787% to 93.5050% with COS3D and to 93.3194% with OpenSplat3D. We also reran source-specific seed selection: final means are 91.0663%/87.0017%, against the common language reference of 85.6195%. The OpenSplat3D route retains the common language inputs and native raw-language fallbacks. Its 1.3822 pp gain over the language reference, together with the fixed-seed result, supports object-guided refinement with the alternative source (Appendix I.1).

These experiments answer the dependency question directly: both fusion and OMR can use alternative instance inputs. GAFF contributes the way geometry and instance structure guide semantic allocation and extent refinement, beyond the choice of COS3D as an input provider.

AuthorsOpenReview ↗

Response to Reviewer 1i4y Part3

W3 & Q1: Why complete editing, how parameters were selected, and sensitivity

“OMR seems to be overly engineered.”

“How were the OMR hyperparameters selected? Was a held-out set used, and how sensitive is performance to these choices?”

Your concern is whether the editing design adds a meaningful benefit over simpler operations. To test this directly, we compared it with morphology and single-candidate alternatives on nine scenes and 363 units, retaining native seeds/candidates and excluding RGB refinement (Appendix I.2, Table S11):

Editor3D-OVS mIoU (%)LERF-OVS mIoU (%)
Retain seed91.489554.7334
Fixed morphology91.502354.7313
Top-candidate replacement82.759052.9788
Same candidate with acceptance gate85.560157.0914
Complete editing (RGB refinement disabled)94.637268.4450

With final RGB refinement disabled, complete editing exceeds the strongest tested simple alternative by 3.1349/11.3536 pp. This supports the hypothesis that object-guided candidate search and extent editing provide benefits beyond morphology and single-candidate replacement. Section 3.3 explains the common selection/editing principle and its two implementations: coverage/morphology on 3D-OVS and crop-language ranking/replacement/patching on LERF.

Selection protocol. The reported benchmark scenes served as development data; a separate held-out selection set was not used. Fusion parameters and the scoring coefficients within each route are shared, while the OMR controllers also include dataset-, scene-, and query-dependent settings. Table S12 (Appendix I.3) lists the principal values, and Appendix D describes the controllers. RGB refinement was selected through two search rounds totaling 18 configurations on five 3D-OVS scenes. We then fixed the selected configurations for the controlled comparisons.

Sensitivity evidence. Perturbing editing weights, gates, and kernel caps with seeds/candidates fixed gives 94.5112–94.6412% on 3D-OVS and 67.8466–68.4450% on LERF, with maximum absolute macro changes of 0.1260/0.5983 pp. Aggregate accuracy is stable around the selected settings. Appendix I.4, Table S13, provides the perturbation details and a complete-controller ablation, complementing the simple-editor comparisons.

Held-out evaluation. We appreciate the importance of testing frozen rules on new scenes. A complete held-out-scene experiment cannot be completed within the rebuttal period. We have prioritized the above simple-editor comparisons and parameter perturbations to provide concrete evidence on the editor’s contribution and sensitivity. These results show a clear benefit over simple alternatives and small aggregate changes around the selected settings.

Comments on the paper

1 comment

Dear reviewers,

As the rebuttal period is coming to an end, I kindly remind you to check the authors' responses. If you have not yet responded, please indicate whether your concerns have been adequately addressed. Many thanks!

Best, AC

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.