- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
GAFF refines existing language representations by weighting region observations according to geometry, local region agreement, and instance compatibility. An anchored update combines these observations with the original features, after which object-guided mask refinement selects and edits query masks. Experiments cover rendered segmentation and point classification, including mass-matched controls intended to isolate relative evidence allocation.
Critical strengths
- The mass-matched uniform control addresses an important confound in weighted aggregation.
- Fusion and mask editing are evaluated separately as well as together.
- Scene-level diagnostics include failures and harmful prediction changes.
- Refinement reuses fixed representations without additional gradient-based optimization.
Critical weaknesses
- The contribution of the geometric weighting factors is uncertain. Beyond COS3D’s instance–language integration, matched weighting adds 0.95 ScanNet mIoU, but instance weighting alone essentially matches the full product in Table S4. Mass-matched factor ablations are needed to establish what depth and local agreement add.
- Most rendered-mask gains depend on an elaborate editor whose transfer is untested. OMR alone improves LERF mIoU by 6.60 points, compared with 1.23 for fusion alone. Document how its dataset-specific gates were selected and test frozen rules on held-out scenes.
- External comparisons retain incompatible evaluation conditions. Appendix F acknowledges different ScanNet point-filtering domains and different cross-view prediction constraints on LERF. Reevaluate principal baselines under shared conditions and measure cross-view consistency explicitly.
[1] COS3D: Collaborative Open-Vocabulary 3D Segmentation
Questions for authors
- What development data and selection procedure determined the OMR thresholds?
- Can the weighting reject mixed-object observations when instance compatibility measures coverage rather than region purity?
5 comments
Response to Reviewer uau6 Part2
W2 & Q1: Development data, thresholds, and frozen-rule evaluation
“Document how its dataset-specific gates were selected and test frozen rules on held-out scenes.”
“What development data and selection procedure determined the OMR thresholds?”
Selection protocol. The reported benchmark scenes served as development data; a separate held-out selection set was not used. Fusion parameters and the scoring coefficients within each route are shared, while the OMR controllers also include dataset-, scene-, and query-dependent settings. Table S12 (Appendix I.3) lists the principal values, and Appendix D describes the controllers. RGB refinement was selected through two search rounds totaling 18 configurations on five 3D-OVS scenes. We then fixed the selected configurations for the controlled comparisons.
To test the practical concerns behind the question—whether the editor adds value and whether that value depends on precise parameter choices—we completed two complementary experiments. Fixed-seed/candidate comparisons show that complete editing exceeds the strongest tested simple alternative by 3.1349/11.3536 pp on 3D-OVS/LERF, excluding RGB refinement. Perturbing editing parameters gives 94.5112–94.6412% / 67.8466–68.4450%, with maximum absolute macro changes of 0.1260/0.5983 pp. Appendices I.2–I.4 provide the parameter settings, perturbations, and complete-controller ablation. These results quantify the complete editor’s benefit and its sensitivity around the selected settings.
Regarding the requested held-out-scene test: we appreciate its value for assessing transfer to new scenes. A complete evaluation with frozen rules cannot be completed within the rebuttal period. We have prioritized the above editing comparisons and sensitivity experiments (Tables S11–S13) to provide concrete evidence within the available time. They establish the editor’s measured benefit and local parameter sensitivity; the requested held-out transfer test remains a separate evaluation.
Response to Reviewer uau6 Part3
W3: Shared scoring of principal baselines and cross-view consistency
“External comparisons retain incompatible evaluation conditions.”
“Reevaluate principal baselines under shared conditions and measure cross-view consistency explicitly.”
Shared scoring of principal baselines. Following your suggestion, we rescored the available DrSplat, COS3D, and GAFF point predictions under a common protocol (Appendix J.3, Tables S18–S19). ScanNet uses the same valid GT points, class mapping, shared text assets, and scene-macro aggregation; unknown predictions count as errors.
| Method | 19-class mIoU (%) | 15-class mIoU (%) | 10-class mIoU (%) |
|---|---|---|---|
| DrSplat (local, rescored; shared text) | 36.2934 | 36.8108 | 44.7791 |
| COS3D (local, rescored; shared text) | 33.4675 | 32.8591 | 41.1075 |
| GAFF with cosine readout () | 37.6899 | 37.0852 | 45.6601 |
| Matched Uniform | 36.7398 | 36.0969 | 44.3463 |
GAFF with cosine readout leads the listed local comparisons across all three class settings. The appendix additionally evaluates every method on the same opacity-filtered domain. DrSplat uses its fixed 4NN interpolation adapter, and GAFF uses the cosine-readout configuration of Appendix J.1.
For LERF, the same units, GT rasterization, grid resizing, and aggregation give 53.7917% for locally rescored LangSplatV2 and 68.4450% for OMR across four scenes. On figurines, locally rescored COS3D obtains 56.8657%, versus 63.8688% for OMR. Table S20 (Appendix J.3) reports both domains explicitly.
These comparisons show GAFF’s prediction benefit when the scoring domain and aggregation are shared, directly strengthening the benchmark evidence.
Cross-view consistency.
We completed the explicit consistency measurement you requested on native seed, morphology, and OMR outputs (Appendix J.2, Tables S16–S17), using reciprocal COLMAP tracks with fixed geometric filters.
OMR achieves 68.4450% full-set 2D mIoU, compared with 54.7334% for the seed, and 76.2115% paired foreground consistency on 40 queries and 203 query–view pairs (Table S17). The paired seed consistency is 78.7467%. Among the 40 queries, 26 retain agreement and four improve; sampled-label disagreement is 1.9401%. Appendix J.2 provides the complete seed/morphology/OMR comparison and the metric definitions. The diagnostic measures sparse tracked surfaces, with 0.00990% mean sampled-pixel coverage.
The results quantify substantial foreground agreement together with more accurate query-mask extent. Fusion’s representation-level gains are established separately through the matched-mass point-prediction controls. This gives a direct evaluation of both the semantic allocation and output refinement contributions.
Response to Reviewer uau6 Part1
W1: Contributions of depth and local agreement
We thank you for recognizing the mass-matched control, separate and combined component evaluation, and refinement of fixed representations without additional gradient-based optimization. Your suggestions focus on identifying the effective weighting cues, validating the editor, and making comparisons more informative. We completed the factor ablations, parameter sensitivity, shared scoring of local baselines, cross-view measurements, and mixed-region analysis. We have incorporated the corresponding clarifications and additional results into the revised manuscript PDF, with the revisions marked in blue text for ease of review.
The new ablation shows that coverage provides the main gain, depth adds a smaller improvement, and local agreement makes a modest adjustment to incoming evidence. We completed an eight-configuration matched-mass ablation on ten ScanNet scenes, 1,158 views, 1,569,975 points, and 18,204,562 accepted associations (Appendix J.1, Tables S14–S15). For a factor subset S, we use
\qquad M_p=\sum_o d_{po}b_{po}h_{po},$$ where $q^{(S)}$ is the factor product, or 1 for Uniform. Accepted observations, initial fields, total mass, effective anchor strength, and cosine readout are fixed; OMR and optimized readout components are disabled. | Factors | 19-class mIoU (%) | 15-class mIoU (%) | 10-class mIoU (%) | | --- | ---: | ---: | ---: | | Uniform | 36.7398 | 36.0969 | 44.3463 | | $d$ | 36.8304 | 36.2013 | 44.5041 | | $b$ | 36.7453 | 36.0942 | 44.3585 | | $h$ | 37.6827 | 37.0532 | 45.6116 | | $dh$ | **37.7081** | **37.1001** | **45.6808** | | $bh$ | 37.6666 | 37.0375 | 45.5923 | | $dbh$ | 37.6899 | 37.0852 | 45.6601 | | $db$ | 36.8237 | 36.1879 | 44.4944 | **The full product preserves the principal allocation gain, with a very small difference from $dh$.** Coverage contributes **0.9429/0.9564/1.2654 pp** over matched Uniform, and adding depth gives **0.0254/0.0469/0.0691 pp**. The advantage of $dh$ over $dbh$ is **0.0182/0.0149/0.0206 pp**, approximately 0.02 pp at most. Full $dbh$ improves over matched Uniform by **0.9501/0.9883/1.3139 pp**. **Why retain $b$?** It supplies local information complementary to depth and instance coverage. From its definition, $b=1$ when all sampled neighbors agree with the projected pixel’s region label; it decreases toward its floor as agreement falls. Thus, even when depth and instance coverage are similar, the weight can respond to local region-boundary ambiguity. This is the design reason for including $b$ in the joint allocation rule. The completed mixed-region diagnostic provides an observation-level check. Across low, medium, and high semantic-purity groups, weighted region-label correctness increases from **18.7310/36.5651/59.2220% with $dh$ to 18.7614/36.5952/59.2552% with $dbh$**. The mean $b$ values of **0.9752/0.9831/0.9853** are consistent with a modest local adjustment. We therefore retain $b$ to encode local association stability alongside surface compatibility and instance coverage, while preserving the full eight-configuration comparison in Appendix J.1. The final point labels come from the argmax after multiview accumulation and anchoring, explaining why observation-level changes need not produce identical changes in final accuracy.