acceptodds
Withdrawn from ICLR 2027

OV-Foam: How 3D Representations Shape Semantic Feature Lifting

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Existing feature lifting approaches for open-vocabulary segmentation rely on simple backprojection to assign features to primitives. In the case of 3D Gaussian splatting reconstruction, the primitives overlap heavily, leading to suboptimal reconstructions, governed by many primitives per ray. Backprojection, however, reduces to a least-squares solvable problem, exactly when no two primitives share a ray. Motivated by this insight, we propose OV-Foam, which uses PowerFoam as the underlying representation. This representation uses a power-diagram to partition the scene, resulting in zero overlap between primitives and a substantial reduction in primitives per ray, which ultimately simplifies the lifting problem and allows for closer-to-optimal feature assignment. We show that the space-partitioning primitives of OV-Foam reach near-optimal assignment, and our experiments demonstrate improved open-vocabulary segmentation over Gaussian baselines.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
open-vocabulary 3D segmentation, 3D Gaussian splatting, power diagrams

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_YvVaOpenReview ↗
Rating
2
Confidence
3
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces OV-Foam, an open-vocabulary 3D segmentation framework based on the PowerFoam representation. Its core concept leverages spatially non-overlapping power cells to reduce the number of contributors along each ray, thereby mitigating feature mixing during the feature-lifting process. Furthermore, the paper provides a systematic analysis of PowerFoam versus 3DGS through the lens of representation-dependent feature lifting, examining aspects such as dispersion, fitting gap, and class consistency. It also introduces a purity metric to quantify the consistency of the semantic evidence received by each primitive. Experiments on datasets including ScanNet and LERF demonstrate that, while keeping upstream semantic observations and feature-lifting methods constant, replacing the underlying 3DGS representation with PowerFoam improves open-vocabulary segmentation results.

Critical strengths
  1. The paper proposes OV-Foam, which utilizes the non-overlapping representation of PowerFoam to decrease the number of contributors per ray, effectively reducing feature mixing and enhancing open-vocabulary segmentation performance.
  2. The paper offers a systematic analysis of how different 3D representations influence the feature-lifting process, explaining their impact on lifting quality through metrics such as dispersion, fitting gap, and semantic consistency.
  3. The paper introduces a purity metric to assess the consistency of semantic evidence associated with each primitive. Figure 4 demonstrates that PowerFoam yields a higher number of high-purity primitives.
Critical weaknesses
  1. The methodological innovation relative to existing work is somewhat limited. The core content in Sections 3.1 and 3.2 is primarily derived from PowerFoam and SFS. The paper’s original contributions are relatively modest.
  2. Overall, the work functions more as an analytical study on representation-dependent feature lifting rather than as a framework introducing significant new methodological content.
  3. In Figures 8 to 10, the feature PCA visualizations and query results do not show a clear or consistent qualitative difference between OV-Foam and the other baselines; consequently, the correspondence between quantitative gains and actual visual improvements remains somewhat ambiguous.
Questions for authors
  1. Much of the content in Sections 3.1 and 3.2 is derived from PowerFoam and SFS. Could the authors clarify which formulations or algorithms are genuinely novel to this work, as opposed to those that are simply direct applications or adaptations of existing methods?
  2. In Figure 4, the difference in classification accuracy between PowerFoam and 3DGS within the same purity bin is minimal. The authors should discuss the reason behind this result.
  3. How exactly is the Geometric Upper Bound in Section 4.2 calculated? Specifically, how are GT points associated with PowerFoam primitives versus 3DGS Gaussians, and do both representations utilize a strictly comparable protocol for assigning points to primitives?
Reviewer_8EhbOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes OV-Foam, an open-vocabulary 3D segmentation framework that replaces overlapping 3D Gaussian splats with PowerFoam. By eliminating heavy primitive overlap along viewing rays, PowerFoam significantly reduces semantic feature mixing and ray dispersion, leading to a more accurate one-shot feature lifting process. Through controlled experiments using ground-truth semantics and evaluations across real datasets (ScanNet and LERF) using existing lifting methods, the authors demonstrate that adopting PowerFoam consistently improves 3D open-vocabulary segmentation performance over standard Gaussian baselines.

Critical strengths
  1. Controlled Experimental Setup: The paper establishes a clean, well-isolated setup that keeps the semantic inputs (such as SAM/CLIP features) and the feature-lifting rules fixed while comparing the 3D representations. This effectively isolates the impact of the 3D geometry and primitive overlap from upstream semantic errors.
  2. Principled Investigation of Feature Mixing: By evaluating PowerFoam's bounded, non-overlapping spatial partitioning against heavy Gaussian overlap, the work provides a clear theoretical and empirical analysis of how representation-induced feature dispersion and ray contributor counts affect one-shot feature lifting.
  3. Consistent Performance Improvements: The adaptation of PowerFoam demonstrates consistent gains across multiple existing open-vocabulary segmentation methods (such as VALA, Occam's LGS, SFS, and LUDVIG) and across diverse benchmarks including ScanNet and LERF.
Critical weaknesses
  1. Lack of Representation Generality: The evaluation is restricted to comparing PowerFoam solely with 3D Gaussian Splatting (3DGS). Testing at least one additional representation—such as a voxel-based model, a mesh-based model, or 2D Gaussian Splatting (2DGS)—using the same 2D features and lifting rules is necessary to clarify whether the observed performance gains are uniquely tied to PowerFoam or broadly shared by other non-overlapping or alternative spatial representations.
  2. Reliance on Ground-Truth Initialization: While initializing both models from ScanNet's ground-truth 3D vertices helps isolate representation effects, the experiments lack validation on standard, noisy image-derived SfM/COLMAP point clouds. Rerunning reconstruction and feature lifting on estimated point clouds is needed to prove whether the performance gains persist in practical, non-ideal deployment scenarios.
  3. Missing Robustness and Sensitivity Analysis: The paper does not evaluate how gracefully the representations handle initialization errors. Introducing controlled position noise to the initial points for both methods would provide crucial insight into their respective sensitivity levels and robustness basins.
Questions for authors
  1. Evaluation on Alternative Representations: Since the claimed advantages stem from non-overlapping spatial partitioning, how do other disjoint or semi-disjoint representations (such as voxel-based models, mesh-based approaches, or 2D Gaussian Splatting) perform under this exact controlled lifting and feature setup? Are the performance gains uniquely tied to PowerFoam's formulation, or are they characteristic of any non-overlapping representation?
  2. Robustness to Real-World Initialization: The evaluation relies heavily on ground-truth vertices for initialization on ScanNet. Could the authors provide results using standard, noisy image-derived SfM/COLMAP point clouds? Rerunning a subset of scenes on estimated point clouds would clarify whether PowerFoam's bounded power cells degrade gracefully or are overly sensitive to upstream reconstruction noise in unconstrained settings.
  3. Sensitivity to Perturbation and Noise: How do PowerFoam and 3DGS compare when subjected to controlled position noise added to the initial points? Providing a sensitivity analysis regarding how performance scales across varying noise levels would offer valuable insight into the stability and robustness basins of both representations.
Reviewer_FNa7OpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper presents itself both as a method paper and as a controlled analysis. The paper analysis whether the ambiguity in 2D-to-3D feature lifting comes from the lifting procedure or from the 3D representation. It compares 3DGS with PowerFoam, a power-diagram representation whose cells do not overlap on ScanNet scenes. The authors move four existing lifting methods (VALA, Occam’s LGS, SFS, LUDVIG) onto PowerFoam and call the result OV-Foam and measure the open-vocabulary segmentation performance on ScanNet and LERF.

Critical strengths
  • This paper proposes an interesting and underexplored question. Most feature-lifting work keeps 3DGS geometry fixed and improves the lifting procedure. This paper asks whether part of the lifting ambiguity comes from the representation itself, which provides insight into what properties we want from 3D representations for accurate and robust 3D open-vocabulary understanding.
Critical weaknesses
  • The positioning of this paper is unclear, and neither framing is fully supported. As a method paper, OV-Foam applies existing lifting methods on PowerFoam instead of 3DGS, does not involve any foam-specific module for improvement, resulting in limited novelty. The comparisons cover only the 3DGS versions of the same lifters, without other open-vocabulary 3D segmentation baselines (e.g., OpenGaussian, LangSplat, Dr. Splat) and no comparison with Semantic Foam. The cost is also not discussed nor compared (reconstruction time, lifting time, memory). On LERF, gains are relatively small.

  • The controlled study does not isolate the variable it claims. The paper credits space partitioning. In its own bound (Eq. 9), the quantity that drives lifting error is how concentrated the weights are along a ray. Weight concentration can, however, also be reached on 3DGS, e.g. with per-ray top-k lifting, and this is not tested. Furthermore, the point assignment uses a dipole-aware rule only for OV-Foam, which might be unfair for 3DGS variant.

Questions for authors
  1. Cost and stronger baselines to examine the method-wise contribution. What are the reconstruction time, lifting time and memory of PowerFoam compared with 3DGS? How does OV-Foam compare with other methods e.g., OpenGaussian or Dr. Splat and with Semantic Foam on ScanNet?

  2. Fairness on point assignment. OV-Foam assigns GT points with a dipole-aware rule, but the rule for 3DGS is not described. If it is native one-one correspondence in OpenGaussian, the comparison might be unfair to 3DGS. Please report Tables 2 and 3 with the same simple rule for both representations, or give 3DGS a comparable rule.

  3. Fewer contributors vs. the representation. The analysis attributes the gain to fewer contributors per ray. Lifting methods such as Dr. Splat already reduce crowded contributors on 3DGS with per-ray top-k weights. Applying the same top-k to both representations would separate the effect of fewer contributors from other effects of the representation.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.