acceptodds
Under review as a conference paper at ICLR 2027

Splat in Place: Point-Pinned Gaussians for 3D Point Cloud Segmentation

19 views

Abstract

3D Gaussian Splatting (3DGS) provides rich geometric and appearance cues for point cloud segmentation. However, existing Gaussian-based methods rely on rendering-oriented primitives and establish point-Gaussian correspondence only after reconstruction, causing misaligned semantic evidence and limiting the use of Gaussian anisotropic support for neighborhood reasoning. To address these limitations, we introduce Splat In Place (SIP), a segmentation-oriented reconstruction framework that establishes point-Gaussian correspondence during reconstruction. SIP proposes Point Pinned Reconstruction, which initializes one Gaussian per point and preserves its identity, enabling direct access to point-aligned Gaussian attributes. Based on this representation, Joint Support Aggregation (JSA) exploits Gaussian support as a structural metric for adaptive neighbor aggregation, while Addressable Gaussian Recurrent Field (AGRF) further refines point features with Gaussian-aware relations. Experiments on ScanNet, S3DIS, and KITTI-360 demonstrate that SIP improves segmentation performance over existing methods, especially under sparse input conditions, validating the effectiveness of point-indexed Gaussian representations for 3D semantic understanding.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Point Cloud Segmentation, 3D Gaussian Splatting, Point-Gaussian Alignment, 3D Representation Learning

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_SHADOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper presents Splat In Place (SIP), a point‑pinned 3D Gaussian‑based framework for point cloud semantic segmentation. Different from existing Gaussian‑based segmentation methods which establish point‑Gaussian correspondence after reconstruction, SIP initializes one Gaussian for each input point and fixes its center/index throughout reconstruction to achieve point‑gaussian addressability.

Built upon this representation, the authors propose Joint Support Aggregation (JSA) which leverages anisotropic Gaussian spatial support to adaptively weight neighborhood aggregation, together with Addressable Gaussian Recurrent Field (AGRF) for multi‑iteration feature refinement.

Extensive evaluations are conducted on ScanNet, S3DIS, and KITTI‑360 covering both indoor and outdoor scenarios. The method shows clear advantages especially under sparse‑point input conditions. Comprehensive ablations, qualitative visualizations, and efficiency comparisons validate each core design. The paper also provides a complete AI‑use, ethics and reproducibility statement.

Critical strengths
  1. Novel point‑pinned Gaussian representation design. By locking Gaussian‑point index correspondence during reconstruction instead of post‑hoc matching, this work addresses a key pain point of existing Gaussian‑based point segmentation. The point‑gaussian addressability enables direct utilization of anisotropic Gaussian support for neighborhood reasoning, which brings clear performance gains especially for sparse point‑cloud inputs. This is a meaningful methodological advance for 3D Gaussian‑based scene understanding.

  2. Thorough, complete empirical validation. The paper conducts abundant comparisons against strong 3D point baselines and recent Gaussian‑driven SOTA methods across three public datasets. It includes sparse‑input stress tests, component‑wise ablations, boundary‑focused metrics, qualitative visualizations, feature analysis, and parameter/latency efficiency benchmark. Experiments are well‑organized and results are convincing, and appendix supplies sufficient implementation details for reproducibility.

Critical weaknesses
  1. Limited real‑world noisy‑input robustness validation. All experiments are built upon clean benchmark datasets. The paper lacks tests for practical noisy point‑cloud inputs (e.g. outlier points, mis‑registered RGB‑point pairs). It remains unclear how point‑pinned Gaussians behave under heavy input corruption, which limits confidence for real‑world deployment. This does not invalidate the main results but is a notable practical gap.

  2. Limited analysis about computational overhead brought by point‑wise Gaussian reconstruction. Though the overall inference latency is reported, the paper does not quantify the extra reconstruction cost introduced by constraining one Gaussian per raw input point. It is hard to tell how this constraint scales for extremely large‑scale scenes with millions of input points.

Questions for authors
  1. How would SIP behave if input point clouds contain severe outliers or RGB‑point mis‑alignment? Could you provide small‑scale corruption robustness results or clarifications on failure modes under noisy real‑world inputs?

  2. What is the time/memory overhead of your point‑pinned Gaussian reconstruction stage separately (decoupled from segmentation inference)? Could you comment on the scalability when processing very large scenes with millions of raw input points?

  3. In your ablation, shuffling point‑Gaussian pairing causes obvious performance drop. Is the performance drop mainly caused by mismatched spatial support, or by misaligned appearance attributes? Please clarify the primary contributing factor.

  4. The authors omitted some recent studies when comparing point-Gaussian methods; it is recommended that the following be included: “Mesh-aligned 3D Gaussian Splatting for Multi-resolution Anti-aliasing Rendering” (TCSVT'2025) and “Segment Any 3D Gaussians” (AAAI'2025).

Reviewer_VWmMOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes Splat In Place (SIP), which maintains a one-to-one Point–Gaussian correspondence during reconstruction via Point Pinned Reconstruction (PPR), and introduces Joint Support Aggregation (JSA), which uses joint Gaussian covariance for local feature aggregation. Point features are then further refined through an Addressable Gaussian Recurrent Field (AGRF). Experiments on ScanNet, S3DIS, and KITTI-360 demonstrate competitive segmentation performance, with particularly clear improvements under sparse input conditions.

Critical strengths
  • A reasonable task-oriented representation design. PPR maintains Point–Gaussian Correspondence during reconstruction, avoiding potential errors from post-hoc matching and providing a direct, explicit Gaussian representation for point-level semantic understanding.
  • Effective geometry-aware feature aggregation. JSA introduces anisotropic Gaussian support into neighborhood attention rather than simply concatenating attributes. Results and ablations across multiple datasets support its effectiveness, especially under sparse point-cloud conditions.
Critical weaknesses
  • Insufficient attribution of core contributions. The current ablations do not compare PPR with conventional Gaussian Reconstruction plus post-hoc matching under a unified architecture, nor do they replace Gaussian Reconstruction with local PCA Covariance. It is therefore difficult to distinguish the respective contributions of the Point-pinned Representation and the additional Attention/Refinement modules.
  • Modest methodological novelty. G2P already uses Gaussian covariance for anisotropic matching, while CDSeg has explored index-preserving one-to-one Gaussian construction, albeit without reconstruction optimization. Although SIP introduces a distinct reconstruction strategy and joint support aggregation, the individual components are relatively straightforward, and AGRF mainly builds on existing Attention and GRU mechanisms. The paper could better clarify its technical novelty relative to prior work.
Questions for authors

Overall, I acknowledge the contributions and the sufficient experiments in this paper, but I have some concerns:

  • Could the authors compare PPR with Free Gaussian Reconstruction + Matching, as well as local PCA Covariance, under the same JSA/AGRF architecture, to further clarify the source of the core performance gains?
  • Compared with G2P and CDSeg, what are the key technical differences of this paper in terms of Point–Gaussian Correspondence and Covariance-based Aggregation?
  • Could the authors provide the computational cost of Gaussian Reconstruction and clarify whether the efficiency comparison in Figure 5 includes this stage?
Reviewer_GQXiOpenReview ↗
Rating
2
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces SIP, a point cloud segmentation framework. SIP takes a 3D point cloud and multi-view images as inputs, and optimizes a 3D Gaussian scene with one Gaussian pinned to each point (hence the name "splat in place"). The reconstructed 3D Gaussians are used to provide more information, via neighbor aggregation and recurrent refinement, for point cloud segmentation.

Critical strengths
  1. The point-pinned Gaussian makes sense. After all, the multi-view images and local reconstructed geometries can provide direct information about the pinned point. This is direct and simple. Good.
  2. The Gaussian neighborhood aggregation (i.e., the "anisotropic support") is also reasonable.
Critical weaknesses

I have one main concern here: inconsistency between the training and evaluation protocols, and inconsistency between the paper and the code, on how the Gaussians are updated.

Inconsistency in the paper, between training and evaluation:

  • During training, we build per-scene optimized Gaussians, and then the Gaussians are updated with the segmentation loss. Lines 153–155 say "during segmentation training, the point-level loss directly updates the reconstructed scale, rotation, opacity, and spherical harmonic coefficients."
  • At evaluation, the inputs are per-scene optimized Gaussians, with no other updates at all. Lines 699–700 say "at evaluation, each scene is reconstructed from its RGB views and processed without segmentation gradient updates."

Inconsistency between the paper and the supplementary code:

  • The GaussianTaskField class in the code is supposed to be the Addressable Gaussian Recurrent Field (AGRF) in the paper, as they both contain the 3 GRU steps. In the code, GaussianTaskField is used to predict Gaussian parameter (scale/rotation/opacity/etc.) updates, while in the paper, AGRF only updates the point features (Eq. 5–6).
  • Unlike the paper, the training and evaluation settings of the code version are consistent, as the Gaussian updates are applied in both the training and evaluation stages.

My take on this:

  1. The code version makes more sense than the paper version. A feed-forward prediction of "Gaussian parameter delta" can be applied in both the training and evaluation stages, and it is what really enables test-time Gaussian refinement. This is what can really address the argument in Line 74 that "Gaussians optimized for rendering are not yet Gaussians prepared for segmentation."
  2. The authors should include an ablation study on disabling the Gaussian update step during the training recipe.
  3. In Figure 1, the reconstructed point clouds in (a) and (b) only give the pseudo results of the reconstructed Gaussians with or without the predicted update. We should add a dedicated figure, showing the real before-and-after Gaussian parameter update visualization. That is, after converting "Gaussians optimized for rendering" to "Gaussians prepared for segmentation," how would the reconstruction and rendering quality be affected?
Questions for authors

Could you explain the potential inconsistency between the train/test Gaussian updates in the paper, and the inconsistency between the paper and the code as decribed in the Weakness section? If I missed some point, please kindly let me know.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.