acceptodds
Under review as a conference paper at ICLR 2027

Learning Shared Object Prototypes for Compositional 3D Gaussian Splatting

Abstract

Existing 3D Gaussian Splatting (3DGS) methods typically reconstruct each scene independently, entangling recurring objects with their original scene configurations and hindering their direct reuse and recombination. We present a prototype-based compositional 3DGS representation that learns globally shared Gaussian prototypes across multiple scenes. Given posed multi-view images with externally provided object masks and category labels, our method learns a canonical prototype for each recurring object category, while representing each object instance with a lightweight instance-specific similarity transformation. Each scene is then constructed by instantiating and composing the shared prototypes according to its scene-specific configuration. To address the strong coupling between prototype parameters and instance transformations, we introduce an alternating optimization strategy with a fixed canonical reference and prototype regularization. Experiments on GSO and OCTScenes show that our learned prototypes capture the geometry and appearance shared by recurring objects, maintain high-quality novel-view synthesis, and enable the instantiation and recombination of objects to construct new scene configurations.

Primary area:
unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords:
3D Gaussian Splatting, Object-Centric 3D Representation, Compositional Scene Representation, Shared Object Prototypes

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_jijQOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes a prototype-based compositional 3D Gaussian Splatting representation that learns globally shared Gaussian prototypes for recurring object categories across multiple scenes. Instead of reconstructing each object independently, the method represents each object instance using a shared canonical prototype together with an instance-specific similarity transformation (rotation, translation, and scale). To address the ambiguity between prototype geometry and instance transformations, the authors propose an alternating optimization strategy with a fixed canonical reference and prototype regularization. The learned prototypes can be reused across scenes and recombined into novel scene configurations while substantially reducing representation redundancy.

Critical strengths

The problem is interesting and well motivated. Rather than treating every scene independently, the paper explores learning shared object-level Gaussian representations across multiple scenes, which is a natural direction toward reusable and compositional 3D scene representations.

Critical weaknesses
  • The method relies on several strong assumptions that limit its applicability. It assumes that instances of the same category share canonical geometry and appearance and differ only in pose and scale. All scenes additionally share the same static background. These assumptions substantially simplify the problem and restrict the method to relatively controlled scenarios.
  • The experimental evaluation is relatively limited. The paper is evaluated only on GSO and OCTScenes, both of which are specifically selected to contain repeated object instances. There is little evidence that the proposed representation would generalize to more diverse real-world datasets or more challenging multi-scene settings.
  • The comparisons are not entirely convincing. The baselines are primarily scene-specific 3DGS methods, while comparisons with recent compositional, object-centric, or multi-scene representation learning approaches are limited. As a result, it is difficult to assess how much of the improvement comes from the proposed prototype representation itself versus the favorable problem setting.
Questions for authors

No

Reviewer_3yS4OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes a compositional 3D Gaussian Splatting (3DGS) representation for scenes containing numerous repeated objects. Given multi-view images, instance masks, and object identifiers, the method learns shared prototypes and represents individual object instances through compositional transformations. In addition, a shared background prototype is introduced to complete the scene representation. Preliminary experimental results demonstrate the advantages of the proposed method.

Critical strengths
  1. This paper proposes a clear and effective method for sharing object representations across different scenes.

  2. The experimental results demonstrate significant storage compression, achieving compression ratios of 19× and 16× compared to previous work and the original datasets, respectively.

Critical weaknesses
  1. Missing Evaluation of Novel Scene Composition: The paper presents visualizations of existing scenes from held-out viewpoints, as well as individual visualizations of object prototypes (Figures 2 and 3). However, it does not provide experimental results for new scenes constructed by recombining these prototypes. Since constructing novel scene layouts is claimed as one of the contributions, the experiments should evaluate the quality and effectiveness of such recomposed scenes. The current results only demonstrate the reconstruction of existing scenes, lacking experimental evidence on the quality of new scenes generated through object rearrangement and recomposition.

  2. Limited Implementation Details: Key implementation details, such as object pose initialization and object placement and alignment within the scene, are insufficiently described. The constraints applied during 3D reconstruction are not clearly explained.

  3. Limited Performance: As shown in Figure 3, the reconstructed objects appear relatively coarse and lack fine details.

  4. Limited Novelty and Insight: Although the proposed method achieves some performance improvements, the paper appears more like a combination of engineering techniques and lacks novel insights into the field. The authors are encouraged to clarify the technical differences between their approach and existing works.

Questions for authors
  1. Performance Gains from Cross-Scene Prototypes: To what extent do the cross-scene prototypes proposed in this paper improve performance? The method initializes shared prototypes by reconstructing individual instances and subsequently optimizes these prototypes across multiple scenes. Under the same experimental settings, how much does the full method improve reconstruction quality and rendering performance compared with a baseline that only optimizes the transformations of the initialized prototypes for each individual scene? Could the authors provide comparative results to evaluate the performance gains of using cross-scene prototypes over directly reusing reference object models?

  2. Handling Appearance Variations: The paper shares spherical harmonic appearance parameters across corresponding object instances and transforms viewing directions into a canonical coordinate system. However, the same object may exhibit different brightness, glossiness, and surface colors when rotated or placed under different lighting conditions. How can a shared canonical representation account for such appearance variations? How does the proposed method handle these differences across object instances and scenes?

Reviewer_VWnvOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes a prototype-based compositional representation for multi-scene 3DGS. Given posed multi-view images together with instance masks and cross-scene object identities from an external model (GOLD), it learns one shared canonical Gaussian prototype per recurring object and a shared background prototype, and places each instance via an instance-specific similarity transformation. Prototypes are initialized from manually selected reference instances and trained by alternating between optimizing instance transformations and shared prototypes, with the reference transformation frozen to fix the canonical frame and a proximal regularizer to prevent prototype drift. Experiments on 20 scenes each from GSO and OCTScenes compare storage and held-out-view quality against 3DGS, Gaussian Grouping, and ObjectGS, along with prototype turntable renderings and ablations on OCTScenes.

Critical strengths

The problem setting is clear and reasonable: decoupling object content from scene configuration so that recurring objects share one explicit Gaussian representation across scenes. The method is simple, and implementation details such as scene registration, Gaussian decomposition, and reference selection are described in reasonable detail. Under the setting of identical recurring objects and an identical background, it substantially reduces total storage (about 16–19×).

Critical weaknesses

First, the central claim lacks evidence. The paper’s main selling point is that prototypes can be reused and recombined across scenes to build new configurations, yet the main paper shows no composition or editing results; Fig. 3 only shows turntable renderings of single prototypes. Since GSO is synthetic, ground truth for novel layouts could have been rendered for quantitative evaluation, but this was not done. The storage savings also appear misattributed: all scenes share one background stored once, and with only 3–6 small objects per scene, background sharing alone could explain the ~19× compression. Yet the paper credits the object prototypes and provides neither a storage breakdown nor a shared-background baseline.

Second, technical novelty is limited and the experimental support is weak. The method is essentially classical instancing applied to 3DGS, with standard block-coordinate descent and gauge fixing. Prototypes are updated with learning rates of 1e-5 under a proximal regularizer and are likely near-identical to the reference instances. Rendering quality is uniformly below ObjectGS, and LPIPS on OCTScenes is the worst among all methods, contradicting the “high-quality” claim. Ablation differences are small, no variance is reported, and claims about drift and floaters are not backed by figures or numbers.

Questions for authors

Clarification 1: How is the initial relative rotation R_{s,i} in Eq. (14) obtained? Does it use ground-truth poses or manual alignment? Does rotation optimization fail for symmetric objects? Clarification 2: What is the storage breakdown among the background prototype, foreground prototypes, and instance parameters? How does a per-scene 3DGS with shared background baseline perform? Clarification 3: Are prototypes densified or pruned? How much do learned prototypes change from their initialization? Can regions unobserved in the reference instance be completed by other instances? Clarification 4: Does the “Joint optimization” variant also freeze the reference instance and use the same learning rates and steps? Since reference freezing already removes the global gauge ambiguity, what exactly does alternation address? Clarification 5: In OCTScenes, are instances of the same category the same physical object? Suggestion 1: Add quantitative evaluation of recomposed scenes on GSO, as well as pose errors and prototype geometry errors (e.g., Chamfer distance). Suggestion 2: Compare under a matched training budget, report training and rendering cost, and compare storage against 3DGS compression methods. Suggestion 3: Test prototype reuse on unseen scenes, e.g., by fitting only instance transformations in a new scene.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.