- Rating
- 1
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
4DSuperG targets open-vocabulary object query in dynamic scenes. It starts with motion-based grouping, uses temporary per-Gaussian encodings to refine associations. Decoded object IDs retrieve full-dimensional CLIP feature. Text queries select canonical primitives whose learned deformation carries the identity across time. Experiments on HyperNeRF and Neu3D report improved primitive-level segmentation, with additional zero-shot language-driven spatio-temporal editing demonstrations.
Critical strengths
None
Critical weaknesses
- This work is an incremental combination of existing approaches: motion grouping comes from SP-GS; identity-based CLIP feature assignment comes from Segment-then-Splat and OpenGaussian; and Super-Gaussians have been explored in SuperGSeg. I do not believe this work offers meaningful insights or contributions to the task of open-vocabulary querying with 3DGS. In my view, it does not meet the bar for acceptance at ICLR.
- The paper is poorly written, making it difficult to understand, assess, and follow the work. I turned to the Method section for answers to my questions raised while reading the abstract and introduction, but found that it reads like a list of implementation steps without adequately justifying the design choices. Important relationships between modules need to be made explicit in the text.
Questions for authors
What is the principal advance beyond the cited components, and which experimental comparison directly isolates and supports that advance?
3 comments
I thank the authors for your response, but my rating is still unchanged. Segment-and-Splat already assigns CLIP embedding IDs to Gaussians in open-vocabulay query in dynamic scenes. However, there is no experimental comparsions with Segment-then-Splat. The remaining novelty narrows to learned soft Gaussian membership from motion and identity, and the paper should be presented that way. The paper's ablation shows this design adds a little: (g)→(e), reduces mIoU by 2.69 and 0.56, respectively, while removing per-Gaussian bootstrap drops to 50.79 in Neu3D which is below SA4D. I therefore keep my original rating.
Response to Reviewer ctLK: Segment then Splat comparison, and the two identity stages
Thank you for these specific points. We are happy to address them.
Regarding comparison with Segment-Then-Splat
We did run it before the submission, and excluded it for a reason we should have stated in the paper. We add a video showing why its design does not work for 4D primitive-level open-vocabulary segmentation to the revised supplementray materials.
In short: primitive-level querying is only useful if the selected primitives keep the object through occlusion, and in 4D Segment then Splat does the opposite: its object loss deforms that geometry away to match the occlusion-truncated mask.
Its object loss, Eq. (6), supervises the isolated rendering of each object’s Gaussians against the ground-truth image masked by that object’s 2D mask: . Where the object is occluded, the mask has a hole, so the object’s own Gaussians are penalised for occupying that region. In static 3D this is largely corrected by other views. In 4D it is not: the deformation network predicts a per-timestep deformation from canonical, so the geometry is free to be carved to the occlusion-truncated silhouette at each t. The supplementary video shows this: an object reconstructed by Segment then Splat with a concave notch where the occluder sits, which persists across viewing angles and is therefore geometry rather than a rendering artifact. We find its design against the motivation of primitive-level segmentation protocol therefore excluding it.
In addition, in its paper it reports result 69.48 and 44.00 mIoU on HyperNeRF and Neu3D in a 3D querying protocol described as “directly retrieving the Gaussians associated with an object” (Sec. 4.2), while our primitive-level results are 79.10 and 76.07. We report its numbers as published, the scene and query sets are not specified there and may differ, but it is still a large margin here and ours is obtained while actually retaining the object under occlusion, as shown in Appendix F and the reply to reviewer e3nb.
Regarding the ablation and the two identity stages
In our design, identity acts on the grouping through two stages: the per-Gaussian identity bootstrap (IB), stage one, and the shared SuperG identity refinement (SIR), stage two.
If we understand the reviews correctly, the reviewer read the table as: (g)→(e): removing SIR causes only a small drop (g)→(f): removing IB causes a large one and the gain is therefore attributed to IB, which the reviewer think that taken to be SA4D’s contribution rather than ours.
We believe this rests on a misreading of stage one. On the mechanism: SA4D learns a per-Gaussian, time-varying identity field, and has no grouping for identity to act on. In our bootstrap the identity losses act on the association logits themselves, and the per-Gaussian encodings are temporary and discarded once the shared encodings are initialised from them (Sec. 3.2, Appendix B.4). Stage one is where membership first becomes identity-aware: it is the first half of the proposed design, not a borrowed component.
The table also disapprove this. If stage one were equivalent to SA4D’s carrier, case (e), which has stage one but not stage two, should match SA4D under the same lookup. Instead (e) reaches 70.54 and 74.75 against 63.06 and 59.41, i.e. +7.48 and +15.34. With both stages, (g) is +10.17 and +15.90.
What the table shows, then, is that identity learning is more effective on the SuperG than on the individual Gaussian. Fig. 3 and the amodal evaluation in Appendix F show the same for occlusion robustness against SA4D, and Appendix D reports substantially lower training and query cost. The learned identities also carry other 2D object-level features without retraining. These are further advantages over SA4D.
A clarification on Super-Gaussians, here and in prior work : “Super-Gaussian” is a name for a group of Gaussians, taken from superpoints, and sharing the name does not mean sharing the mechanism. We hope this does not hinder our novelty.
- SuperGSeg forms static groups from geometric and segmentation cues and distils language features onto them.
- SP-GS groups by motion alone, so a group may cross object boundaries, which makes it unsuitable as a unit for segmentation (Fig. 6c).
Ours is formed from motion and identity jointly, which is what makes it usable as a query unit in 4D, and what yields the state-of-the-art primitive-level results together with the efficiency, occlusion robustness and reusability above.
We hope this addresses the concern, and would be grateful if you would reconsider the assessment.
Response to Reviewer ctLK
We thank the reviewer for the review and answer the question directly below. Line numbers refer to the revised PDF, and all cited passages are unchanged from the submission.
Answer: The principal advance is 4DSuperG, a representation of identity-aware Super-Gaussians whose grouping is learned jointly from motion and identity, so that the unit of identity learning is also the unit of motion in dynamic scenes. This representation yields state-of-the-art primitive-level open-vocabulary 4D segmentation (Table 1), robustness to occlusion (Fig. 3, Table 11, Fig. 12), a reusable carrier for object-level features (Fig. 4), zero-shot language-driven spatio-temporal editing (Fig. 5), and efficient training and querying (Appendix D). The controlled comparison with SA4D (Table 1) isolates the contribution of the 4DSuperG carrier, and the identity-stage ablations (Table 2) that of identity-reshaped grouping.
Principal advance (W1, Question). W1 frames the task as open-vocabulary querying with 3DGS, but our paper targets dynamic 4D scenes, with complex motion and occlusion. Our insight is that, in dynamic scenes, open-vocabulary querying requires the selected object to retain its identity as it moves and through occlusion (L031–032), so the representation cannot be designed apart from the dynamics: the unit of identity learning must also be the unit of motion (L090–091). Sharing identity and motion within a group pools evidence across members and time, so visible members constrain occluded ones (L079–082). Each group carries one time-invariant identity, which a query selects once in canonical space. The paper already cites every listed work and distinguishes 4DSuperG from each:
None of these works couples semantics and motion in the same unit, and putting their components together does not produce one: motion-only groups merge co-moving objects, and identities attached afterwards cannot split them. Making identity reshape the grouping requires the two-stage design of Sec. 3.2: per-Gaussian identity bootstrapping, then shared identity refinement.
Experiments that support this advance (Question).
Writing (W2). The other reviewers found the paper clear and easy to follow. The overview of Sec. 3 states how the modules connect (L163–168), and each design choice is motivated where it is introduced: shared motion because per-Gaussian deformation fragments a selection (L172–173), identity because motion alone merges co-moving objects (L222), and per-Gaussian bootstrapping because shared identity learning and association refinement are interdependent (L227–229).
We hope this clarifies our contributions, and that the reviewer will reconsider the assessment in light of the comparisons above.