- Rating
- 1
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
The paper proposes Partner, a method for interactive repartitioning of 3D point clouds. It takes as input an initial segmentation representing object parts and provides an updated part segmentation given user feedback. It allows the user to provide positive/negative prompts to refine a selected part. The user can only provide feedback to one part segmentation at a time, but the method jointly readjusts all part masks. The resulting masks are mutually exclusive and combined cover the entire 3D shape. The approach follows a 2-stage pipeline. First, an interactive model refines a single mask, and then a holistic module refines the part masks given the initial segmentation and the refined single mask. Additional point-wise features are used from a PointTransformer V3 backbone to produce the final result.
Critical strengths
- Interactive segmentation of 3D data is an important problem for 3D annotation and 3D asset editing.
- The ability to refine the entire segmentation through corrections to individual parts can be useful, but most interactive object segmentation methods have this attribute inherently. They update multiple object masks jointly, as in AGILE3D or object/background masks. In fact, the challenge, especially for editing, is often the opposite: preventing unintended changes to other parts when correcting a single part.
Critical weaknesses
- Comparison with relevant interactive 3D segmentation baselines, such as AGILE3D and NPISeg3D, is missing. The paper argues that methods such as AGILE3D require clicks on every object part, but differences in the interaction protocol do not justify excluding them. What matters is the segmentation quality (IoU) as a function of the number of user interventions. AGILE3D and NPISeg3D support joint updates of multiple object masks. After all, the proposed method also reports the number of parts refined. These methods could also be adapted to use automatically generated initial segmentations or sampled clicks from SAM, as in the proposed approach. Such comparisons are important given the relatively weak results on ScanNet, the most challenging dataset in the evaluation.
- Evaluation protocol. The metric in Eq. 6 is not the standard in the community that evaluates IOU on a stricter 1-1 matching. It leaves over-segmentation of parts without penalty and uses only the best match. It also allows the same predicted part to be matched in multiple ground truths.
- ScanNet performance. The performance on ScanNet is rather weak despite the pre-trained backbone on ScanNet, with only 42.5mIoU after refining 10 parts with 5 corrections per part. The IoU@T, where T is the number of parts, is confusing given that the standard in the community is to report IoU@UserFeedback
- Dependence on initial partition.
- The methodology is difficult to follow, and methodological choices are not sufficiently motivated.
- The protocol is rather restrictive, the user is allowed to focus on one part at a time and hope the corrections will influence remaining segmented parts positively.
Liu, J., Zhou, P., Xiao, Z., Shen, J., Yin, W., Sonke, J.-J., & Gavves, E. (2025). Probabilistic Interactive 3D Segmentation with Hierarchical Neural Processes (NPISeg3D). Proceedings of the 42nd International Conference on Machine Learning (ICML 2025)
Questions for authors
- Comparison with additional baselines is important to conclude which methods are more effective and should be preferred for interactive 3D segmentation and editing tasks.
- Clarification of the chosen evaluation metric and evaluation under stricter metrics to account for incorrect part assignments and the number of predicted parts.
Reply to reviewer 863p
Thank you for recognizing the importance of interactive 3D segmentation for annotation and asset editing, and the potential usefulness of refining the entire segmentation through corrections to individual parts. We address your specific concerns below, referring to the shared response where appropriate.
(W1-Q1) Regarding your assessment that joint mask prediction is not new and your question about method effectiveness: As clarified in C1, Partner's contribution is interactive editing within a holistic segmentation context, rather than segmentation from scratch or joint mask prediction itself. The reported same-source pipeline improves mIoU@10 by 5.5–18.2 points across four datasets. We will describe Partner's contribution more precisely in terms of this editing task.
(W1-Q1) Regarding missing AGILE3D/NPISeg3D comparisons and adaptation through automatic holistic segmentation results or sampled clicks: C6 details why conversion of holistic segmentation results to initial automatic prompts, background handling, and dynamic-target management require explicit rules. We believe that the measured performance differences would depend on several adaptation parameters, whose effects may also vary across datasets. This makes a fair comparison difficult.
(W2-Q2) Regarding matching direction, repeated matches, and over-segmentation penalties: C2 gives the GT-to-prediction definition and explains why fragmentation can lower best-match IoU while repeated matches remain allowed. Best-overlap and one-to-one metrics share the overall evaluation objective and provide complementary details. We will consider adding clearly defined stricter matching and part-count evaluation.
(W3.1) Regarding your assessment of limited ScanNet performance: Removing Stage 1 reduces mIoU@10 from 74.6 to 63.5 on PartObj-Tiny and from 81.7 to 75.0 on PartNeXt, supporting primary-mask refinement. These ablations do not isolate the cause of the ScanNet result. As shown in C5, using PointSAM initialization improves Partner's ScanNet result to 50.0, compared with 45.9 for PointSAM with heuristic refinement from the same initial value of 39.6. This demonstrates benefit from a stronger initial holistic segmentation result produced by another method, not a tested replacement of Stage 1.
(W3.2) Regarding reporting refinement rounds rather than actual user feedback: C3 clarifies that T counts selection/refinement rounds, including repeated selections, rather than clicks or distinct parts. The current protocol compares refinement efficiency through the rounds needed to reach the same quality. We will consider adding cumulative actual-click distributions for each method at each reported checkpoint to make feedback costs more informative and intuitive.
(W4) Regarding dependence on an initial holistic segmentation result: As explained in C1, an existing holistic segmentation result is the task's starting point, not an incidental requirement. Compatibility with initial holistic segmentation results from other methods lets Partner benefit from strong automatic segmentation, as illustrated in C5. This compatibility allows the same editing framework to use initial holistic segmentation results from different sources and should not be viewed as a weakness of the method.
(W5) Regarding your assessment that the method is difficult to follow and insufficiently motivated: Thank you for this feedback. We would appreciate clarification of the specific passages or design choices you find unclear, so that we can address them directly and improve the presentation in the next version.
(W6) Regarding the single-part interaction protocol: Selecting one primary part makes the meaning of positive and negative prompts explicit and facilitates evaluation and comparison. In a multi-label interface, each click also specifies one part label, rather than the desired holistic segmentation result. Partner uses such local feedback to jointly update the holistic segmentation result. Multi-label methods can accumulate feedback for several parts within a round, so the protocols are not identical. Unintended changes to non-target parts are a separate evaluation issue addressed in C2.