acceptodds
Under review as a conference paper at ICLR 2027

Organize Primitives into Semantic Parts: Reinforcement Reasoning for 3D Segmentation

Abstract

Primitive-based 3D segmentation offers a compact and explicit alternative to dense surface prediction, naturally supporting structural abstraction and boundary localization. However, geometric decomposition alone does not determine how primitives should be organized into semantic parts: a single part may span multiple primitives, while geometrically similar or touching primitives may belong to different parts. We therefore introduce RePart (Reinforcement Part Reasoning), which formulates primitive-to-part organization as a finite-horizon Markov decision process and learns semantic organization through trajectory-level reinforcement reasoning. RePart constructs a Composable Primitive Workspace from fine-grained superquadrics and applies a merge-and-stop policy whose decisions are optimized by their downstream effects on the resulting partition rather than local primitive compatibility. The inferred part identities are then mapped back to the original mesh through Boundary-Aware Surface Labeling, preserving accurate surface boundaries beyond the primitive approximation. On PartNet, RePart achieves the strongest results across all four aggregate partition metrics; on 3DCoMPaT++, it obtains the highest RI and SC without target-dataset fine-tuning. These results demonstrate that reinforcement reasoning provides an effective mechanism for organizing geometric primitives into semantic parts while retaining dense segmentation accuracy. Code is available at https://anonymous.4open.science/r/RePart-F01E/https://anonymous.4open.science/r/RePart-F01E.

Primary area:
learning on graphs and other geometries & topologies
Keywords:
3D part segmentation, reinforcement learning

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_1XayOpenReview ↗
Rating
2(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

This paper proposed Repart method for 3D segmentation. RePart is a 3-stage method: 1. fit 3D meshes with geometric primitives; 2. group geometric primitives into semantic parts; 3. project the part labels back to the original mesh. The grouping process is formulated as a Markov decision process. At each step, RePart decides whether to merge two primitive groups or stop. The decision policy is optimized by reinforcement learning, using rewards based on partition overlap and the number of predicted parts. Experiments demonstrate the effectiveness of the proposed method.

Critical strengths
  • 3D mesh segmentation is an important research topic.
  • The approach of fitting geometric primitives and then merging them to obtain a mesh segmentation is interesting and offers a novel perspective on part segmentation.
Critical weaknesses
  • The experimental evaluation is limited. The training and testing datasets are relatively simple. Recent part segmentation methods are often trained and evaluated on HY3D-Bench and PartObjaverse, whereas this paper reports results only on PartNet and 3DCoMPaT++.
  • The segmentation depends heavily on the initial primitive fitting. Once a primitive is fitted incorrectly, the merge-only procedure cannot correct it. Because primitive fitting relies on geometry rather than semantic information, such errors may be common when geometric and semantic boundaries do not align.
  • Geometric primitives may be less suitable for complex shapes. Structures with intricate openings or rich surface detail can be difficult to approximate with simple primitives. A poor or fragmented fit could make the subsequent grouping process less reliable.
Questions for authors
  • How does RePart perform on more challenging datasets, such as HY3D-Bench and PartObjaverse?
  • How could the method detect and correct errors in the initial primitive fitting?
  • What additional strategies could make the approach more robust to complex geometry, including intricate openings and fine surface details?
4 comments
AuthorsOpenReview ↗

Response to Reviewer 1Xay

We thank the reviewer for raising the limits of transfer evaluation and the dependence of primitive grouping on the underlying geometry.

R3.1 — HY3D-Bench and PartObjaverse. We evaluated the PartNet-trained model directly on 200 generated meshes from HY3D-Bench, without fine-tuning. The set includes categories not present in the PartNet training data. The main results are:

MethodmIoU ↑RI ↑VoI ↓SC ↑
P3-SAM0.83240.90010.51280.8433
RePart0.50960.72510.90150.6218
SAMPart3D0.50820.74611.09190.6101
PartField0.48750.67281.59090.4387
MPS0.25290.61282.08380.2305
PrimitiveAnything0.23890.61002.82370.2200

Compared with MPS, RePart increases mIoU by 0.2567 and reduces VoI by 1.1823, demonstrating that sequential grouping remains useful on these generated meshes. It ranks second in mIoU, VoI, and SC, but remains substantially below P3-SAM; we will qualify our generalization claims accordingly. Our current PartObjaverse experiments, in contrast, show limited transfer. We will explicitly report this limitation rather than attribute it to a single cause without a controlled analysis. The present evidence supports transfer to the tested HY3D-Bench distribution, not universal cross-dataset generalization.

R3.2 — Detection and correction of initial fitting errors. The reviewer is correct that a cross-part SQ cannot be split by the current merge-and-stop policy. We use a moderate MPS initialization and additionally measure the purity of SQ occupied supports; mean purity is 90.22% on PartNet (8,317 SQs) and 88.73% on 3DCoMPaT++ (10,805 SQs). Here, purity is the dominant ground-truth part fraction among independently occupied points in an SQ, averaged equally over valid SQs. It describes the initial semantic mixing, not an inference-time error detector. Geometric fit residuals and assignment uncertainty are available as state descriptors, but the current model has no validated primitive-splitting or repair operation. Boundary-aware surface labeling can refine the final mesh boundary but likewise cannot correct a fundamentally mixed primitive. We will state this distinction and the remaining limitation explicitly.

R3.3 — Intricate openings and fine surface details. We agree that intricate openings, thin structures, and high-frequency detail can lead to inaccurate or fragmented SQ fitting. RePart uses SQs as a workspace for part organization, not as the final reconstructed surface. After grouping, Boundary-Aware Surface Labeling transfers the inferred part identities to curvature-consistent regions of the original mesh, so final boundaries need not coincide with SQ surfaces. This can preserve local surface detail when the underlying part grouping is correct. However, it cannot recover missing structural components or reverse cross-part mixing already present in the initial SQs. We will clarify these two cases and add representative complex-shape successes and failures, without claiming that surface labeling resolves severe initialization errors.

Thank you for the additional experiments and clarifications. The HY3D-Bench evaluation and expanded test sets address a substantial part of my concern about the evaluation scope. The initialization study is also useful: using the same trained policy across different decomposition granularities provides evidence that the method tolerates some variation in the primitive workspace. These results make my original clear-rejection assessment too strong, and I am raising my score from 1 to 2 (weak rejection).

My remaining concerns are about the extent of this robustness and transfer. On HY3D-Bench, RePart substantially improves over MPS, but its mIoU is close to SAMPart3D and remains well below P3-SAM. The reported limited transfer to PartObjaverse would also benefit from quantitative results and the evaluation settings. Regarding initialization, the mean SQ purity is informative, but it does not show how frequently severely mixed primitives occur or how strongly they affect the final segmentation. A breakdown of low-purity cases and representative complex-shape failures would help establish the method’s practical limits. I appreciate the authors’ explicit acknowledgment of these limitations; the new evidence strengthens the paper, although it does not yet fully resolve my concerns.

AuthorsOpenReview ↗

Follow-up Response to Reviewer 1Xay

Thank you for the careful reading and for updating your assessment. We address the remaining points below.

1. HY3D-Bench: gap to P3-SAM, proximity to SAMPart3D. This benchmark is in-domain for P3-SAM: its part labels come from the same kind of connected-component annotation pipeline, by the same team, that labeled P3-SAM's 3.7M training meshes. RePart is trained on 900 PartNet shapes, from geometry only, on a single RTX 3090. The ranking reverses under PartNet's label convention, where RePart leads P3-SAM on both test sets (mIoU 0.5460 vs. 0.4858, and 0.5083 vs. 0.4326). On 3DCoMPaT++, which neither method was trained on, P3-SAM leads in mIoU by 0.04–0.05, and RI and SC differ by about 0.02 or less in either direction. Against SAMPart3D on HY3D-Bench, RePart is ahead in mIoU, VoI (0.9015 vs. 1.0919), and SC, and it leads PartField on all four metrics. A policy trained on 900 shapes therefore doubles the mIoU of its initialization (0.2529 to 0.5096) on object types it has never seen.

2. How often mixed primitives occur and how much they matter. Two analyses answer this.

Per-shape breakdown. We grouped the 200 test shapes of each dataset by their mean SQ purity under the default setting:

Mean SQ purityPartNet shapesPartNet mIoU3DCoMPaT++ shapes3DCoMPaT++ mIoU
< 0.510.104510.1276
0.5–0.7170.2220200.2669
0.7–0.9800.4896850.4128
≥ 0.91020.6492940.5078

Severely mixed decompositions are rare: one shape per dataset falls below 0.5, and 9% (PartNet) and 10.5% (3DCoMPaT++) fall below 0.7. When they occur, they matter: mIoU on these shapes is 0.22 (PartNet) and 0.26 (3DCoMPaT++), against 0.49–0.65 and 0.41–0.51 on the remaining 90%. This is the practical limit of a merge-only policy, and it is confined to roughly one shape in ten.

Controlled change of purity. With the trained policy held fixed, we changed only the MPS setting:

DatasetSettingMean puritymIoU
PartNetCoarse87.40%0.5193
PartNetDefault90.22%0.5460
PartNetFine91.26%0.5512
3DCoMPaT++Coarse85.52%0.4253
3DCoMPaT++Default88.73%0.4476
3DCoMPaT++Fine90.29%0.4522

On PartNet, raising mean purity by 3.86 points changes mIoU by 0.032, and the last 1.04 points change it by 0.0052. On 3DCoMPaT++, 4.77 points change mIoU by 0.027, and the last 1.56 points by 0.0046 (RI, VoI, SC, and runtimes are in our reply to Reviewer mBL1, R2.2). Refining the decomposition therefore helps, with diminishing returns once most shapes are already well decomposed. Table 4 (Sec. 4.3) shows the other lever: with the primitives fixed and only the grouping objective changed, PartNet-furniture mIoU goes from 0.3757 for the best alternative, Greedy-BC, to 0.4280 for RePart-S, our policy trained on the same 300 furniture shapes.

3. Complex shapes. The 200 HY3D-Bench meshes include characters, food, and buildings, none of which appear in PartNet training, and the HY3D-Bench results above are computed over all of them. Final boundaries are not tied to SQ surfaces: with the groups fixed, replacing facewise nearest-SQ labeling with Boundary-Aware Surface Labeling raises mIoU by 0.03 to 0.09 across the three PartNet groups (Appendix Table C.8). The remaining failures occur when an SQ bridges two parts or a thin structure is fitted in fragments, and we will add representative cases of both to the revision.

AuthorsOpenReview ↗

PartObjaverse-Tiny Results (Reviewer 1Xay)

Following up on your request for quantitative PartObjaverse results and the evaluation settings. We randomly split PartObjaverse-Tiny into two halves of 100 shapes and report RePart on one half under two settings: (a) zero-shot, where the PartNet-trained policy is applied without any further training, and (b) adapted, where the same policy receives 50 further training steps on the other 100 shapes. Metrics are computed as in Table 2.

MethodmIoU ↑RI ↑VoI ↓SC ↑
EMS0.14240.3731.7700.283
MPS (initialization)0.23720.7192.3700.295
PrimAny0.26540.7442.4530.292
SAMPart3D0.46740.8181.3000.541
PartSAM0.49150.7721.5630.481
PartField0.66770.8151.0880.548
RePart, zero-shot0.43880.7741.3300.530
RePart, adapted (50 steps)0.51250.8091.1680.588

Zero-shot, RePart raises its MPS initialization from 0.2372 to 0.4388 mIoU and lowers VoI from 2.370 to 1.330. It trails SAMPart3D by 0.029 mIoU, PartSAM by 0.053, and PartField by 0.229; this is the limited transfer we noted earlier. After 50 training steps on 100 shapes, mIoU rises to 0.5125, above SAMPart3D and PartSAM, with the second-best VoI and the highest SC, while PartField still leads by 0.155 mIoU. A brief adaptation thus recovers 0.074 mIoU, which indicates that much of the zero-shot gap comes from the shift in object types and label convention rather than from the grouping formulation. We will add this table to the revision.

Reviewer_mBL1OpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes RePart for 3D object part segmentation by decoupling structural abstraction, semantic part organization, and boundary localization. First, the authors employ Marching-Primitives to decompose an input 3D mesh into an over-segmented set of superquadric primitives, forming a compact geometric representation. They then formulate primitive grouping as a finite-horizon merge-and-stop Markov decision process and train the grouping policy using PPO to optimize sequential merging decisions. Finally, a boundary-aware surface labeling module transfers the inferred part identities back to the original mesh using curvature-consistent regions and geometric evidence from the grouped primitives. By reasoning over combinations of geometric primitives rather than directly predicting dense part labels, RePart achieves semantic part segmentation using geometry-based information. Experiments on PartNet and 3DCoMPaT++ demonstrate strong segmentation performance and a degree of cross-dataset robustness without target-dataset fine-tuning.

Critical strengths

A key strength of this paper is its formulation of semantic part organization as a Markov decision process, where a grouping policy is optimized using PPO. This formulation allows the model to focus on the sequential organization of geometric primitives rather than directly reasoning over dense, low-level surface representations, providing a simple and structured approach to semantic part segmentation. Moreover, the ablation results demonstrate the advantages of optimizing long-term partition quality over local or greedy merging strategies, supporting the effectiveness of the proposed formulation.

Critical weaknesses
  1. Insufficient implementation details. The paper lacks several important implementation details necessary for reproducibility. For example, the exact architecture of the actor–critic networks is not clearly specified, including the MLP configurations and the design of the action heads. Moreover, Sec. 3.3 introduces the features Φtprim\Phi_t^{\mathrm{prim}}, Φtglobal\Phi_t^{\mathrm{global}}, and Φtpair\Phi_t^{\mathrm{pair}} without clearly explaining how they are computed. These missing details make it difficult to faithfully reproduce the proposed method and assess the impact of specific architectural choices on the reported performance.

  2. Dependence on the initial primitive decomposition. The proposed method relies on a fixed set of SQ primitives generated by MPS, and the subsequent grouping process only supports merge and stop operations. Consequently, errors in the initial decomposition, particularly primitives that span multiple semantic parts, cannot be corrected during the grouping process. Although the authors acknowledge this limitation, the paper lacks a systematic analysis of how the quality and granularity of the initial primitive decomposition affect the final segmentation performance. This raises concerns about the robustness of the proposed framework.

Questions for authors

Clarification on implementation details. Could the authors provide more details about the implementation of the proposed framework? Specifically, (1) what are the exact architectures of the actor and critic networks, including the MLP configurations and the design of the merge and stop action heads? (2) How are the features Φtprim\Phi_t^{\mathrm{prim}}, Φtglobal\Phi_t^{\mathrm{global}}, and Φtpair\Phi_t^{\mathrm{pair}} in Sec. 3.3 computed? Clarifying these details would help assess the reproducibility of the proposed method and better understand the architectural choices underlying its performance.

3 comments
AuthorsOpenReview ↗

Response to Reviewer mBL1 (1/2)

We thank the reviewer for recognizing the value of sequential primitive organization and for identifying missing implementation details and a limitation of fixed primitive decompositions.

R2.1 — Actor–critic architecture and state features. We provide the active network configuration and feature definitions below. The actor uses only geometry and the current grouping; labels and ground-truth part counts are privileged training signals, not test-time inputs. The default configuration is a maximum of M=200M=200 primitive slots, 512 sampled points, and a 200-step episode horizon. Hidden MLP layers use ReLU.

ModuleArchitecture / aggregation
Point encoder7→64→1287\to64\to128; max + mean pooling, LayerNorm (256-D)
SQ encoder11→64→12811\to64\to128; masked mean pooling (128-D)
Primitive/group encoder17→64→6417\to64\to64; masked mean pooling (64-D)
Actor context(M+456)→256→256(M+456)\to256\to256
Actor primitive token192→128→128192\to128\to128
Target-count predictor256→64→1256\to64\to1, sigmoid
Pair-feature encoder17→32→3217\to32\to32
Stop / merge heads273→256→1273\to256\to1 / 689→256→1689\to256\to1
Critic primitive encoder34→128→6434\to128\to64; masked attention pooling
Critic global encoder4→32→324\to32\to32
Critic value head352→256→256→1352\to256\to256\to1

Each point contains original xyzxyz, centered radius-normalized xyzxyz, and normalized radial distance (7-D). Each SQ contains two shape exponents, three axis lengths, three Euler angles, and three translations (11-D). The actor context concatenates a 256-D point summary, 128-D SQ summary, 64-D primitive summary, an MM-D validity mask, and eight global features, giving M+456M+456 dimensions. The stop head takes the 256-D context, predicted target-count ratio, and a 16-D action-type embedding (273-D). The merge head additionally uses the two 128-D group-representative tokens, their absolute difference, and a 32-D pair embedding (689-D). Valid merge actions and stop share one masked categorical distribution.

The state descriptors have the following fixed and reproducible composition:

  • Φprim\Phi_{\mathrm{prim}} (17-D): seven geometry/group descriptors (nearest primitive-center distance, normalized group size, group AABB diagonal, group-to-scene-center distance, point-support fraction, neighboring-group primitive fraction, and minimum inter-group AABB gap); four assignment statistics (normalized entropy, mean top-1 probability, fraction of top-1/top-2 margins below 0.2, and mean margin); and six shape/fit statistics (PCA linearity, planarity, scattering, principal-axis/SQ-axis alignment, mean residual, and 90th-percentile residual). Assignment probabilities are softmax weights over negative approximate point-to-SQ surface distances, with temperature equal to median distance plus 0.001.
  • Φglobal\Phi_{\mathrm{global}} (8-D): t/Ht/H, n/Mn/M, G/MG/M, G/G0G/G_0, singleton-group fraction, normalized mean group size, valid-merge fraction, and mean primitive-neighbor fraction.
  • Φpair\Phi_{\mathrm{pair}} (17-D): six representative-primitive descriptors (validity, center distance, AABB gap, scale similarity, volume ratio, orientation similarity); eight full-group descriptors (combined size, center distance, AABB IoU, support similarity, diagonal ratio, group AABB gap, union-volume expansion, diagonal growth); and three contact/orientation descriptors (projected AABB contact, PCA-axis alignment, normal alignment). Spatial distances and extents are scene-radius normalized. PCA linearity, planarity, and scattering are respectively (λ1−λ2)/λ1(\lambda_1-\lambda_2)/\lambda_1, (λ2−λ3)/λ1(\lambda_2-\lambda_3)/\lambda_1, and λ3/λ1\lambda_3/\lambda_1. Pairwise scale similarity is exp⁡(−∣log⁡ai−log⁡aj∣2)\exp(-\\|\log a_i-\log a_j\\|_2), and volume similarity is min⁡(Vi,Vj)/max⁡(Vi,Vj)\min(V_i,V_j)/\max(V_i,V_j). Union-volume expansion is (Vunion−Vi−Vj)/(Vi+Vj)(V_{\mathrm{union}}-V_i-V_j)/(V_i+V_j) (clipped to [−1,10][-1,10]); diagonal growth is the union-box diagonal minus the larger group diagonal (normalized and clipped to [−1,5][-1,5]). The contact proxy uses maximum projected AABB overlap area near contact, divided by the smaller box surface area.

Each group is indexed by its lowest-index member; candidate merges are the union of the four AABB-nearest neighbors of each group. Group features and candidate pairs are recomputed after every merge. The actor context pools over valid primitive slots, not separately within each group; we will correct that description in the paper.

AuthorsOpenReview ↗

Response to Reviewer mBL1 (2/2)

R2.1 (continued). The critic augments the 11-D SQ and 17-D geometry features with six training-only statistics (point-support fraction, label purity, second-largest label fraction, normalized label count, label entropy, and one minus purity), encoded as 34→128→6434\to128\to64. Four privileged global features (target-count ratio, signed and absolute relative count error, and an exact-count indicator) are encoded as 4→32→324\to32\to32. Together with a detached 256-D actor context, this produces the 352-D critic input. Auxiliary heads predict terminal mIoU and normalized group-count error. Ground-truth counts supervise the actor's count predictor during training, but only its geometry-based prediction enters the action heads at inference. Thus, neither ground-truth labels nor critic features are needed at test time. We will make these specifications explicit in the method and appendix.

R2.2 — Sensitivity to initial SQ granularity. We agree that a merge-only policy cannot split an SQ spanning different semantic parts. To examine how the initial decomposition affects the final segmentation, we evaluated three MPS granularity settings on PartNet and 3DCoMPaT++, the two datasets evaluated in the paper. All three settings use the same trained RePart model; only the initial decomposition is changed, without retraining. Default is the configuration used for the paper's main results.

We jointly vary four controls of the MPS initialization: Grid resolution, the number of SDF grid samples per axis (rr gives an r3r^3 grid); minArea, the minimum voxel count for a connected region to enter SQ fitting; nanRange, the signed-distance threshold around a fitted SQ used to mark covered SDF samples for removal before subsequent fitting; and maxDivision, the limit on outer MPS decomposition rounds. Here, hh denotes the grid spacing at the corresponding resolution. A higher resolution, smaller minArea and nanRange, and larger maxDivision favor a finer decomposition.

SettingGrid resolutionminAreananRangemaxDivision
Coarse6481.0h1.0h25
Default10050.5h0.5h50
Fine12830.25h0.25h75

We measure each SQ's semantic purity on its independently occupied point support, Sj=x:Fj(x)≤1S_j=\\{x:F_j(x)\leq1\\}:

Purity(j)=max⁡k∣x∈Sj:y(x)=k∣∣Sj∣.\mathrm{Purity}(j)=\frac{\max_k|\\{x\in S_j:y(x)=k\\}|}{|S_j|}.

Mean purity is the equally weighted average over nonempty SQ supports. Overlapping SQs are assessed independently, without competitive point-to-primitive assignment.

DatasetSettingTotal SQsMean purity
PartNetCoarse3,60287.40%
PartNetDefault8,31790.22%
PartNetFine14,37791.26%
3DCoMPaT++Coarse4,70085.52%
3DCoMPaT++Default10,80588.73%
3DCoMPaT++Fine19,18190.29%

The resulting segmentation metrics and average preprocessing times per mesh are:

DatasetSettingmIoU ↑RI ↑VoI ↓SC ↑SDF construction (s)MPS fitting (s)
PartNetCoarse0.51930.78450.87140.66893.2531.22
PartNetDefault0.54600.79650.84710.689113.3847.53
PartNetFine0.55120.79680.84240.691722.17157.78
3DCoMPaT++Coarse0.42530.73821.23490.57203.5849.55
3DCoMPaT++Default0.44760.75751.28380.579013.4061.36
3DCoMPaT++Fine0.45220.76031.22860.588422.10213.03

These results show that Fine achieves the best segmentation metrics on both datasets, but its gains over Default are modest relative to the additional computational cost. From Coarse to Fine, mean purity rises from 87.40% to 91.26% on PartNet and from 85.52% to 90.29% on 3DCoMPaT++. Fine also achieves the highest mIoU, RI, and SC and the lowest VoI among the three settings on both datasets. However, compared with Default, mIoU increases by only 0.0052 on PartNet and 0.0046 on 3DCoMPaT++ (0.52 and 0.46 percentage points, respectively).

The additional purity from Default to Fine is 1.04 and 1.56 percentage points, respectively, while MPS fitting time increases from 47.53 to 157.78 s on PartNet (3.32×) and from 61.36 to 213.03 s on 3DCoMPaT++ (3.47×). SDF construction also becomes slower. Thus, achieving these additional accuracy gains requires more than tripling the MPS fitting cost. Considering segmentation quality and runtime together, Default is a more practical choice, retaining performance close to Fine at substantially lower preprocessing cost. This experiment measures sensitivity to jointly configured granularity under a fixed policy; it does not establish the effect of each parameter in isolation or remove the structural limitation of merge-only grouping.

Thank you for the clarification. After reading the rebuttal, I believe my primary concerns have been addressed, and I have decided to maintain my original score.

Reviewer_r2YkOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper presents RePart, a framework for class-agnostic 3D part segmentation that organizes geometric primitives into coherent parts. The method first constructs an over-segmented superquadric representation using Marching-Primitives, then learns a sequential merge-and-stop policy with PPO. RePart achieves the best aggregate results among the evaluated methods on PartNet and competitive transfer performance on 3DCoMPaT++, with additional experiments on unseen PartNet categories.

Critical strengths
  • The paper clearly distinguishes geometric decomposition from semantic part organization.
  • The in-domain results are consistently strong.
  • Overall, this paper is well-written and easy to follow.
Critical weaknesses
  • The experiments are limited to PartNet and 3DCoMPaT++, leaving the applicability of RePart to AI-generated meshes unclear.
  • Section 4.1 states that ground-truth labels are used for granularity selection, but it is unclear which methods receive this selection, what candidate granularities are considered, and whether all reported metrics use the same selected partition.
  • The main experiments use 200 test shapes per dataset, and the unseen-category evaluation uses only ten shapes per category.
  • The pipeline requires signed-distance-field construction, superquadric fitting, sequential policy evaluation, and dense surface labeling. Reporting a single GPU model does not establish practical efficiency.
Questions for authors

Please see the weaknesses.

3 comments
AuthorsOpenReview ↗

Response to Reviewer r2Yk

We thank the reviewer for recognizing the distinction between geometric decomposition and semantic part organization, and for identifying important questions about evaluation and efficiency.

R1.1 — Applicability to AI-generated meshes. We evaluated the PartNet-trained RePart model, without target-dataset fine-tuning, on 200 AI-generated HY3D-Bench meshes. The set includes furniture and containers as well as object types absent from PartNet training, including characters, food, and buildings.

MethodmIoU ↑RI ↑VoI ↓SC ↑
P3-SAM0.83240.90010.51280.8433
SAMPart3D0.50820.74611.09190.6101
RePart0.50960.72510.90150.6218
PartField0.48750.67281.59090.4387
MPS0.25290.61282.08380.2305
PrimitiveAnything0.23890.61002.82370.2200

RePart ranks second in mIoU, VoI, and SC. P3-SAM's lead reflects an in-domain setting: HY3D-Bench's part labels come from the same kind of connected-component annotation pipeline, by the same team, that labeled P3-SAM's 3.7M training meshes, whereas RePart is trained on 900 PartNet shapes under a different label convention. Under this shift, RePart still raises the MPS initialization from 0.2529 to 0.5096 mIoU and lowers VoI from 2.0838 to 0.9015. We present this as evidence of transfer, not state-of-the-art performance on HY3D-Bench.

R1.2 — Training and evaluation granularity. We agree that Sec. 4.1 was incorrectly worded. RePart is trained with the finest available part annotations, but its test-time granularity is determined by the learned merge-and-stop policy, not by ground-truth labels. All methods are evaluated against the same finest-granularity reference. For baselines that expose multiple output granularities, including PartField, we use a fixed finest-granularity setting rather than selecting a partition with test labels. For each shape, mIoU, RI, VoI, and SC are computed from one fixed predicted partition; no metric-specific or ground-truth-based granularity selection is performed. We will correct Sec. 4.1 to state this protocol explicitly.

R1.3 — Evaluation set size and unseen categories. We expanded the evaluation with 200 additional PartNet shapes, 200 additional 3DCoMPaT++ shapes, and 100 shapes in the unseen PartNet evaluation set, none overlapping the original test samples. The additional unseen set contains 15 shapes each from Bed, Hat, Refrigerator, TrashCan, and Laptop, and 25 shapes from Display, totaling 100 shapes across six categories. All baselines follow the same metric protocol.

Additional PartNet (200 shapes):

MethodmIoU ↑RI ↑VoI ↓SC ↑
RePart0.50830.78191.08160.6189
P3-SAM0.43260.71831.40250.5271
PartField0.42390.69401.58670.4890
SAMPart3D0.37510.67991.71410.4796
MPS0.24640.60791.30670.2319
PrimitiveAnything0.26830.61952.52190.2698

Additional 3DCoMPaT++ (200 shapes):

MethodmIoU ↑RI ↑VoI ↓SC ↑
RePart0.37270.71241.42930.5216
P3-SAM0.41420.73351.39130.5336
PartField0.28460.66321.64100.4412
SAMPart3D0.35570.69871.52810.5106
MPS0.24700.63861.33450.2298
PrimitiveAnything0.23990.62682.35930.2889

Unseen PartNet evaluation set (100 shapes):

MethodmIoU ↑RI ↑VoI ↓SC ↑
RePart0.39190.68220.94980.5642
P3-SAM0.40720.72660.95640.6005
PartField0.40200.69711.35610.5243
SAMPart3D0.27280.52171.48780.4183
MPS0.23120.60211.08520.2608
PrimitiveAnything0.21040.59402.57670.2585

RePart leads all four metrics on the additional PartNet set. On the additional 3DCoMPaT++ set, it ranks second in mIoU, RI, and SC, and third in VoI. On unseen PartNet shapes, it achieves the lowest VoI and second-highest SC, while ranking third in mIoU and RI. The larger evaluation supports the original in-domain results and shows competitive, but not uniformly leading, performance on transferred and unseen distributions.

R1.4 — End-to-end computational cost. We profiled all four stages on the 200 PartNet test shapes used in the paper (RTX 3090 GPU, AMD 5800 CPU). The following times are averaged per mesh over this test set:

StageAverage per mesh
SDF construction13.38 s
SQ fitting47.53 s
Sequential policy inference18.47 s
Dense surface labeling8.97 s
Total88.35 s

SQ fitting accounts for approximately 53.8% of total runtime and is the primary bottleneck; the learned sequential policy takes 18.47 s. This breakdown makes both preprocessing and boundary recovery costs explicit. We will include the stage-wise measurements and timing setup in the revision.

AuthorsOpenReview ↗

Follow-up to Reviewer r2Yk

Dear Reviewer r2Yk,

Thank you again for your detailed review. As the discussion period closes soon, we would be grateful if you could let us know whether our response addresses your concerns, in particular the clarified evaluation-granularity protocol (R1.2), the additional evaluation on 700 shapes (R1.1, R1.3), and the stage-wise runtime breakdown (R1.4). We are happy to provide any further clarification in the time remaining.

Best regards, The Authors

AuthorsOpenReview ↗

Additional Results on PartObjaverse-Tiny (Reviewer r2Yk)

Further to R1.1 on evaluation beyond PartNet and 3DCoMPaT++, we report results on an additional dataset, PartObjaverse-Tiny. We randomly split PartObjaverse-Tiny into two halves of 100 shapes and report RePart on one half under two settings: (a) zero-shot, where the PartNet-trained policy is applied without any further training, and (b) adapted, where the same policy receives 50 further training steps on the other 100 shapes. Metrics are computed as in Table 2.

MethodmIoU ↑RI ↑VoI ↓SC ↑
EMS0.14240.3731.7700.283
MPS (initialization)0.23720.7192.3700.295
PrimAny0.26540.7442.4530.292
SAMPart3D0.46740.8181.3000.541
PartSAM0.49150.7721.5630.481
PartField0.66770.8151.0880.548
RePart, zero-shot0.43880.7741.3300.530
RePart, adapted (50 steps)0.51250.8091.1680.588

Zero-shot, RePart raises its MPS initialization from 0.2372 to 0.4388 mIoU and lowers VoI from 2.370 to 1.330. It trails SAMPart3D by 0.029 mIoU, PartSAM by 0.053, and PartField by 0.229; this is the limited transfer we noted in our response to Reviewer 1Xay (R3.1). After 50 training steps on 100 shapes, mIoU rises to 0.5125, above SAMPart3D and PartSAM, with the second-best VoI and the highest SC, while PartField still leads by 0.155 mIoU. A brief adaptation thus recovers 0.074 mIoU, which indicates that much of the zero-shot gap comes from the shift in object types and label convention rather than from the grouping formulation. We will add this table to the revision.

Comments on the paper

1 comment
AuthorsOpenReview ↗

General Response

We thank the reviewers for their constructive comments. Two concerns are shared across reviews; we summarize our response to them here and give full tables in the individual replies.

1. Evaluation scope (r2Yk, 1Xay). We added 700 evaluation shapes. On 200 AI-generated HY3D-Bench meshes, the PartNet-trained model, without fine-tuning, raises the MPS initialization from 0.2529 to 0.5096 mIoU and lowers VoI from 2.0838 to 0.9015, ranking second in mIoU, VoI, and SC. On 200 additional PartNet shapes, RePart leads all four metrics (mIoU 0.5083 vs. 0.4326 for the next-best method). On 200 additional 3DCoMPaT++ shapes it ranks second in mIoU, RI, and SC, and on 100 unseen-category PartNet shapes it has the lowest VoI.

2. Dependence on the initial SQ decomposition (mBL1, 1Xay). A merge-only policy cannot split an SQ that spans two parts, so we measured how often this occurs and how much it affects the result. Mean SQ purity is 90.22% on PartNet and 88.73% on 3DCoMPaT++. Across three MPS granularities with the same trained policy, purity rises from 87.40% to 91.26% on PartNet and mIoU from 0.5193 to 0.5512; refining beyond the default setting adds only 0.0052 mIoU at 3.32× the fitting cost (reply to mBL1, R2.2).

3. Clarifications. We provide a stage-wise runtime breakdown (88.35 s per mesh; r2Yk, R1.4), the full actor–critic architecture and state-feature definitions (mBL1, R2.1), and the corrected description of the evaluation-granularity protocol (r2Yk, R1.2). We will update the manuscript accordingly.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.