acceptodds
Withdrawn from ICLR 2027

Taxonomy-Aligned Hyperbolic Learning for 3D Point Cloud

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Hierarchical 3D segmentation requires accurate point-wise predictions and coherent representations across semantic levels. However, enforcing consistency among predicted labels does not by itself ensure that feature geometry reflects the underlying taxonomy. We introduce Hy3D, a hierarchical hyperbolic prototype network that learns multi-level point features in a shared Poincar\'e ball for 3D semantic and part segmentation. A shared hyperbolic projection layer maps features from all hierarchy levels into a common space with a unified scale, while an online prototype bank maintains per-level, per-class prototypes through cloud-balanced exponential moving averages in the tangent space. During training, a parent-prototype triplet loss and a depth-aware radius loss encourage parent affinity and radial progression, aligning the learned representations with the label hierarchy. M\"obius classification heads produce class logits from the hyperbolic features without modifying the underlying Euclidean backbone. Experiments on Campus3D and PartNeXt demonstrate the effectiveness of Hy3D in improving segmentation performance across both scene-level semantic hierarchies and fine-grained part hierarchies.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Hierarchical Learning, 3D Point Cloud, Hyperbolic Learning

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_ZKLDOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

The paper presents Hy3D for hierarchical 3D point-cloud segmentation. It maps point features from different semantic levels into a shared Poincaré ball and uses class prototypes, a parent-affinity loss, and a depth-aware radial loss to align the features with the taxonomy. Experiments cover Campus3D and a custom five-level hierarchical segmentation protocol on PartNeXt. The authors use PointNet++ on both datasets and additionally evaluate a PointNeXt encoder on PartNeXt, with Euclidean controls and component ablations.

Critical strengths

None

Critical weaknesses
  1. The advance over existing Poincaré embeddings, prototype-margin constraints, and the Campus3D decoder is limited. The theoretical results are mainly hinge identities and a radial budget. The paper evaluates PartNeXt [3] using a custom five-level protocol rather than the dataset’s original class-agnostic part-segmentation benchmark, which includes PartField [4], SAMPart3D [5], and SAMesh [6]. The paper lacks comparisons with related methods such as DHL [1] and HyCoRe-seg [2], and does not compare with PartField [4], SAMPart3D [5], or SAMesh [6] under a common class-agnostic evaluation protocol. With PointNeXt, HE mean mIoU improves by only about 0.10 percentage points, compared with seed standard deviations of approximately 0.66–0.69 points.

The presentation is excessively defensive and repetitive. Statements that the theory does not establish "a special advantage of hyperbolic space," three-seed results "do not establish statistical significance," and qualitative examples represent local predictions "rather than test-set averages" repeatedly interrupt the argument and obscure the contribution. The figures are poorly explained: Figure 3 uses unnamed "leaf" labels, Figure 4 lacks a semantic legend, and neither identifies the errors being corrected.

  1. On Campus3D, M0 without the geometric losses already performs close to the full model. With the classification heads held fixed, adding the losses improves OA/mIoU by only 1.65/2.30 percentage points; on PartNeXt, the gains are about 0.6/1.0 points. PartNeXt path consistency barely changes, and prototype–taxonomy Spearman correlations are mostly only 0.1–0.3. The paper also does not report parent-retrieval and radial-constraint diagnostic results. The existing results do not show whether the parent-prototype constraint reduces cross-parent-branch errors in the classification heads before HE decoding.

[1] Chongshou Li et al. Deep Hierarchical Learning for 3D Semantic Segmentation. IJCV, 2025. Paper.

[2] Antonio Montanaro, Diego Valsesia, and Enrico Magli. Towards Hyperbolic Regularizers for Point Cloud Part Segmentation. ICASSP, 2023.

[3] Penghao Wang et al. PartNeXt: A Next-Generation Dataset for Fine-Grained and Hierarchical 3D Part Understanding. NeurIPS, 2025. Paper.

[4] Minghua Liu et al. PartField: Learning 3D Feature Fields for Part Segmentation and Beyond. 2025. Paper.

[5] Yunhan Yang et al. SAMPart3D: Segment Any Part in 3D Objects. arXiv:2411.07184, 2024. Paper.

[6] George Tang et al. Segment Any Mesh. arXiv:2408.13679, 2024. Paper.

Questions for authors
  1. What specific limitations of the original protocol do the five-level mapping and the 8:1:1 split on PartNeXt respectively address? The original benchmark evaluates PartField, SAMPart3D, and SAMesh on class-agnostic part segmentation. Could you provide comparisons with these methods under a common class-agnostic protocol, and with DHL or HyCoRe-seg under your custom hierarchical protocol? Merely stating that "we study a given taxonomy" is insufficient to explain the absence of competitive baselines.

  2. The parent-margin loss only constrains distances between child-level features and parent prototypes. The prototype bank is discarded at inference, and the classification weights are not tied to the prototypes. Please identify which result shows that this constraint reduces cross-parent-branch errors before HE decoding, rather than merely improving overall segmentation accuracy. Do the existing ablations distinguish improvements in classification within the same parent branch from reductions in errors across parent branches? If not, the paper has not demonstrated that the prototype-based geometric constraint improves the parent–child classification relationships it targets.

Reviewer_RBhXOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper introduces Hy3D for hierarchical semantic and part segmentation of 3D point clouds. It projects point features from multiple semantic levels into a shared Poincare ball and maintains class prototypes through exponential moving averages. A triplet loss encourages features to approach their correct parent category, while a depth-aware radial loss encourages deeper semantic representations to lie farther from the origin. Separate classification heads predict labels at each level, with an optional existing hierarchical decoder enforcing valid output paths. Experiments on Campus3D and PartNeXt evaluate segmentation accuracy, component contributions, and alignment between prototype distances and taxonomy structure.

Critical strengths
  • The paper explicitly acknowledges that valid-path decoding guarantees consistency by construction and that auxiliary geometric constraints do not guarantee correct classifier outputs. The bounded-radius feasibility analysis also identifies a practical constraint on choosing radial margins.

  • The model shows some results improvemnet on PartNeXt, mean mIoU improves from 13.31 to 14.32, with improvements at all five levels. Campus3D also improves from 57.22 to 59.52 mIoU. This provides evidence that the losses contribute beyond the classification head alone.

Critical weaknesses
  • The experiments do not isolate the contribution of hyperbolic geometry from changes in classifier behavior and distance scale, as the authors acknowledge in Section 5.1 and Appendix E.3. Although the complete hyperbolic model performs better, the result leaves uncertainty about which changes lead to the improvement. The gains on PartNeXt are also much smaller than on Campus3D. Futhermore, a scale-calibrated comparison or a controlled curvature sweep would strengthen the empirical motivation for using hyperbolic geometry.

  • The authors discuss HyCoRe-seg, which also uses hyperbolic representations with radial and contrastive regularization, but do not compare against it experimentally. Moreover, the M1 baseline remove the proposed losses and does not replace them with an existing regularizer. Therefore, the improvement over M1 demonstrates that the proposed losses are useful, but does not establish their advantage over related regularization approaches under the same experimental setting.

Questions for authors
  • Since the Euclidean and hyperbolic models also differ in their classifiers and distance scales, how much of the improvement comes from hyperbolic geometry itself? Could the authors test different curvatures or calibrate the distance scales to help separate these effects?

  • Why was the improvement over the Euclidean model much larger on Campus3D than on PartNeXt? Is there any underlying observation for this behavior?

  • The authors should compare the losses with HyCoRe-seg’s regularization using the same backbone and training setup? The comparison with M1 shows that the losses help, but it would be useful to see how they compare with an existing regularizer.

  • The average scores improve, but performance at the finest levels of PartNeXt remains low. Which categories benefit most from the proposed losses, how about rare or fine-grained categories, are they also improve?

Reviewer_hfcQOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

In this paper, the authors proposed Hy3D, a hierarchical hyperbolic learning framework for 3D point cloud semantic and part segmentation. The method first projects multi-level point features into a shared Poincaré ball and then introduces an online prototype memory to capture hierarchical semantic relationships. Finally, a parent-prototype triplet loss and a depth-aware radial loss are designed to align feature representations with the semantic taxonomy. Experiments on Campus3D and PartNeXt demonstrate improvements in segmentation performance and hierarchical representation learning.

Critical strengths
  • The paper addresses the limitation of conventional hierarchical segmentation methods that mainly enforce consistency at the prediction level. By introducing hyperbolic embeddings and taxonomy-aware geometric constraints, the proposed framework provides a meaningful approach to modeling hierarchical relationships directly in the feature space.
  • Extensive results can demonstrate the effectiveness of the proposed method for 3D point cloud semantic and part segmentation.
Critical weaknesses
  • The proposed framework builds upon established techniques, including Poincaré embeddings, prototype-based learning, triplet losses, and radial regularization. In particular, the closely related HyCoRe-seg already employs hyperbolic representations and geometric regularization for point cloud part segmentation. Although Hy3D introduces taxonomy-aligned prototypes and depth-aware constraints, the paper could more clearly demonstrate the fundamental advances and insights over existing hyperbolic segmentation approaches.
  • As shown in Table 1, the improvements over the Euclidean control are relatively small on PartNeXt. With PointNet++, Hy3D improves mIoU by only 0.53 percentage points under MT and 0.62 points under MT+HE. With PointNeXt, the improvements are 0.45 and 0.10 points, respectively. These modest gains raise questions about whether the additional complexity of hyperbolic representation learning is sufficiently justified.
  • As shown in Table 2, although Hy3D improves segmentation accuracy, it does not consistently improve hierarchical prediction consistency. On PartNeXt, raw path consistency slightly decreases from 58.62% for M1 to 58.57% for Hy3D. Since learning taxonomy-aligned representations is a central motivation of the paper, the limited improvement in prediction consistency deserves further investigation.
  • As shown in Table 3, the Spearman correlations between prototype distances and taxonomy distances remain relatively low, even with hyperbolic representations. For example, the correlation for unique taxonomy nodes increases only from 0.2298 to 0.2622, while the L2-L5 macro-average correlation increases from 0.0788 to 0.1068. These results suggest that the learned feature geometry may still be insufficiently aligned with the underlying semantic hierarchy, weakening one of the paper's central claims.
Questions for authors

Please refer to the weaknesses.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.