acceptodds
Withdrawn from ICLR 2027

Revisiting Triplet Contrastive Learning for 3D Shape Understanding

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

In multi-modal 3D pre-training, aligning 3D shapes, their 2D image counterparts, and language descriptions has made tremendous progress. However, we find that supervision from rendered images is unreliable, the semantics encoded in synthetic views are distorted due to domain gap, self-occlusion, and rendering artifacts. Instead, we learn transferable 3D representations from language only. We observe that, in real world, 3D shapes are organized in a tree-like structure, ranging from general concepts to fine-grained instances with specific attributes, while language typically denotes generic concepts and subsumes these various 3D instantiations. Hence, we propose to embed 3D representations in hyperbolic space, whose geometry is well-suited for modeling tree-like data without distortion. To capture the asymmetric hierarchy between language and 3D shapes, we further impose a partial-order relation that language entails 3D shapes. Our method, Hyperbolic Contrastive Learning (HyperCL), jointly optimizes a hyperbolic geodesic-based contrastive objective and an entailment loss to learn discriminative cross-modal representations while explicitly preserving this hierarchy. We also introduce a rigorously verified benchmark for comprehensively evaluating cross-modal 3D alignment. Across two widely used benchmarks and ours, we improve the performance up to 4.6% absolute percentage on 3D shape classification and 5.5% on cross-modal text-to-3D retrieval under the same parameter budget. The code and evaluation benchmark will be made publicly available.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Contrastive Learning, 3D Shape Analysis, Hyperbolic Geometry

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_wsx9OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper revisits the role of supervision from rendered images in trimodal 3D contrastive pretraining and proposes HyperCL, which uses only text supervision for cross-modal alignment. The method maps point cloud and text representations into hyperbolic space and combines a contrastive objective based on geodesic distance with a batch-wise entailment loss. These objectives aim to capture asymmetric hierarchical relationships between general textual concepts and specific 3D instances. The authors also introduce c-OLVIS, a manually verified version of Objaverse-LVIS, and report improvements on zero-shot classification and cross-modal retrieval.

Critical strengths
  1. Reassessing the necessity of supervision from rendered images addresses a relevant practical question in 3D pretraining. The method builds on an existing point cloud backbone and a frozen text encoder, with training costs comparable to those of the baselines.
  2. The reported improvements span multiple datasets and pretraining settings. When LVIS objects are excluded from pretraining, HyperCL achieves a text-to-3D retrieval R@1 of 25.2%, compared with 21.9% for TAMM, the strongest baseline for this metric, yielding a gain of 3.3 percentage points (Table 4).
  3. The method framework and loss functions are presented clearly and are easy to follow. The exploration of hyperbolic geometry for modeling cross-modal hierarchical relationships may also provide useful insights for the broader multimodal representation learning community.
Critical weaknesses
  1. The improvements in Table 5 do not establish that the underlying semantic structure is tree-like, since the comparison changes both feature normalization and the similarity function. Part of the gain could therefore arise from feature norms or score scaling. Similarly, the modality separation in Figure 4 does not by itself demonstrate a semantic hierarchy, and the sofa → loveseat transition in Figure 5 is inconsistent with progressively greater abstraction. The claims concerning hierarchy preservation and its contribution to the performance gains should be qualified.
  2. In Table 1, removing image supervision decreases OLVIS accuracy from 46.9% to 45.5%, while increasing MN40 accuracy only slightly, from 84.2% to 84.5%. These results provide limited support for a general claim that image supervision harms representation learning. Moreover, this diagnostic experiment uses a different encoder configuration from the main method, leaving the effect of image supervision insufficiently tested under the main experimental setup.
  3. The method assumes that a general textual concept can subsume multiple 3D instances, yet Equation (8) pushes every unpaired shape outside the text’s entailment cone. For example, if two shapes in a batch share the caption “a chair,” the frozen text encoder produces identical cones. Each shape is then encouraged to lie inside that cone for its positive pair and outside the same cone as a negative for the other pair. These incompatible constraints may favor unique instance matching, and their prevalence and impact on the intended hierarchical representation remain unexamined.
Questions for authors
  1. Could the authors include an unnormalized Euclidean distance baseline with comparable temperature tuning? Could they also evaluate the learned hierarchy against verified semantic relationships, using measures such as ancestor prediction accuracy or the monotonicity of abstraction along retrieval paths?
  2. Under the same backbone, initialization, and training budget, how do the four combinations of Euclidean/hyperbolic geometry and text-only/image–text supervision compare? Under which conditions does removing image supervision consistently improve performance?
  3. How common are duplicate or semantically similar captions that create semantic false negatives within training batches? Could the authors compare entailment using only positive pairs, the proposed batch-wise loss, and a version that masks semantic false negatives, to assess the contribution and semantic consistency of the negative term?

If the authors can address most of my concerns, I will raise my score.

Reviewer_rue3OpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper presents HyperCL, a hyperbolic contrastive learning framework for text-3D representation learning. HyperCL aligns text and 3D shape representations in a shared hyperbolic space, leveraging its geometric properties to capture hierarchical semantic relationships. Additionally, a batch-wise entailment loss is introduced to model asymmetric relationships between textual concepts and 3D instances. Experiments on 3D classification and cross-modal retrieval benchmarks demonstrate competitive performance.

Critical strengths

The paper explores hyperbolic representation learning for 3D shape understanding, which is a promising research, and provides both qualitative and quantitative results on multiple datasets.

Critical weaknesses
  1. Limited novelty. Several prior works have explored hyperbolic representations for text, image, and 3D shape. The proposed entailment loss is also closely related to existing methods.

Related paper: COMPOSITIONAL ENTAILMENT LEARNING FOR HYPERBOLIC VISION-LANGUAGE MODELS

  1. Lack of hierarchy visualization. Although the experimental results are promising, the paper lacks a visualization showing whether the learned 3D shape embeddings preserve hierarchical relationships in hyperbolic space.

  2. Insufficient retrieval visualization. The retrieval examples in Figure 5 do not clearly show the hierarchical correspondence between text and 3D shapes. Visualizing the retrieval results at each step would be helpful.

  3. Lack of explicit shape hierarchy supervision. The current loss mainly focuses on text-3D alignment. It would be interesting to introduce an additional loss that explicitly models hierarchical relationships among 3D shapes.

  4. Limited exploration of hyperbolic geometry. The use of hyperbolic geometry could be explored further, beyond simply projecting features into hyperbolic space and applying contrastive and entailment losses.

Questions for authors

see the weakness

Reviewer_WKW8OpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

The paper proposes Hyperbolic Contrastive Learning (HyperCL), replacing unreliable rendered-image supervision with language-only supervision and introducing hyperbolic contrastive learning and a language-to-3D entailment loss. It also presents a manually verified evaluation benchmark.

Critical strengths

The proposed HyperCL uses hyperbolic geometry to represent the tree-like semantic structure of 3D shapes and introduces a partial-order constraint to model the asymmetric relationship between language and 3D shapes.

Critical weaknesses

Novelty: Hyperbolic representation learning and entailment cones have already been explored in hierarchical representation learning and vision-language alignment. Although applying these techniques to 3D shape understanding is interesting, the paper could more clearly distinguish its methodological contributions from existing hyperbolic contrastive learning approaches. In particular, which aspects of the proposed objective are specifically designed for 3D shape understanding rather than directly adapted from previous work?

Limited Evidences: The paper argues that hyperbolic geometry is particularly suitable for modelling the inherent hierarchical structure of 3D shapes. However, the experimental evidence primarily focuses on classification and retrieval performance. Although the hyperbolicity analysis in Table 2 provides some supporting evidence, additional quantitative evaluation of hierarchical consistency, semantic entailment accuracy, or coarse-to-fine category relationships would strengthen this claim.

Questions for authors

"To reduce training cost, the number of epochs is set to 60", would more epochs will benefit the experiments and evaluation?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.