- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper revisits the role of supervision from rendered images in trimodal 3D contrastive pretraining and proposes HyperCL, which uses only text supervision for cross-modal alignment. The method maps point cloud and text representations into hyperbolic space and combines a contrastive objective based on geodesic distance with a batch-wise entailment loss. These objectives aim to capture asymmetric hierarchical relationships between general textual concepts and specific 3D instances. The authors also introduce c-OLVIS, a manually verified version of Objaverse-LVIS, and report improvements on zero-shot classification and cross-modal retrieval.
Critical strengths
- Reassessing the necessity of supervision from rendered images addresses a relevant practical question in 3D pretraining. The method builds on an existing point cloud backbone and a frozen text encoder, with training costs comparable to those of the baselines.
- The reported improvements span multiple datasets and pretraining settings. When LVIS objects are excluded from pretraining, HyperCL achieves a text-to-3D retrieval R@1 of 25.2%, compared with 21.9% for TAMM, the strongest baseline for this metric, yielding a gain of 3.3 percentage points (Table 4).
- The method framework and loss functions are presented clearly and are easy to follow. The exploration of hyperbolic geometry for modeling cross-modal hierarchical relationships may also provide useful insights for the broader multimodal representation learning community.
Critical weaknesses
- The improvements in Table 5 do not establish that the underlying semantic structure is tree-like, since the comparison changes both feature normalization and the similarity function. Part of the gain could therefore arise from feature norms or score scaling. Similarly, the modality separation in Figure 4 does not by itself demonstrate a semantic hierarchy, and the sofa → loveseat transition in Figure 5 is inconsistent with progressively greater abstraction. The claims concerning hierarchy preservation and its contribution to the performance gains should be qualified.
- In Table 1, removing image supervision decreases OLVIS accuracy from 46.9% to 45.5%, while increasing MN40 accuracy only slightly, from 84.2% to 84.5%. These results provide limited support for a general claim that image supervision harms representation learning. Moreover, this diagnostic experiment uses a different encoder configuration from the main method, leaving the effect of image supervision insufficiently tested under the main experimental setup.
- The method assumes that a general textual concept can subsume multiple 3D instances, yet Equation (8) pushes every unpaired shape outside the text’s entailment cone. For example, if two shapes in a batch share the caption “a chair,” the frozen text encoder produces identical cones. Each shape is then encouraged to lie inside that cone for its positive pair and outside the same cone as a negative for the other pair. These incompatible constraints may favor unique instance matching, and their prevalence and impact on the intended hierarchical representation remain unexamined.
Questions for authors
- Could the authors include an unnormalized Euclidean distance baseline with comparable temperature tuning? Could they also evaluate the learned hierarchy against verified semantic relationships, using measures such as ancestor prediction accuracy or the monotonicity of abstraction along retrieval paths?
- Under the same backbone, initialization, and training budget, how do the four combinations of Euclidean/hyperbolic geometry and text-only/image–text supervision compare? Under which conditions does removing image supervision consistently improve performance?
- How common are duplicate or semantically similar captions that create semantic false negatives within training batches? Could the authors compare entailment using only positive pairs, the proposed batch-wise loss, and a version that masks semantic false negatives, to assess the contribution and semantic consistency of the negative term?
If the authors can address most of my concerns, I will raise my score.