- Rating
- 2(+1)
- Confidence
- 4
- Informativeness
- 3
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes AURORA, a hyperbolic vision–language pretraining framework that separates semantic alignment from hierarchical ordering. It aligns matched image–text representations using their root angles, leaving their radial positions available to encode specificity. For hierarchical supervision, it introduces the Radial–Angular Order Cone (RAOC), which defines entailment based on both the parent–child radial gap and their angular separation. The paper further uses hierarchical entailment contrastive (HEC) training to encourage annotated parent–child relations and suppress spurious relations between unmatched pairs, together with a norm regularizer to stabilize embedding magnitudes.
Critical strengths
- A clear separation of angular and radial information. : The paper assigns semantic similarity to angular direction and hierarchical specificity to radial position. This is a compelling way to address the tension between cross-modal alignment and hierarchy modeling in hyperbolic space, and could be useful beyond the particular framework studied here.
- Contrastive supervision for entailment. : HEC explicitly uses both positive and negative pairs, encouraging valid hierarchical relations while penalizing spurious ones. This is a meaningful extension to approaches that supervise positive entailment relations without directly constraining unmatched pairs.
Critical weaknesses
-
The baseline comparisons are difficult to interpret and may not be reliable. : The reported results for several baselines, including HyCoCLIP, PHyCLIP and all other baselines appear SUBSTANTIALLY LOWER than their published results. ( For example, on SUN397 classification, HyCoCLIP reports 52.5% / 57.2% for ViT-S / ViT-B, while on AURORA it reports 47.88% / 52.63%. The paper reports 50.27% / 55.51% on AURORA, which is lower than HyCoCLIP’s results. Similar performance drops appear across the other baselines. ) This is particularly concerning because, in my own reproduction of HyCoCLIP, I obtained results close to those reported in its original paper. The authors should explain why they retrained these baselines rather than using their publicly available pretrained checkpoints and provide a detailed comparison of the reproduction settings, including batch size and training duration. At a minimum, the paper should report the originally published results alongside its own results, clearly distinguishing reproduced from reported numbers. Without these checks, it is difficult to determine whether AURORA’s gains come from the proposed method or from differences in training and evaluation. Please clarify why these results are presented as evidence of improved performance.
-
The comparison set and similarity metric need stronger justification. : The paper should include UNCHA [1] as a relevant baseline, since it also aligns box-level representations and uses norm regularization, albeit with an entropy-based formulation. In addition, AURORA evaluates similarity using root-angle similarity, while other models may use different metrics. The paper should analyze whether root-angle similarity itself accounts for some of the gains, compare it directly with alternative similarity metrics, and explain why it is particularly suitable for AURORA and the evaluated tasks.
[1] Kim, Hayeon, et al. "Uncertainty-guided Compositional Alignment with Part-to-Whole Semantic Representativeness in Hyperbolic Vision-Language Models." CVPR (2026).
Questions for authors
-
Figure 5 shows a substantially larger radial gap in AURORA with the cone than without it. : What causes this difference? Could the authors analyze whether it reflects the intended hierarchical organization learned by RAOC, or whether it is partly due to changes in embedding norms or other training effects?
-
The sensitivity analysis appears to report results only for ImageNet and Food-101. : Why were these two datasets selected? Could the authors report sensitivity results across additional representative task types (for example, full benchmark for classification datasets, etc ) to show whether the method’s conclusions hold beyond these two classification datasets?
9 comments
W1: “At a minimum, the paper should report the originally published results alongside its own results, clearly distinguishing reproduced from reported numbers.” “Without these checks, it is difficult to determine whether AURORA’s gains come from the proposed method or from differences in training and evaluation.”
R: We have added the original published results alongside our reproductions in the revised appendix, explicitly recording the available GRIT training-set size for each experiment. Table a summarizes the six representative metrics used in our sensitivity analysis: zero-shot ImageNet and Food-101 classification, COCO text-to-image and image-to-text Recall@5, and WordNet TIE and Jaccard similarity. All rows use ViT-B/16. The table distinguishes originally published results, reproductions reported by PHyCLIP [3], and our reproductions trained on the 10.3M-pair GRIT subset available for this submission.
Table a. Published and reproduced results for HyCoCLIP and PHyCLIP with ViT-B/16.
| Model | source | GRIT samples (M) | ImageNet ↑ | Food-101 ↑ | COCO T→I R@5 ↑ | COCO I→T R@5 ↑ | TIE ↓ | J ↑ |
|---|---|---|---|---|---|---|---|---|
| HyCoCLIP | Published [2] | 20.5 | 45.80 | 59.20 | 58.40 | 72.00 | 3.170 | 0.8100 |
| HyCoCLIP | Published [3] | 14.0 | 43.80 | 52.64 | 57.11 | 69.51 | 3.319 | 0.8043 |
| HyCoCLIP | Our reproduction | 10.3 | 40.51 | 51.82 | 55.12 | 69.84 | 3.483 | 0.7912 |
| PHyCLIP | Published [3] | 14.0 | 44.31 | 57.26 | 58.03 | 70.94 | 3.294 | 0.8059 |
| PHyCLIP | Our reproduction | 10.3 | 41.63 | 52.74 | 55.83 | 70.72 | 3.407 | 0.7954 |
| AURORA | Our reproduction | 10.3 | 46.43 | 55.71 | 59.41 | 73.82 | 3.176 | 0.8129 |
Table a shows consistent performance declines across several major metrics as the available GRIT training data decreases. For HyCoCLIP, ImageNet accuracy decreases from 45.80% to 43.80% and 40.51%, while COCO text-to-image Recall@5 decreases from 58.40 to 57.11 and 55.12. The same trend is observed for PHyCLIP when comparing its published 14.0M-pair result with our reproduction on 10.3M pairs. The table also makes individual non-monotonic measurements visible, including HyCoCLIP's COCO image-to-text recall.
To check evaluation fidelity independently of retraining, we evaluated the released HyCoCLIP ViT-B/16 checkpoint with our evaluation pipeline and obtained 45.77% on ImageNet and 57.41% on SUN397, against its published 45.80% and 57.20%. On the common 10.3M subset, the controlled comparisons in our original Tables 1–4 isolate the learning-objective effect. AURORA with ViT-B/16 reaches 46.43% ImageNet and 59.41%/73.82% COCO R@5, competitive even with the originally published 20.5M HyCoCLIP checkpoint (45.80% and 58.40%/72.00%). Its 16-dataset mean is 44.80%, versus 44.82% for that checkpoint. These checks support both the fidelity of our evaluation pipeline and the comparative value of the controlled pretraining experiments.
W2: “The paper should include UNCHA [1] as a relevant baseline, since it also aligns box-level representations and uses norm regularization, albeit with an entropy-based formulation.”
R: We have included UNCHA [10] in the revised main comparison as requested. The original submission already discussed it in Appendix E under “Uncertainty-guided compositional alignment,” including its conceptual relation to AURORA and the difficulties encountered during our reproduction. UNCHA uses norm-derived uncertainty to estimate how representative a crop is of its parent image. This uncertainty reweights Lorentz-distance contrastive alignment and calibrates an entropy-regularized entailment term. AURORA uses root-angle similarity for semantic matching and a Radial–Angular Order Cone (RAOC) to learn directional parent–child order from relative radial depth and root angle. Hierarchical entailment contrastive (HEC) supervision further distinguishes annotated relations from unmatched parent–candidate pairs. These approaches differ both in alignment scoring and in how part–whole supervision shapes the representation.
The absence of UNCHA from our original controlled tables arose from limitations in the publicly released training code. In the accessible repository [11], the training data mapper lacks classes required by the training configuration, and other training interfaces differ from the paper's description. We reconstructed the missing components using the published methodology and surrounding code, then repeated training three times on the available 10.3M-pair GRIT subset. The released training path left the exact fidelity of this reconstruction unresolved, so the original submission only discussed it in Appendix E. We contacted the authors to request clarification and have not received a response. Table c now reports our three-run results alongside the published UNCHA measurements, with the distinct data settings stated explicitly.
Table c. UNCHA results and controlled baselines on ViT-S/16. The three-run UNCHA reproduction reports mean ± standard deviation.
| Model | GRIT pairs | ImageNet ↑ | COCO T→I R@5 ↑ | COCO I→T R@5 ↑ | TIE ↓ | J ↑ |
|---|---|---|---|---|---|---|
| UNCHA [10], published | Nominal 20.5M | 43.90 | 56.20 | 69.90 | 3.39 | 0.80 |
| UNCHA, our reproduction (3 runs) | 10.3M | 37.47 | 52.41 | 65.63 | 3.93 | 0.76 |
| HyCoCLIP, our reproduction | 10.3M | 36.38 | 51.35 | 64.84 | 3.97 | 0.76 |
| AURORA | 10.3M | 41.54 | 55.77 | 69.42 | 3.62 | 0.78 |
Under our common pretraining corpus, the reconstructed UNCHA improves on the reproduced HyCoCLIP ImageNet result (37.47% versus 36.38%), while AURORA achieves 41.54%. The revised paper now discusses UNCHA in the main text and distinguishes its published results from our reconstruction and the associated implementation limitations.
[10] Kim H, Jang J H, Kim J J, Chun S Y. Uncertainty-guided Compositional Alignment with Part-to-Whole Semantic Representativeness in Hyperbolic Vision-Language Models[C]//CVPR. 2026: 36861-36870.
[11] Kim H, et al. UNCHA official code repository[EB/OL]. https://github.com/jeeit17/UNCHA.
R: We retrained the baselines because the availability of the training data, that is, GRIT images, has declined as the original source links expired. GRIT, introduced in Kosmos-2 [1], distributes image URLs, captions, and grounded phrase–box annotations. Acquiring its training images depends on the continued availability of those URLs. In fact, researchers using the same public metadata at different times obtain training sets of different sizes.
This is a shared difficulty in the research community. HyCoCLIP [2] originally trained on 20.5 million image–text pairs and 35.9 million box annotations. PHyCLIP [3] explicitly reports obtaining only 14.0 million pairs and 26.6 million boxes because many image URLs had expired(Section 4.1 in its paper). In the HyCoCLIP GitHub repository, Issue #3 [4] documents download success rates of approximately 50–60%, while Issue #2 [5] requests a complete training archive and has received no reply. A January 2025 GRIT discussion [6] reports about 15.3 million downloadable images from 20.5 million metadata entries. OpenMMLab's Grounding DINO preparation instructions also contain a processed GRIT9M configuration [7].
We have tried our best to obtain as much of GRIT as possible. We downloaded the data in three rounds, checked the recovered images and grounding annotations, removed duplicates, and merged the valid samples. The resulting training corpus contains 10.3 million image–text pairs and 17.8 million box annotations. Expired source URLs were outside our control.
For a fair comparison, we retrained CLIP [8], MERU [9], HyCoCLIP, PHyCLIP, and the other reproducible baselines on this available GRIT subset, applying the same training budget to AURORA. Following HyCoCLIP and PHyCLIP [2,3], the global batch size was 768 over 500,000 optimization steps; we used AdamW with (β1,β2)=(0.9,0.98), peak learning rate 5×10−4, weight decay 0.2, 4,000 warm-up steps, cosine learning-rate decay, 224×224 input images, and random resized crops with scale [0.5,1.0]. Appendix F gives the complete configuration. We have documented the data acquisition and controlled-training protocol in the revised manuscript. The anonymous training and evaluation resources are available in a separate comment visible only to the reviewers and Area Chairs.
[1] Peng Z, Wang W, Dong L, et al. Kosmos-2: Grounding Multimodal Large Language Models to the World[C]//ICLR. 2024.
[2] Pal A, van Spengler M, D'Amely di Melendugno G M, et al. Compositional Entailment Learning for Hyperbolic Vision-Language Models[C]//ICLR. 2025.
[3] Yoshikawa D, Matsubara T. PHyCLIP: ℓ1-Product of Hyperbolic Factors Unifies Hierarchy and Compositionality in Vision-Language Representation Learning[C]//ICLR. 2026.
[4] HyCoCLIP GitHub Repository. Issue #3: GRIT image download failures[EB/OL]. 2025. https://github.com/PalAvik/hycoclip/issues/3.
[5] HyCoCLIP GitHub Repository. Issue #2: Complete GRIT dataset request[EB/OL]. 2025. https://github.com/PalAvik/hycoclip/issues/2.
[6] GRIT Dataset Community. Discussion #2: GRIT image download availability[EB/OL]. 2025. https://huggingface.co/datasets/zzliang/GRIT/discussions/2.
[7] OpenMMLab. Grounding DINO dataset preparation documentation[EB/OL]. https://github.com/open-mmlab/mmdetection/blob/main/configs/mm_grounding_dino/dataset_prepare.md.
[8] Radford A, Kim J W, Hallacy C, et al. Learning Transferable Visual Models from Natural Language Supervision[C]//ICML. 2021: 8748-8763.
[9] Desai K, Nickel M, Rajpurohit T, et al. Hyperbolic Image-Text Representations[C]//ICML. 2023: 7694-7731.