acceptodds
Under review as a conference paper at ICLR 2027

AURORA: Separating Angular Alignment from Radial Ordering in Hyperbolic Vision–Language Models

11 views

Abstract

Hyperbolic vision-language models are well suited to modeling the hierarchical structure of multimodal data, because hyperbolic volume grows exponentially with radius, enabling low-distortion representations of tree-like taxonomies from general to specific concepts. However, existing pretraining objectives for hyperbolic vision-language models entangle semantic alignment with entailment modeling, forcing radial coordinates to simultaneously encode semantic matching and hierarchical structure, leading to conflicting geometric constraints. In this paper, we introduce AURORA, a hyperbolic vision-language framework that decouples the objectives through angular alignment and a novel Radial-Angular Order Cone(RAOC) for entailment modeling. Angular alignment captures semantic similarity while preserving the radial structure for representing hierarchies. Unlike conventional entailment cones, RAOC determines angular allowance from relative parent-child radial gaps. It is incorporated into hierarchical entailment contrastive training to capture partial-order relations. We further introduce a norm regularization term that prevents undesirable radial degeneration for stable training. We train models on millions of image-text pairs and evaluate it across zero-shot classification, image-text retrieval, hierarchical reasoning, and compositional understanding. Our model consistently outperforms strong Euclidean and hyperbolic vision-language baselines, demonstrating the effectiveness of decoupling semantic alignment from hierarchical entailment modeling.

Primary area:
learning on graphs and other geometries & topologies
Keywords:
hyperbolic geometry, modality alignment, multimodal learning

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_USBGOpenReview ↗
Rating
2(+1)
Confidence
4
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes AURORA, a hyperbolic vision–language pretraining framework that separates semantic alignment from hierarchical ordering. It aligns matched image–text representations using their root angles, leaving their radial positions available to encode specificity. For hierarchical supervision, it introduces the Radial–Angular Order Cone (RAOC), which defines entailment based on both the parent–child radial gap and their angular separation. The paper further uses hierarchical entailment contrastive (HEC) training to encourage annotated parent–child relations and suppress spurious relations between unmatched pairs, together with a norm regularizer to stabilize embedding magnitudes.

Critical strengths
  1. A clear separation of angular and radial information. : The paper assigns semantic similarity to angular direction and hierarchical specificity to radial position. This is a compelling way to address the tension between cross-modal alignment and hierarchy modeling in hyperbolic space, and could be useful beyond the particular framework studied here.
  2. Contrastive supervision for entailment. : HEC explicitly uses both positive and negative pairs, encouraging valid hierarchical relations while penalizing spurious ones. This is a meaningful extension to approaches that supervise positive entailment relations without directly constraining unmatched pairs.
Critical weaknesses
  1. The baseline comparisons are difficult to interpret and may not be reliable. : The reported results for several baselines, including HyCoCLIP, PHyCLIP and all other baselines appear SUBSTANTIALLY LOWER than their published results. ( For example, on SUN397 classification, HyCoCLIP reports 52.5% / 57.2% for ViT-S / ViT-B, while on AURORA it reports 47.88% / 52.63%. The paper reports 50.27% / 55.51% on AURORA, which is lower than HyCoCLIP’s results. Similar performance drops appear across the other baselines. ) This is particularly concerning because, in my own reproduction of HyCoCLIP, I obtained results close to those reported in its original paper. The authors should explain why they retrained these baselines rather than using their publicly available pretrained checkpoints and provide a detailed comparison of the reproduction settings, including batch size and training duration. At a minimum, the paper should report the originally published results alongside its own results, clearly distinguishing reproduced from reported numbers. Without these checks, it is difficult to determine whether AURORA’s gains come from the proposed method or from differences in training and evaluation. Please clarify why these results are presented as evidence of improved performance.

  2. The comparison set and similarity metric need stronger justification. : The paper should include UNCHA [1] as a relevant baseline, since it also aligns box-level representations and uses norm regularization, albeit with an entropy-based formulation. In addition, AURORA evaluates similarity using root-angle similarity, while other models may use different metrics. The paper should analyze whether root-angle similarity itself accounts for some of the gains, compare it directly with alternative similarity metrics, and explain why it is particularly suitable for AURORA and the evaluated tasks.

[1] Kim, Hayeon, et al. "Uncertainty-guided Compositional Alignment with Part-to-Whole Semantic Representativeness in Hyperbolic Vision-Language Models." CVPR (2026).

Questions for authors
  1. Figure 5 shows a substantially larger radial gap in AURORA with the cone than without it. : What causes this difference? Could the authors analyze whether it reflects the intended hierarchical organization learned by RAOC, or whether it is partly due to changes in embedding norms or other training effects?

  2. The sensitivity analysis appears to report results only for ImageNet and Food-101. : Why were these two datasets selected? Could the authors report sensitivity results across additional representative task types (for example, full benchmark for classification datasets, etc ) to show whether the method’s conclusions hold beyond these two classification datasets?

9 comments

W1: “The baseline comparisons are difficult to interpret and may not be reliable.” “The authors should explain why they retrained these baselines rather than using their publicly available pretrained checkpoints and provide a detailed comparison of the reproduction settings, including batch size and training duration.”

R: We retrained the baselines because the availability of the training data, that is, GRIT images, has declined as the original source links expired. GRIT, introduced in Kosmos-2 [1], distributes image URLs, captions, and grounded phrase–box annotations. Acquiring its training images depends on the continued availability of those URLs. In fact, researchers using the same public metadata at different times obtain training sets of different sizes.

This is a shared difficulty in the research community. HyCoCLIP [2] originally trained on 20.5 million image–text pairs and 35.9 million box annotations. PHyCLIP [3] explicitly reports obtaining only 14.0 million pairs and 26.6 million boxes because many image URLs had expired(Section 4.1 in its paper). In the HyCoCLIP GitHub repository, Issue #3 [4] documents download success rates of approximately 50–60%, while Issue #2 [5] requests a complete training archive and has received no reply. A January 2025 GRIT discussion [6] reports about 15.3 million downloadable images from 20.5 million metadata entries. OpenMMLab's Grounding DINO preparation instructions also contain a processed GRIT9M configuration [7].

We have tried our best to obtain as much of GRIT as possible. We downloaded the data in three rounds, checked the recovered images and grounding annotations, removed duplicates, and merged the valid samples. The resulting training corpus contains 10.3 million image–text pairs and 17.8 million box annotations. Expired source URLs were outside our control.

For a fair comparison, we retrained CLIP [8], MERU [9], HyCoCLIP, PHyCLIP, and the other reproducible baselines on this available GRIT subset, applying the same training budget to AURORA. Following HyCoCLIP and PHyCLIP [2,3], the global batch size was 768 over 500,000 optimization steps; we used AdamW with (β1,β2)=(0.9,0.98)(\beta_1,\beta_2)=(0.9,0.98), peak learning rate 5×10−45\times10^{-4}, weight decay 0.2, 4,000 warm-up steps, cosine learning-rate decay, 224×224224\times224 input images, and random resized crops with scale [0.5,1.0][0.5,1.0]. Appendix F gives the complete configuration. We have documented the data acquisition and controlled-training protocol in the revised manuscript. The anonymous training and evaluation resources are available in a separate comment visible only to the reviewers and Area Chairs.

[1] Peng Z, Wang W, Dong L, et al. Kosmos-2: Grounding Multimodal Large Language Models to the World[C]//ICLR. 2024.
[2] Pal A, van Spengler M, D'Amely di Melendugno G M, et al. Compositional Entailment Learning for Hyperbolic Vision-Language Models[C]//ICLR. 2025.
[3] Yoshikawa D, Matsubara T. PHyCLIP: ℓ1\ell_1-Product of Hyperbolic Factors Unifies Hierarchy and Compositionality in Vision-Language Representation Learning[C]//ICLR. 2026.
[4] HyCoCLIP GitHub Repository. Issue #3: GRIT image download failures[EB/OL]. 2025. https://github.com/PalAvik/hycoclip/issues/3.
[5] HyCoCLIP GitHub Repository. Issue #2: Complete GRIT dataset request[EB/OL]. 2025. https://github.com/PalAvik/hycoclip/issues/2.
[6] GRIT Dataset Community. Discussion #2: GRIT image download availability[EB/OL]. 2025. https://huggingface.co/datasets/zzliang/GRIT/discussions/2.
[7] OpenMMLab. Grounding DINO dataset preparation documentation[EB/OL]. https://github.com/open-mmlab/mmdetection/blob/main/configs/mm_grounding_dino/dataset_prepare.md.
[8] Radford A, Kim J W, Hallacy C, et al. Learning Transferable Visual Models from Natural Language Supervision[C]//ICML. 2021: 8748-8763.
[9] Desai K, Nickel M, Rajpurohit T, et al. Hyperbolic Image-Text Representations[C]//ICML. 2023: 7694-7731.

W1: “At a minimum, the paper should report the originally published results alongside its own results, clearly distinguishing reproduced from reported numbers.” “Without these checks, it is difficult to determine whether AURORA’s gains come from the proposed method or from differences in training and evaluation.”

R: We have added the original published results alongside our reproductions in the revised appendix, explicitly recording the available GRIT training-set size for each experiment. Table a summarizes the six representative metrics used in our sensitivity analysis: zero-shot ImageNet and Food-101 classification, COCO text-to-image and image-to-text Recall@5, and WordNet TIE and Jaccard similarity. All rows use ViT-B/16. The table distinguishes originally published results, reproductions reported by PHyCLIP [3], and our reproductions trained on the 10.3M-pair GRIT subset available for this submission.

Table a. Published and reproduced results for HyCoCLIP and PHyCLIP with ViT-B/16.

ModelsourceGRIT samples (M)ImageNet ↑Food-101 ↑COCO T→I R@5 ↑COCO I→T R@5 ↑TIE ↓J ↑
HyCoCLIPPublished [2]20.545.8059.2058.4072.003.1700.8100
HyCoCLIPPublished [3]14.043.8052.6457.1169.513.3190.8043
HyCoCLIPOur reproduction10.340.5151.8255.1269.843.4830.7912
PHyCLIPPublished [3]14.044.3157.2658.0370.943.2940.8059
PHyCLIPOur reproduction10.341.6352.7455.8370.723.4070.7954
AURORAOur reproduction10.346.4355.7159.4173.823.1760.8129

Table a shows consistent performance declines across several major metrics as the available GRIT training data decreases. For HyCoCLIP, ImageNet accuracy decreases from 45.80% to 43.80% and 40.51%, while COCO text-to-image Recall@5 decreases from 58.40 to 57.11 and 55.12. The same trend is observed for PHyCLIP when comparing its published 14.0M-pair result with our reproduction on 10.3M pairs. The table also makes individual non-monotonic measurements visible, including HyCoCLIP's COCO image-to-text recall.

To check evaluation fidelity independently of retraining, we evaluated the released HyCoCLIP ViT-B/16 checkpoint with our evaluation pipeline and obtained 45.77% on ImageNet and 57.41% on SUN397, against its published 45.80% and 57.20%. On the common 10.3M subset, the controlled comparisons in our original Tables 1–4 isolate the learning-objective effect. AURORA with ViT-B/16 reaches 46.43% ImageNet and 59.41%/73.82% COCO R@5, competitive even with the originally published 20.5M HyCoCLIP checkpoint (45.80% and 58.40%/72.00%). Its 16-dataset mean is 44.80%, versus 44.82% for that checkpoint. These checks support both the fidelity of our evaluation pipeline and the comparative value of the controlled pretraining experiments.

W2: “The paper should include UNCHA [1] as a relevant baseline, since it also aligns box-level representations and uses norm regularization, albeit with an entropy-based formulation.”

R: We have included UNCHA [10] in the revised main comparison as requested. The original submission already discussed it in Appendix E under “Uncertainty-guided compositional alignment,” including its conceptual relation to AURORA and the difficulties encountered during our reproduction. UNCHA uses norm-derived uncertainty to estimate how representative a crop is of its parent image. This uncertainty reweights Lorentz-distance contrastive alignment and calibrates an entropy-regularized entailment term. AURORA uses root-angle similarity for semantic matching and a Radial–Angular Order Cone (RAOC) to learn directional parent–child order from relative radial depth and root angle. Hierarchical entailment contrastive (HEC) supervision further distinguishes annotated relations from unmatched parent–candidate pairs. These approaches differ both in alignment scoring and in how part–whole supervision shapes the representation.

The absence of UNCHA from our original controlled tables arose from limitations in the publicly released training code. In the accessible repository [11], the training data mapper lacks classes required by the training configuration, and other training interfaces differ from the paper's description. We reconstructed the missing components using the published methodology and surrounding code, then repeated training three times on the available 10.3M-pair GRIT subset. The released training path left the exact fidelity of this reconstruction unresolved, so the original submission only discussed it in Appendix E. We contacted the authors to request clarification and have not received a response. Table c now reports our three-run results alongside the published UNCHA measurements, with the distinct data settings stated explicitly.

Table c. UNCHA results and controlled baselines on ViT-S/16. The three-run UNCHA reproduction reports mean ± standard deviation.

ModelGRIT pairsImageNet ↑COCO T→I R@5 ↑COCO I→T R@5 ↑TIE ↓J ↑
UNCHA [10], publishedNominal 20.5M43.9056.2069.903.390.80
UNCHA, our reproduction (3 runs)10.3M37.4752.4165.633.930.76
HyCoCLIP, our reproduction10.3M36.3851.3564.843.970.76
AURORA10.3M41.5455.7769.423.620.78

Under our common pretraining corpus, the reconstructed UNCHA improves on the reproduced HyCoCLIP ImageNet result (37.47% versus 36.38%), while AURORA achieves 41.54%. The revised paper now discusses UNCHA in the main text and distinguishes its published results from our reconstruction and the associated implementation limitations.

[10] Kim H, Jang J H, Kim J J, Chun S Y. Uncertainty-guided Compositional Alignment with Part-to-Whole Semantic Representativeness in Hyperbolic Vision-Language Models[C]//CVPR. 2026: 36861-36870.

[11] Kim H, et al. UNCHA official code repository[EB/OL]. https://github.com/jeeit17/UNCHA.

Reviewer_MJ3COpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies the optimization conflict between semantic alignment and hierarchical modeling in hyperbolic vision–language models. The authors propose AURORA, which uses root-angle alignment for semantic matching and a Radial–Angular Order Cone (RAOC) to define hierarchical relations through relative radial gaps and angular separation. Hierarchical Entailment Contrastive (HEC) training incorporates negative pairs, while norm regularization stabilizes the radial scale. The authors prove that RAOC defines a partial order and evaluate the method on classification, retrieval, hierarchical classification, and compositional understanding using two visual backbones.

Critical strengths
  1. The motivation and geometric design are clearly connected. The decomposition of hyperbolic distance explains how semantic alignment can reduce the radial separation needed for hierarchical modeling. The partial-order and descendant-set nesting properties of RAOC have clear mathematical support, providing a sound basis for interpreting the design.
  2. The evaluation is broad and shows consistent gains across multiple tasks. Comparisons include Euclidean and hyperbolic baselines, with classification and retrieval improvements on both ViT-S/16 and ViT-B/16. Ablations provide useful comparisons that support the effectiveness of the overall training framework.
Critical weaknesses

Direct validation of the learned hierarchical relations is limited. The WordNet metrics measure how close predicted labels are to ground-truth labels in the taxonomy. Improvements may therefore result from higher classification accuracy, without directly showing whether general concepts lie closer to the origin or whether parent–child relations are correctly captured. The visualizations show selected examples, but a broader quantitative evaluation of these relations is missing.

Questions for authors
  1. Could the authors directly evaluate radial ordering and parent–child relation accuracy on examples with known hierarchical relations, addressing Weakness 1?
  2. Could the authors compare positive-only hierarchical supervision with full HEC while keeping RAOC, root-angle alignment, and norm regularization unchanged? This would help determine whether negative pairs reduce incorrect hierarchical relations.
4 comments

W1: Direct validation of the learned hierarchical relations is limited. The WordNet metrics measure how close predicted labels are to ground-truth labels in the taxonomy. Improvements may therefore result from higher classification accuracy, without directly showing whether general concepts lie closer to the origin or whether parent–child relations are correctly captured. The visualizations show selected examples, but a broader quantitative evaluation of these relations is missing.

Q1: Could the authors directly evaluate radial ordering and parent–child relation accuracy on examples with known hierarchical relations, addressing Weakness 1?

R: We have evaluated the learned hierarchy through two additional protocols: taxonomic open-set (TOS) classification across multiple levels of class hierarchies and direct geometric testing of annotated parent–child relations. These evaluations assess hierarchical consistency and radial order at the representation level, complementing the WordNet metrics computed from leaf-label predictions in the original paper.

TOS classification, introduced by ProTeCt [1] and applied to hyperbolic vision–language models by Wu et al. [2], evaluates recognition at several levels of a predefined taxonomy. For an image whose ground-truth label is a leaf (e.g., a dog breed), the corresponding correct labels at higher levels are its ancestors (dog, mammal, animal). Leaf Accuracy (LA) is ordinary top-1 classification accuracy at the leaf level. Hierarchically Consistent Accuracy (HCA) counts an image as correct only when its predictions agree with the ground-truth ancestor path at every evaluated level. Mean Treecut Accuracy (MTA) averages prediction accuracy over the taxonomic tree cuts, measuring how well the model recognizes concepts as the label granularity changes. CIFAR-100, SUN, and ImageNet taxonomies were never used to supervise the GRIT pretraining. Table a reports zero-shot results for the three datasets.

Table a. Zero-shot taxonomic open-set classification (%). Each entry is LA/HCA/MTA; the HyCoCLIP-B row is the released checkpoint evaluated by Wu et al. [2], and ViT-S/16 models use 10.3M GRIT pairs.

ModelCIFAR-100 LA/HCA/MTA ↑SUN LA/HCA/MTA ↑ImageNet LA/HCA/MTA ↑
HyCoCLIP-B, published60.21/9.31/42.9658.40/26.52/48.9845.74/0.98/38.82
HyCoCLIP-S, reproduced47.91/8.47/36.4647.88/14.75/38.7236.38/0.52/32.67
AURORA-S (w/o RAOC)49.34/9.76/43.1848.94/16.61/41.8840.27/1.25/42.29
AURORA-S51.20/12.47/48.7650.27/21.43/46.7641.54/5.94/50.75

AURORA's Radial–Angular Order Cone (RAOC), trained through hierarchical entailment contrastive (HEC) supervision, increases LA by 1.86/1.33/1.27 points on CIFAR-100/SUN/ImageNet, whereas HCA increases by 2.71/4.82/4.69 and MTA by 5.58/4.88/8.46. The larger gains in multi-level consistency indicate improved hierarchical recognition alongside the gains in leaf accuracy.

We also evaluated the four annotated image, caption, crop, and phrase relation types on 5,000 held-out images. Correct radial order requires the parent to lie closer to the Lorentz origin than its child. Hard RAOC membership additionally requires ρchild−ρparent≥θ/β\rho_{\rm child}-\rho_{\rm parent}\ge\theta/\beta, where θ\theta is the root angle and the default β=8\beta=8. The two models use the same relation labels and evaluation protocol.

Table b. Direct radial-precedence and hard parent–child cone membership (%), ViT-S/16.

ModelCorrect radial precedence ↑Matched pairs inside hard RAOC ↑
AURORA-S (w/o RAOC)15.460.18
AURORA-S84.220.60

As shown in Table b, RAOC improves the correct parent–child radial ordering by 68.76 percentage points. Hard membership is a stricter geometric event because it also requires sufficient radial gap for the observed angle; its lower rate is consistent with the continuous, sigmoid-calibrated HEC training objective. Tables a and b jointly show improvements in hierarchical decisions and direct geometric organization across held-out examples. We have added the protocols and results to the revised appendix.

[1] Wu T Y, Ho C H, Vasconcelos N. ProTeCt: Prompt Tuning for Taxonomic Open Set Classification[C]//CVPR. 2024: 16531-16540.
[2] Wu W, Fan X, Wu Y, et al. Modality Alignment across Trees on Heterogeneous Hyperbolic Manifolds[C]//ICLR. 2026.

Q2: Could the authors compare positive-only hierarchical supervision with full HEC while keeping RAOC, root-angle alignment, and norm regularization unchanged? This would help determine whether negative pairs reduce incorrect hierarchical relations.

R: We performed the requested controlled ablation with ViT-S/16. Both runs use 10.3 million GRIT image–text pairs, 17.8 million box annotations, 500,000 optimization steps, the same root-angle alignment loss, RAOC geometry, norm regularizer, and all hyperparameters. The positive-only version supervises the annotated parent–child relations. Full hierarchical entailment contrastive (HEC) training also penalizes unmatched in-batch parent–candidate pairs as negative entailment examples. Table c isolates this difference.

Table c. Effect of negative entailment supervision under the same RAOC geometry, ViT-S/16.

ObjectiveImageNet ↑Food-101 ↑COCO T→I R@5 ↑COCO I→T R@5 ↑TIE ↓J ↑
RAOC, positive-only38.3046.1253.6867.093.800.77
RAOC + full HEC41.5450.1355.7769.423.620.78

As shown in Table c, negative entailment supervision raises ImageNet and Food-101 accuracy by 3.24 and 4.01 percentage points, improves COCO text-to-image/image-to-text Recall@5 by 2.09/2.33, and reduces WordNet TIE by 0.18. This improvement is obtained with the order region and all other training settings held fixed. The original manuscript's Table 4(a) reports 38.12% ImageNet and 3.79 TIE for positive-only conventional entailment-cone supervision, and Table 4(d) reports 40.27% and 3.74 without the order objective. The full HEC variant surpasses both, showing the contribution of unmatched-pair supervision to an effective hierarchical objective.

The geometric reason follows from the RAOC condition g(p,q)=ρq−ρp≥θpq/βg(p,q)=\rho_q-\rho_p\ge\theta_{pq}/\beta: once g≥π/βg\ge\pi/\beta, the hard order region admits every angular direction, including unrelated candidates. An objective using only positive relations can encourage such broad admission by increasing radial separation. HEC supplies an explicit entailment-specific penalty on unmatched pairs. In our HyCoCLIP and PHyCLIP reproductions, the conventional positive-only entailment loss becomes negligible within roughly the first 1,000 of 500,000 iterations; the HEC loss decreases throughout training from approximately 7 to 1. These optimization traces and the held-out radial-order improvement in Table b are consistent with sustained relational learning under HEC.

Thank you for the additional experiments and clarifications. The results provide clear evidence of improved radial ordering. However, I share Reviewer 3aUW’s concern that only 0.60% of matched pairs satisfy the hard RAOC condition. I therefore maintain my rating of 3.

“However, I share Reviewer 3aUW’s concern that only 0.60% of matched pairs satisfy the hard RAOC condition.”

R: The key measure of a semantic entailment cone is its ability to distinguish valid parent–child relations from unrelated parent–candidate relations. This requires considering matched and unmatched hard-cone membership together. We have therefore expanded our held-out evaluation from 5,000 to 20,000 image–text examples and compared the ViT-S/16 models MERU, HyCoCLIP, PHyCLIP, and AURORA. Table d evaluates each model using its respective hard-cone membership condition on matched and unmatched pairs.

Table d. Matched and unmatched hard-cone membership on 20,000 held-out examples, ViT-S/16 (%).

ModelMatched inside cone ↑Unmatched inside cone ↓
MERU67.4697.86
HyCoCLIP100.0099.88
PHyCLIP99.98100.00
AURORA0.640.01

As shown in Table d, HyCoCLIP and PHyCLIP admit nearly all matched and unmatched pairs, while MERU admits 67.46% of matched and 97.86% of unmatched pairs. Their hard-cone membership offers little semantic discrimination. Our RAOC learns the intended relative radial order beyond this hard threshold: in our earlier held-out analysis, correct parent–child precedence rises from 15.46% without RAOC to 84.22% with RAOC. Hierarchical Entailment Contrastive (HEC) training optimizes a continuous RAOC score, thereby facilitating semantic entailment learning.

We further examined whether conventional cones and the selective RAOC provide useful training supervision. Table e compares the conventional-cone ablations of MERU, HyCoCLIP, and PHyCLIP, alongside AURORA w/o RAOC, w/ cone, and full RAOC/HEC, keeping the backbone and pretraining setup unchanged.

Table e. Effect of entailment supervision on downstream tasks with ViT-S/16. Δ is the score with the conventional cone minus the score without it for MERU, HyCoCLIP, and PHyCLIP, and the score of AURORA minus that of AURORA w/ cone for AURORA. A negative Δ indicates improvement for TIE.

ModelImageNet ↑Food-101 ↑COCO T→I R@5 ↑COCO I→T R@5 ↑TIE ↓J ↑
MERU w/o cone31.8641.2249.8365.164.430.71
MERU31.7441.0949.7965.014.440.71
HyCoCLIP w/o cone36.4146.7551.2864.783.970.76
HyCoCLIP36.3846.7351.3564.843.970.76
PHyCLIP w/o cone37.0447.2152.1665.483.920.75
PHyCLIP37.0647.2652.2465.653.920.76
AURORA w/o RAOC40.2747.0655.2968.923.740.77
AURORA w/ cone38.1245.9253.4866.723.790.76
AURORA41.5450.1355.7769.423.620.78
MERU Δ−0.12−0.13−0.04−0.15+0.010.00
HyCoCLIP Δ−0.03−0.02+0.07+0.060.000.00
PHyCLIP Δ+0.02+0.05+0.08+0.170.00+0.01
AURORA Δ+3.42+4.21+2.29+2.70−0.17+0.02

As shown in Table e, removing the conventional cone changes classification and retrieval metrics of MERU, HyCoCLIP, and PHyCLIP by no more than 0.17 points, without a consistent loss of performance. For AURORA, RAOC/HEC improves over the conventional cone by 3.42/4.21 points on ImageNet/Food-101 and 2.29/2.70 on COCO T→I/I→T R@5, while reducing TIE by 0.17. Against w/o RAOC, the gains are 1.27/3.07 and 0.48/0.50, with TIE reduced by 0.12. The combined results illustrate the central angular–radial separation of AURORA: root-angle similarity supplies semantic alignment, while relative radial depth learns a directed partial order shaped by both matched and unmatched relations. To our knowledge, this is the first contrastively supervised angular–radial separation of this form in hyperbolic VLM pretraining. We will continue exploring adaptive entailment strength to improve hard-cone positive coverage while retaining selectivity. We hope the new measurements and controlled ablation resolve the remaining concern about semantic entailment recognition.

Reviewer_3aUWOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

AURORA addresses the tension between semantic alignment and hierarchical ordering in hyperbolic vision-language pretraining. It replaces distance-based alignment with root-angle cosine similarity and introduces a Radial-Angular Order Cone (RAOC), where a candidate descendant must have sufficient radial depth relative to its angular separation from the parent. The framework combines this relation with hierarchical entailment contrastive training on matched and unmatched image, caption, crop, and phrase pairs, plus norm regularization. The paper proves partial-order and nesting properties and reports improvements on classification, retrieval, hierarchical classification, and compositional benchmarks using GRIT pretraining with ViT-S/16 and ViT-B/16.

Critical strengths
  1. Clear formulation with a useful geometric guarantee. The paper identifies a concrete source of radial pressure in distance-based alignment and presents a simple alternative. RAOC has an explicit membership rule, and the partial-order and nesting arguments in Appendix C appear sound. The broader aperture analysis and explicit discussion of non-convexity help delimit what the construction guarantees.

  2. Substantial and reasonably broad empirical improvements. The experiments cover 16 classification datasets, bidirectional retrieval, and compositional tests at two backbone sizes. Relative to ARGENT, ImageNet accuracy improves from 37.55 to 41.54 for ViT-S and from 42.31 to 46.43 for ViT-B; COCO retrieval also improves in both directions. Table 4 includes useful controls for the alignment score, cone objective, norm regularization, and loss. These results make the method a promising pretraining recipe even if its hierarchy-specific interpretation requires further validation.

Critical weaknesses
  1. The principal quantitative hierarchy evaluation does not directly validate the proposed radial order. Appendix H.3 computes WordNet metrics from ImageNet class predictions, while AURORA makes those predictions using root-angle similarity alone. These metrics can therefore improve simply because classification accuracy improves. For example, using the rounded ViT-B results in Tables 7-8 and the fact that correct predictions have zero TIE, mean TIE conditional on an incorrect prediction is approximately 3.18/(1-0.4643)=5.94 for AURORA and 3.34/(1-0.4231)=5.79 for ARGENT. This does not invalidate the classification gains, but it shows why lower overall TIE is insufficient evidence for better learned radial hierarchy. The qualitative plots do not replace a direct assessment of semantic RAOC membership or generalization to unseen ancestor-descendant relations. This gap concerns a central claim of the paper.

  2. The distinctive contribution and its empirical attribution remain insufficiently isolated. ATMG already studies the conflict between proximity alignment and hierarchy and uses an angular objective, while the norm regularizer has the same form as ARGENT's. The main novelty therefore rests on RAOC and its contrastive supervision. Yet RAOC depends only on a direction and a scalar depth: under the stated parameterization, u=z/||z|| and rho=sqrt(c)||z||. The paper does not establish what the hyperbolic parameterization contributes beyond cosine alignment with a scalar ordering constraint. Table 4 is useful and does include a no-hierarchy control: adding RAOC to the norm-regularized variant improves ImageNet from 40.27 to 41.54. However, there is no otherwise-matched positive-only versus positive-and-negative RAOC comparison to isolate HEC's claimed suppression of spurious entailment. Treating every unmatched pair as non-entailing also needs scrutiny when generic phrases can validly describe multiple examples.

Questions for authors
  1. Can the authors clarify whether the visualizations use the same scaled radial coordinate and beta as Eq. (8), and report direct positive/negative RAOC membership statistics on held-out semantic relations? In Figure 5(b), the bag text/crop point appears to have child-minus-parent radial gap about 0.0345 and angle about 1.31; with beta=8, Eq. (8) would require a gap of about 0.164. If there is a coordinate conversion, please specify it. Also, because gaps at least pi/beta admit every direction, reporting the frequency of this regime would help establish the selectivity of the learned relation. These checks would materially affect the hierarchy assessment.

  2. What role does hyperbolic geometry play beyond the equivalent direction-plus-depth formulation? An explicit analysis, or a matched scalar-depth control with the same supervision, would help distinguish the contribution of the parameterization from the ordering objective. This is a request to clarify the existing mechanism, not to extend the method to a new task.

  3. Can the contribution of negative entailment supervision be isolated using the same RAOC score with positive-only versus positive-and-negative training, and can the authors explain how repeated generic phrases or cross-example valid entailments are handled? These results would clarify whether HEC learns semantic non-entailment rather than primarily distinguishing training instances.

My current recommendation is weak rejection. Convincing direct hierarchy validation and clarification of the distinctive contribution could move it toward weak acceptance.

7 comments

W1: The principal quantitative hierarchy evaluation does not directly validate the proposed radial order. Appendix H.3 computes WordNet metrics from ImageNet class predictions, while AURORA makes those predictions using root-angle similarity alone. These metrics can therefore improve simply because classification accuracy improves. For example, using the rounded ViT-B results in Tables 7-8 and the fact that correct predictions have zero TIE, mean TIE conditional on an incorrect prediction is approximately 3.18/(1-0.4643)=5.94 for AURORA and 3.34/(1-0.4231)=5.79 for ARGENT. This does not invalidate the classification gains, but it shows why lower overall TIE is insufficient evidence for better learned radial hierarchy. The qualitative plots do not replace a direct assessment of semantic RAOC membership or generalization to unseen ancestor-descendant relations. This gap concerns a central claim of the paper.

R: The WordNet metrics adopted from HyCoCLIP [1] and PHyCLIP [2] evaluate the taxonomic distance and ancestor overlap of predicted versus correct leaf classes. The conditional mean TIE suggested in the review depends on the set of mistakes made by each model. Correcting a large number of mild errors can increase this conditional mean even when no prediction worsens. Therefore, the conditional mean TIE proposed by the reviewer is inherently sensitive to which errors remain and cannot serve as an accurate evaluation metric for comparing models. Accordingly, we have supplemented the WordNet results with multi-level taxonomic recognition and direct held-out geometric measurements. We have supplemented the WordNet results with multi-level taxonomic recognition and direct held-out geometric measurements.

The first evaluation is taxonomic open-set (TOS) classification, introduced by ProTeCt [4] and adopted for hyperbolic vision–language models by Wu et al. [5]. Given a taxonomy such as dog breed → dog → mammal → animal, the model is tested at the leaf and at multiple coarser taxonomic levels. Leaf Accuracy (LA) measures top-1 correctness of the specific leaf label. Hierarchically Consistent Accuracy (HCA) requires correct predictions along the entire ground-truth ancestor path. Mean Treecut Accuracy (MTA) averages classification accuracy across cuts through the taxonomy, capturing the ability to recognize categories at different levels of abstraction. CIFAR-100, SUN, and ImageNet taxonomies are excluded from our GRIT pretraining supervision. Table a reports zero-shot measurements.

Table a. Taxonomic open-set classification (%). Each entry is LA/HCA/MTA; HyCoCLIP-B uses the released checkpoint reported by Wu et al. [5], while ViT-S/16 models are trained on 10.3M GRIT pairs.

ModelCIFAR-100 LA/HCA/MTA ↑SUN LA/HCA/MTA ↑ImageNet LA/HCA/MTA ↑
HyCoCLIP-B, published60.21/9.31/42.9658.40/26.52/48.9845.74/0.98/38.82
HyCoCLIP-S, reproduced47.91/8.47/36.4647.88/14.75/38.7236.38/0.52/32.67
AURORA-S (w/o RAOC)49.34/9.76/43.1848.94/16.61/41.8840.27/1.25/42.29
AURORA-S51.20/12.47/48.7650.27/21.43/46.7641.54/5.94/50.75

Relative to the matched w/o-RAOC model, AURORA improves LA by 1.86/1.33/1.27 points on CIFAR-100/SUN/ImageNet, HCA by 2.71/4.82/4.69, and MTA by 5.58/4.88/8.46. The larger increases in hierarchy-wide consistency provide evidence of better hierarchical behavior alongside improved leaf classification. The next response reports a direct test of the radial order and hard RAOC relation on 5,000 held-out grounded examples.

[1] Pal A, van Spengler M, D'Amely di Melendugno G M, et al. Compositional Entailment Learning for Hyperbolic Vision-Language Models[C]//ICLR. 2025.
[2] Yoshikawa D, Matsubara T. PHyCLIP: ℓ1\ell_1-Product of Hyperbolic Factors Unifies Hierarchy and Compositionality in Vision-Language Representation Learning[C]//ICLR. 2026.
[3] Huynh C, Souri H, Kumar A, et al. ARGENT: Adaptive Hierarchical Image-Text Representations[C]//ECCV. 2026: 557-573.
[4] Wu T Y, Ho C H, Vasconcelos N. ProTeCt: Prompt Tuning for Taxonomic Open Set Classification[C]//CVPR. 2024: 16531-16540.
[5] Wu W, Fan X, Wu Y, et al. Modality Alignment across Trees on Heterogeneous Hyperbolic Manifolds[C]//ICLR. 2026.

Q1: (1)Can the authors clarify whether the visualizations use the same scaled radial coordinate and beta as Eq. (8), and report direct positive/negative RAOC membership statistics on held-out semantic relations?

(2)In Figure 5(b), the bag text/crop point appears to have child-minus-parent radial gap about 0.0345 and angle about 1.31; with beta=8, Eq. (8) would require a gap of about 0.164. If there is a coordinate conversion, please specify it.

(3)Also, because gaps at least pi/beta admit every direction, reporting the frequency of this regime would help establish the selectivity of the learned relation. These checks would materially affect the hierarchy assessment.

R(1): Both panels of Figure 5 use the same scaled radial coordinate ρ\rho and root angle θ\theta defined in Equation (8), with β=8\beta=8; there is no conversion. The horizontal axis is ρtext−ρregion\rho_{\rm text}-\rho_{\rm region}, the negative of the child-minus-parent radial gap gg. In Figure 5(b), the “bag” crop has g≈0.0345g\approx0.0345 and θ≈1.31\theta\approx1.31 radians. It satisfies the desired radial precedence g>0g>0. The hard RAOC membership rule requires g≥θ/β≈0.164g\ge\theta/\beta\approx0.164, so this crop lies outside the hard cone, as the reviewer calculated.

R(2): We directly measured the four annotated relation types—full text → full image, box phrase → image box, image box → full image, and box phrase → full text—on 5,000 held-out images with captions, region boxes, and phrases. Table b reports radial-order accuracy and strict membership using the same ρ,θ,β\rho,\theta,\beta and the same parent–candidate pairing protocol for both checkpoints. Unmatched candidate relations are drawn outside the annotated positive pair.

Table b. Direct geometric relation statistics on held-out data, AURORA ViT-S/16.

MeasurementAURORA-S (w/o RAOC)AURORA-S
Correct parent–child radial precedence ↑15.46%84.22%
Matched pairs inside hard Cβ\mathcal C_\beta ↑0.18%0.60%
Unmatched pairs inside hard Cβ\mathcal C_\beta ↓7.24%0.02%

The Radial–Angular Order Cone (RAOC) is trained through hierarchical entailment contrastive (HEC) supervision, using a sigmoid-calibrated continuous score σ(yij[aeDRAOC+be])\sigma(y_{ij}[a_eD_{\rm RAOC}+b_e]). Theorem 1 establishes the partial-order and nested-descendant properties of the hard relation DRAOC≥0D_{\rm RAOC}\ge0. The learned scale and bias allow the model to rank positive relations above unmatched candidates during training while the strict boundary remains demanding. As shown in Table b, the correct radial precedence increases by 68.76 percentage points and unmatched hard-cone admission decreases from 7.24% to 0.02%. The hard cone admits 0.60% of positive pairs; this measures a stricter condition than correctly directed radius alone.

R(3): At β=8\beta=8, the all-direction regime starts at g≥π/β≈0.393g\ge\pi/\beta\approx0.393. Its observed frequency was 0.00% in the evaluated pairs for both models. Thus the measured hard-cone behavior lies in the angle-selective regime. We have included the coordinate interpretation, pairwise definitions, and held-out statistics in the revised appendix.

W2: “The paper does not establish what the hyperbolic parameterization contributes beyond cosine alignment with a scalar ordering constraint.”

Q2: “What role does hyperbolic geometry play beyond the equivalent direction-plus-depth formulation? An explicit analysis, or a matched scalar-depth control with the same supervision, would help distinguish the contribution of the parameterization from the ordering objective.”

R(1): AURORA's geometric contribution is the separation of angular semantic alignment and radial specificity within hyperbolic space, together with a transitive order region operating on the resulting coordinates. The Lorentz hyperbolic law of cosines, given as Equation (5) in our paper, exposes the conflict in conventional distance-based semantic alignment:

cosh⁡δ(x,y)=cosh⁡(ρy−ρx)+sinh⁡ρxsinh⁡ρy(1−cos⁡θxy).\cosh\delta(x,y)=\cosh(\rho_y-\rho_x)+\sinh\rho_x\sinh\rho_y(1-\cos\theta_{xy}).

Reducing Lorentz distance between matched image–text representations can exert pressure on their radial separation, although hierarchical relations need this separation to represent specificity. AURORA assigns semantic matching to the root angle, preserving radial depth for directed hierarchy learning. This mechanism was also motivated by the limitations of distance-based hyperbolic alignment explored in ATMG [6]. ARGENT [3] instead adapts conventional entailment-cone supervision using norm-related regularization. AURORA changes the order geometry itself: its Radial–Angular Order Cone (RAOC) requires the relative child–parent depth gap g(p,q)=ρq−ρpg(p,q)=\rho_q-\rho_p to satisfy g(p,q)≥θpq/βg(p,q)\ge\theta_{pq}/\beta. Theorem 1 proves the partial-order property and nested descendant regions, while the analysis in Appendix C shows how conventional cones depend on the parent's absolute radial location.

Every Lorentz embedding can equivalently be expressed in polar coordinates as x=c−1/2[cosh⁡ρx;sinh⁡ρx ux]x=c^{-1/2}[\cosh\rho_x;\sinh\rho_x\,u_x], with ux=zx/∥zx∥u_x=z_x/\|z_x\| and ρx=c ∥zx∥\rho_x=\sqrt c\,\|z_x\| under the exponential map. The direction-plus-scalar-depth formulation in the question is exactly this hyperbolic representation in coordinates. Applying the same RAOC score and supervision to these coordinates yields the same relation and gradients through the parameterization. Curvature sets the geometric length scale, and the hyperbolic metric provides the radial expansion and distance decomposition motivating the order construction.

The contribution of this order mechanism is measured in the original Table 4: with root-angle alignment held fixed, replacing conventional entailment-cone supervision (Table 4(a)) by RAOC and its contrastive training improves ImageNet from 38.12% to 41.54% and WordNet TIE from 3.79 to 3.62. The norm-regularized no-order control (Table 4(d)) achieves 40.27% ImageNet, confirming the additional value of the learned radial order. The next reply isolates the unmatched-pair component under identical RAOC geometry.

[6] Ramasinghe S, Shevchenko V, Avraham G, Thalaiyasingam A. Accept the Modality Gap: An Exploration in the Hyperbolic Space[C]//CVPR. 2024: 27263-27272.

Comments on the paper

2 comments
AuthorsOpenReview ↗

Final Remarks

Dear Area Chair,

We sincerely thank you for your time and effort in handling our submission. We appreciate the reviewers' thoughtful assessments and constructive feedback, which have helped us further strengthen the theoretical analysis, experimental validation, and presentation of our work.

Summary of Recognized Strengths: We are grateful for the reviewers' recognition of AURORA and its key contributions:

  • A compelling separation of angular semantic alignment and radial hierarchical ordering (All reviewers), addressing their inherent tension in hyperbolic space with potential applicability beyond the proposed framework (USBG).
  • A theoretically grounded Radial–Angular Order Cone (RAOC), with proven partial-order and nested-descendant properties (3aUW, MJ3C).
  • Contrastive supervision of both positive and negative hierarchical relations (USBG).
  • Consistent performance improvements across diverse downstream tasks and backbone architectures (All reviewers).

Summary of the Main Concerns Raised by Reviewers and Our Rebuttal Efforts:

Baseline Reliability: Concerns regarding the fairness and reproducibility of baseline comparisons (USBG).

  • Response: The declining availability of GRIT images has been a significant challenge in the research community. We addressed its impact through fair retraining, independent evaluation using released checkpoints, and explicit comparisons with published results. The reviewer subsequently acknowledged the fairness of our comparisons and raised the rating.

Related Work and Similarity Metrics: Requests for additional baseline comparisons and justification of root-angle similarity (USBG).

  • Response: We incorporated UNCHA into the comparisons and conducted inference-time similarity-score swapping experiments, demonstrating that AURORA's improvements persist under different evaluation metrics. We further strengthened the theoretical analysis of angular–radial separation.

Hierarchical Representation and Visualization: Concerns regarding direct validation of learned hierarchical relations and the interpretation of radial organization (All reviewers).

  • Response: We introduced taxonomic open-set classification and direct geometric evaluations on held-out data, demonstrating substantially improved hierarchical consistency and parent–child radial ordering. We also clarified the geometric interpretation of Figure 5.

Entailment Selectivity and Ablation Studies: Requests to validate semantic entailment recognition and isolate the contribution of hierarchical supervision (3aUW, MJ3C).

  • Response: We extended the analysis to compare positive and negative cone membership across different methods, revealing the degeneracy of conventional cones and the improved selectivity of RAOC. Additional controlled ablations demonstrate the effectiveness of contrastive entailment supervision and its consistent benefits for downstream performance.

Sensitivity Analysis: Questions regarding the robustness of the proposed method across representative tasks (USBG).

  • Response: Our original sensitivity analysis already covered classification, retrieval, and hierarchical evaluation. We further extended all hyperparameter studies to the complete downstream benchmark suite, confirming consistent trends across diverse tasks.

Summary of Revisions: We have incorporated the rebuttal updates into the revised manuscript and appendix to further strengthen the presentation and experimental evidence. Specifically, we have:

  • Expanded the experimental comparisons, including published and reproduced baselines, UNCHA, and alternative similarity metrics.
  • Added direct hierarchical evaluations, including taxonomic open-set classification, radial-order statistics, cone selectivity, and additional entailment ablations.
  • Extended the sensitivity analysis to the full downstream benchmark suite.
  • Strengthened the theoretical discussions of angular–radial separation, geometric ordering, and hierarchical supervision.
  • Refined the manuscript and appendix to improve clarity, reproducibility, and the discussion of future research directions.

We sincerely thank the Area Chair and reviewers again for their time and consideration. We hope that the additional evidence and revisions presented throughout the discussion will contribute to a comprehensive final assessment of our work.

Best regards,

Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.