- Rating
- 3(+1)
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper identifies a fundamental limitation of hyperbolic entailment cones: exact transitivity forces a trade-off between cone capacity and the region where cones can be defined, making it difficult to represent abstract concepts near the origin. The authors introduce a relaxed notion called -transitivity, which allows bounded violations of cone containment, and use it to construct cones that are defined throughout the space except at the origin while retaining strong capacity.
Critical strengths
- It removes the empty region near the origin.: The existing cone-based approach cannot define cones around the origin, where abstract concepts may be located. The proposed method defines cones over nearly the entire space, excluding only the origin.
- It shows improved learning of hierarchical relations.: On the word-hierarchy data, the method achieves higher F1 scores than the existing approach on main comparisons. The gains are exist on datasets with multiple-parent relations or deeper hierarchies.
Critical weaknesses
-
The motivation for relaxing entailment is unclear.:
- How does the proposed method differ from simply reducing the entailment loss weight near the origin? Please add a direct comparison.
- UNCHA[2] appears to use a piecewise modification to the entailment objective. Please compare against that loss as well.
- Why is it important to relax the constraint near the origin? Please explain the causal link between this design choice and improved learning of hierarchical relations.
- How concentrated are embeddings near the origin? Do embeddings from other models also cluster there? If they do not, why is this relaxation necessary?
- If the issue occurs only in MERU, is the method useful only for MERU? Please compare embedding distributions across three representative VLMs: HyCoCLIP [1], UNCHA [2], and ATMG [3].
-
The experimental evaluation has important limitations.:
- The current comparisons appear to focus only on MERU. Would the same experiments produce similar results with HyCoCLIP[1], UNCHA[2], ATMG[3], and other models? Please broaden the set of baselines.
- The zero-shot classification results raise concerns about the method’s broader effectiveness. Table 10 shows substantial variability in zero-shot classification performance. Although the setting appears more favorable, its results are not clearly or consistently reported in the main table. Were these results omitted because they were unfavorable, particularly on hierarchical structure benchmarks? More broadly, does the method’s focus on hierarchical structure come at the cost of zero-shot classification or retrieval performance? If there is such a trade-off, please explain precisely why the proposed method remains useful and in what settings its benefits outweigh the costs.
[1] Pal, Avik, et al. "Compositional entailment learning for hyperbolic vision-language models." ICLR (2025).
[2] Kim, Hayeon, et al. "Uncertainty-guided Compositional Alignment with Part-to-Whole Semantic Representativeness in Hyperbolic Vision-Language Models." CVPR (2026).
[3] Ramasinghe, Sameera, et al. "Accept the modality gap: An exploration in the hyperbolic space." CVPR. (2024).
Questions for authors
-
To understand how the embedding space changes after applying the proposed method, please add a comparison of the before-and-after embedding spaces using HoroPCA [4] and norm-distance visualizations.
-
As embeddings are distributed farther from the origin, how does their behavior under the original entailment loss change? What effect does this change have on VLM performance? Please analyze this relationship.
[4] Chami, Ines, et al. "Horopca: Hyperbolic dimensionality reduction via horospherical projections." ICML (2021).
8 comments
Official Comment by Authors (2/6)
W1-3. Why relaxing the constraint near the origin matters
The causal link between the relaxation and the improved learning of hierarchies is the following.
- With the original cones, the parents stop at the boundary of the infeasible region. Just outside the boundary () the aperture widens with a diverging derivative, so the gradient of the entailment loss drives a parent toward the boundary, where its cone is widest. In the word-embedding experiments the embeddings are projected outside the region at every step, so a parent that reaches the boundary is held there: its radial update is cancelled by the projection, and its optimization stops (in Figure 4, the nodes closest to the root stay on the boundary). In the vision-language models there is no projection, the parents enter the region, and there the clipped aperture is constant, so the loss has no gradient through the aperture (W1-4).
- A parent at the boundary no longer constrains its children. Its aperture is (nearly) , so its cone is (nearly) a half-space: the hinge loss is zero for almost every child, the parent receives no signal from its children, and the children receive no signal to arrange themselves inside the parent's cone.
- With the proposed cones the gradient flows everywhere except the origin. The aperture is below with a finite derivative for every , so a parent can move into the former infeasible region (in Figure 4 the top of the hierarchy occupies the center of the ball), and its cone keeps constraining its children there. Since containment is nested (the cone of a child must lie inside the cone of its parent), an improved placement of the parents propagates to the levels below them, and the whole embedding improves.
The direct evidence that this is what drives the gains comes from the cones, which coincide with the original cones outside the infeasible region and differ only by being defined inside it. On the word-embedding benchmarks they alone improve the original cones by up to 17.0 points on MCG and Hearst-25k (Table 2), and they keep an F1 of 67.6 at where the original cones drop to 0.0 (Table 1); the smooth cones improve further. In the vision-language models the same mechanism moves the images outward (W1-4) and lets the text$\to$image term decrease under the original loss (Q2).
Official Comment by Authors (3/6)
W1-4, W2-1, and Q1. Embedding distributions across models, and whether the issue is specific to MERU
Following the reviewer's suggestion, we pretrained HyCoCLIP (ViT-S/16, 3 seeds) and UNCHA (ViT-S/16, 1 seed) on GRIT under the same conditions as MERU, replacing only the aperture in all of their entailment terms. We visualized the validation embeddings of all three models with HoroPCA (Figure 6 of the revision) and as distributions of the normalized radius (Table 3 of the revision; complete versions including the cones: Figure 18 and Table 16). In this coordinate, is the infeasible region of . We could not include ATMG as its public implementation does not match the objective stated in the paper (as also noted by Kim et al., 2026, Appendix I.3).
Normalized radii of the validation texts and images on 19,072 GRIT pairs (seed 1) (from Table 3 and the bottom row of Figure 6 of the revision)
| cones | texts with | mean , text | mean , image | share of text$\to$image loss from | |
|---|---|---|---|---|---|
| MERU (ViT-B/16) | 96% | 0.97 | 1.99 | 92% | |
| 100% | 0.85 | 3.27 | (loss 0) | ||
| HyCoCLIP (ViT-S/16) | 100% | 0.63 | 1.01 | 100% | |
| 100% | 0.65 | 1.43 | 100% | ||
| UNCHA (ViT-S/16) | 90% | 0.98 | 2.01 | 83% | |
| 0% | 1.41 | 2.50 | 0% |
The concentration near the origin is not specific to MERU. With the original cones, the texts, i.e., the parents of the text$\to$image entailment term, lie at or inside the boundary of the infeasible region in all three models (96%, 100%, and 90% of the texts). These texts carry most of the text$\to$image entailment loss (92%, 100%, and 83%). The mechanism is the shape of the original aperture . For the implementations clip it to a constant, so the loss has no gradient with respect to the radius of a parent through its aperture. Just outside , its derivative diverges and the cone narrows steeply. A parent therefore gains nothing from leaving the region and is pushed back when it does, so the configuration cannot be rearranged according to the hierarchy. This agrees with the audit of Kim et al. (2026), who find the text-side cones of MERU, HyCoCLIP, and PHyCLIP saturated at and the parents ending near . This edge is exactly .
The proposed cones have an aperture below with a finite derivative for every , so the gradient does not vanish. The effect is the same in all three models: the images move consistently outward (mean 1.99 3.27 in MERU, 1.01 1.43 in HyCoCLIP, 2.01 2.50 in UNCHA). We note that the texts do not move uniformly. In MERU and HyCoCLIP they stay at , where the proposed cones are feasible, while in UNCHA they move out to . Since hyperbolic space near the origin is almost Euclidean, a larger radius means that the embeddings make more use of the hyperbolic geometry. The relaxation is thus needed, and useful, for every model that places the parents of its entailment terms near the origin, which all three models do.
Official Comment by Authors (1/6)
We thank the reviewer for the detailed questions. They led us to extend the vision-language experiments from MERU to HyCoCLIP and UNCHA and to add the embedding analysis the reviewer asked for. Section 6.3–6.4 of the manuscript has been rewritten around these results (Figures 5–6 and Table 3 in the revision). We answer the weaknesses (W1-1 to W1-4, W2-1, W2-2) and questions (Q1, Q2) below, grouping the ones that share an answer.
W1-1 and W1-2. Comparison with down-weighting the entailment loss near the origin and with the piecewise loss of UNCHA
Both alternatives modify the loss while keeping the cones. We therefore compared them under the same training pipeline on the hierarchical word-embedding benchmarks, where the effect of the aperture is isolated from the contrastive loss. Starting from the hinge loss of Ganea et al., we added:
Test F1 (%) under the containment criterion, protocol of Ganea et al., 3 seeds (mean ± std)
Neither loss-side modification closes the gap to the proposed cones. Down-weighting near the origin changes the original cones by −23.5 to +3.8 points and hurts in 8 of 12 cells. The piecewise term of UNCHA changes them by −1.9 to +2.3 points, i.e., it is nearly neutral. Applied to the proposed cones, it slightly lowers them (−0.4 to −7.9, except MCG at d=5). The proposed cones with the plain hinge loss are the best in 9 of 12 cells and improve the original by 11 to 27 points on MCG and Hearst-25k.
This is what our analysis predicts. Down-weighting the loss near the origin removes the gradient there. However, the lack of a gradient inside the infeasible region is precisely the problem (see W1-3 and W1-4 below): the points pushed against the boundary r=ε receive no signal to move. Down-weighting only enlarges the region in which this happens. The piecewise term adds a pressure on the exterior angle that is independent of the cone. It therefore neither restores the aperture inside the region nor changes where the aperture saturates. Both modifications keep the cones of Ganea et al., and by Proposition 3.2 any exactly transitive cones have an infeasible region. Removing it requires changing what transitivity demands. This is exactly why we relax transitivity to (ε,δ)-transitivity instead of modifying the loss. In the vision-language experiments the comparison with the piecewise loss is also direct: our UNCHA runs keep its piecewise term and change only the aperture (W2-1).