acceptodds
Under review as a conference paper at ICLR 2027

New Hyperbolic Entailment Cones Beyond the Capacity–Feasibility Trade-off

Abstract

Entailment cones in hyperbolic space (Ganea et al., 2018b) represent the partial order of a hierarchy as containment between cones, and serve as a foundation of recent hyperbolic vision-language models. However, the cones' aperture cannot be defined everywhere; there is necessarily an infeasible ball region around the origin. The size of the infeasible ball trades off against the capacity (total aperture) of the cones, and at both ends the performance of the entailment cones degrades significantly. We first formalize this capacity–feasibility trade-off and show that this trade-off extends to any cones satisfying transitivity, i.e., the trade-off is an unavoidable problem in the original formulation. To overcome this trade-off, we then introduce -transitivity, which reformulates transitivity with two parameters: the radius of an infeasible region and an allowance for violations of transitivity. Building on this, we propose cones that can be defined everywhere except the origin, have at least the same capacity, and generalize the original entailment cones. On hierarchical word embeddings (WordNet, MCG, Hearst), the proposed method outperforms the original entailment cones in 10 of 12 settings, with gains of up to 27.6 points. On the hyperbolic vision-language model MERU, it preserves accuracy on zero-shot classification while yielding embeddings that better satisfy the entailment condition of the original formulation.

Primary area:
learning on graphs and other geometries & topologies
Keywords:
hyperbolic entailment cones, hyperbolic embeddings, hyperbolic space, hierarchical representation learning

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_QP5SOpenReview ↗
Rating
3(+1)
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper identifies a fundamental limitation of hyperbolic entailment cones: exact transitivity forces a trade-off between cone capacity and the region where cones can be defined, making it difficult to represent abstract concepts near the origin. The authors introduce a relaxed notion called (ε,δ)(\varepsilon,\delta)-transitivity, which allows bounded violations of cone containment, and use it to construct cones that are defined throughout the space except at the origin while retaining strong capacity.

Critical strengths
  • It removes the empty region near the origin.: The existing cone-based approach cannot define cones around the origin, where abstract concepts may be located. The proposed method defines cones over nearly the entire space, excluding only the origin.
  • It shows improved learning of hierarchical relations.: On the word-hierarchy data, the method achieves higher F1 scores than the existing approach on main comparisons. The gains are exist on datasets with multiple-parent relations or deeper hierarchies.
Critical weaknesses
  1. The motivation for relaxing entailment is unclear.:

    • How does the proposed method differ from simply reducing the entailment loss weight near the origin? Please add a direct comparison.
    • UNCHA[2] appears to use a piecewise modification to the entailment objective. Please compare against that loss as well.
    • Why is it important to relax the constraint near the origin? Please explain the causal link between this design choice and improved learning of hierarchical relations.
    • How concentrated are embeddings near the origin? Do embeddings from other models also cluster there? If they do not, why is this relaxation necessary?
      • If the issue occurs only in MERU, is the method useful only for MERU? Please compare embedding distributions across three representative VLMs: HyCoCLIP [1], UNCHA [2], and ATMG [3].
  2. The experimental evaluation has important limitations.:

    • The current comparisons appear to focus only on MERU. Would the same experiments produce similar results with HyCoCLIP[1], UNCHA[2], ATMG[3], and other models? Please broaden the set of baselines.
    • The zero-shot classification results raise concerns about the method’s broader effectiveness. Table 10 shows substantial variability in zero-shot classification performance. Although the δ1\delta_1 setting appears more favorable, its results are not clearly or consistently reported in the main table. Were these results omitted because they were unfavorable, particularly on hierarchical structure benchmarks? More broadly, does the method’s focus on hierarchical structure come at the cost of zero-shot classification or retrieval performance? If there is such a trade-off, please explain precisely why the proposed method remains useful and in what settings its benefits outweigh the costs.

[1] Pal, Avik, et al. "Compositional entailment learning for hyperbolic vision-language models." ICLR (2025).

[2] Kim, Hayeon, et al. "Uncertainty-guided Compositional Alignment with Part-to-Whole Semantic Representativeness in Hyperbolic Vision-Language Models." CVPR (2026).

[3] Ramasinghe, Sameera, et al. "Accept the modality gap: An exploration in the hyperbolic space." CVPR. (2024).

Questions for authors
  • To understand how the embedding space changes after applying the proposed method, please add a comparison of the before-and-after embedding spaces using HoroPCA [4] and norm-distance visualizations.

  • As embeddings are distributed farther from the origin, how does their behavior under the original entailment loss change? What effect does this change have on VLM performance? Please analyze this relationship.

[4] Chami, Ines, et al. "Horopca: Hyperbolic dimensionality reduction via horospherical projections." ICML (2021).

8 comments
AuthorsOpenReview ↗

Official Comment by Authors (1/6)

We thank the reviewer for the detailed questions. They led us to extend the vision-language experiments from MERU to HyCoCLIP and UNCHA and to add the embedding analysis the reviewer asked for. Section 6.3–6.4 of the manuscript has been rewritten around these results (Figures 5–6 and Table 3 in the revision). We answer the weaknesses (W1-1 to W1-4, W2-1, W2-2) and questions (Q1, Q2) below, grouping the ones that share an answer.

W1-1 and W1-2. Comparison with down-weighting the entailment loss near the origin and with the piecewise loss of UNCHA

Both alternatives modify the loss while keeping the cones. We therefore compared them under the same training pipeline on the hierarchical word-embedding benchmarks, where the effect of the aperture is isolated from the contrastive loss. Starting from the hinge loss of Ganea et al., we added:

  • (a) a weight w(r)=clip((r−ε)/ε,0,1)w(r)=\mathrm{clip}((r-\varepsilon)/\varepsilon,0,1) on the positive term ("origin down-weight"). It switches the entailment loss off at the boundary r=εr=\varepsilon of the infeasible region and ramps it up to full weight at r=2εr=2\varepsilon;
  • (b) the piecewise-continuous term of UNCHA. It adds 0.1 Ξ0.1\,\Xi (Ξ\Xi: exterior angle) to the hinge, so that the loss is nonzero even inside the cone. The table below reports test F1 (3 seeds) for the original (ε,0)(\varepsilon,0) cones with each loss and for the proposed (0,δ2)(0,\delta_2) cones.

Test F1 (%) under the containment criterion, protocol of Ganea et al., 3 seeds (mean ±\pm std)

(ε,0)(\varepsilon,0) hinge(ε,0)(\varepsilon,0) + origin down-weight(ε,0)(\varepsilon,0) + UNCHA piecewise(0,δ2)(0,\delta_2) hinge(0,δ2)(0,\delta_2) + UNCHA piecewise
WordNet nouns, 50%d=5d=589.6 ±\pm 1.680.5 ±\pm 1.089.7 ±\pm 0.492.2 ±\pm 0.389.1 ±\pm 0.4
d=10d=1092.7 ±\pm 1.269.2 ±\pm 1.191.6 ±\pm 1.195.3 ±\pm 0.192.0 ±\pm 0.2
WordNet nouns, recon.d=5d=597.0 ±\pm 0.297.5 ±\pm 0.495.1 ±\pm 0.095.1 ±\pm 0.491.5 ±\pm 0.3
d=10d=1098.4 ±\pm 0.398.5 ±\pm 0.096.0 ±\pm 0.197.6 ±\pm 0.291.7 ±\pm 0.2
MCG, 50%d=5d=542.7 ±\pm 2.046.5 ±\pm 0.942.8 ±\pm 2.563.3 ±\pm 1.265.1 ±\pm 1.5
d=10d=1054.9 ±\pm 0.846.7 ±\pm 1.754.7 ±\pm 0.173.0 ±\pm 1.169.0 ±\pm 0.6
MCG, recon.d=5d=561.6 ±\pm 4.664.3 ±\pm 2.962.8 ±\pm 5.075.8 ±\pm 1.668.6 ±\pm 1.0
d=10d=1069.5 ±\pm 3.766.9 ±\pm 1.069.3 ±\pm 2.582.6 ±\pm 1.273.4 ±\pm 0.3
Hearst-25k, 50%d=5d=553.3 ±\pm 1.546.0 ±\pm 0.753.1 ±\pm 2.377.5 ±\pm 0.776.1 ±\pm 0.6
d=10d=1057.5 ±\pm 1.141.7 ±\pm 0.558.6 ±\pm 1.584.7 ±\pm 0.882.8 ±\pm 0.6
Hearst-25k, recon.d=5d=572.3 ±\pm 12.268.4 ±\pm 5.174.6 ±\pm 11.682.7 ±\pm 0.979.6 ±\pm 1.1
d=10d=1085.3 ±\pm 1.774.3 ±\pm 0.385.6 ±\pm 0.789.5 ±\pm 0.983.9 ±\pm 0.6

Neither loss-side modification closes the gap to the proposed cones. Down-weighting near the origin changes the original cones by −23.5-23.5 to +3.8+3.8 points and hurts in 8 of 12 cells. The piecewise term of UNCHA changes them by −1.9-1.9 to +2.3+2.3 points, i.e., it is nearly neutral. Applied to the proposed cones, it slightly lowers them (−0.4-0.4 to −7.9-7.9, except MCG at d=5d=5). The proposed cones with the plain hinge loss are the best in 9 of 12 cells and improve the original by 11 to 27 points on MCG and Hearst-25k.

This is what our analysis predicts. Down-weighting the loss near the origin removes the gradient there. However, the lack of a gradient inside the infeasible region is precisely the problem (see W1-3 and W1-4 below): the points pushed against the boundary r=εr=\varepsilon receive no signal to move. Down-weighting only enlarges the region in which this happens. The piecewise term adds a pressure on the exterior angle that is independent of the cone. It therefore neither restores the aperture inside the region nor changes where the aperture saturates. Both modifications keep the cones of Ganea et al., and by Proposition 3.2 any exactly transitive cones have an infeasible region. Removing it requires changing what transitivity demands. This is exactly why we relax transitivity to (ε,δ)(\varepsilon,\delta)-transitivity instead of modifying the loss. In the vision-language experiments the comparison with the piecewise loss is also direct: our UNCHA runs keep its piecewise term and change only the aperture (W2-1).

AuthorsOpenReview ↗

Official Comment by Authors (2/6)

W1-3. Why relaxing the constraint near the origin matters

The causal link between the relaxation and the improved learning of hierarchies is the following.

  1. With the original cones, the parents stop at the boundary of the infeasible region. Just outside the boundary (s→1+s\to1^+) the aperture ψK=arcsin⁡(1/s)\psi_K=\arcsin(1/s) widens with a diverging derivative, so the gradient of the entailment loss drives a parent toward the boundary, where its cone is widest. In the word-embedding experiments the embeddings are projected outside the region at every step, so a parent that reaches the boundary is held there: its radial update is cancelled by the projection, and its optimization stops (in Figure 4, the nodes closest to the root stay on the boundary). In the vision-language models there is no projection, the parents enter the region, and there the clipped aperture is constant, so the loss has no gradient through the aperture (W1-4).
  2. A parent at the boundary no longer constrains its children. Its aperture is (nearly) π/2\pi/2, so its cone is (nearly) a half-space: the hinge loss is zero for almost every child, the parent receives no signal from its children, and the children receive no signal to arrange themselves inside the parent's cone.
  3. With the proposed cones the gradient flows everywhere except the origin. The aperture is below π/2\pi/2 with a finite derivative for every s>0s>0, so a parent can move into the former infeasible region (in Figure 4 the top of the hierarchy occupies the center of the ball), and its cone keeps constraining its children there. Since containment is nested (the cone of a child must lie inside the cone of its parent), an improved placement of the parents propagates to the levels below them, and the whole embedding improves.

The direct evidence that this is what drives the gains comes from the (0,δ∞)(0,\delta_\infty) cones, which coincide with the original cones outside the infeasible region and differ only by being defined inside it. On the word-embedding benchmarks they alone improve the original cones by up to 17.0 points on MCG and Hearst-25k (Table 2), and they keep an F1 of 67.6 at K=10K=10 where the original cones drop to 0.0 (Table 1); the smooth q=1,2q=1,2 cones improve further. In the vision-language models the same mechanism moves the images outward (W1-4) and lets the text$\to$image term decrease under the original loss (Q2).

AuthorsOpenReview ↗

Official Comment by Authors (3/6)

W1-4, W2-1, and Q1. Embedding distributions across models, and whether the issue is specific to MERU

Following the reviewer's suggestion, we pretrained HyCoCLIP (ViT-S/16, 3 seeds) and UNCHA (ViT-S/16, 1 seed) on GRIT under the same conditions as MERU, replacing only the aperture in all of their entailment terms. We visualized the validation embeddings of all three models with HoroPCA (Figure 6 of the revision) and as distributions of the normalized radius s=c ∥x∥/(2K)s=\sqrt{c}\,\lVert x\rVert/(2K) (Table 3 of the revision; complete versions including the (0,δ1)(0,\delta_1) cones: Figure 18 and Table 16). In this coordinate, s<1s<1 is the infeasible region of ψK\psi_K. We could not include ATMG as its public implementation does not match the objective stated in the paper (as also noted by Kim et al., 2026, Appendix I.3).

Normalized radii of the validation texts and images on 19,072 GRIT pairs (seed 1) (from Table 3 and the bottom row of Figure 6 of the revision)

conestexts with s<1s<1mean ss, textmean ss, imageshare of text$\to$image loss from s<1s<1
MERU (ViT-B/16)(ε,0)(\varepsilon,0)96%0.971.9992%
(0,δ2)(0,\delta_2)100%0.853.27(loss ≈\approx 0)
HyCoCLIP (ViT-S/16)(ε,0)(\varepsilon,0)100%0.631.01100%
(0,δ2)(0,\delta_2)100%0.651.43100%
UNCHA (ViT-S/16)(ε,0)(\varepsilon,0)90%0.982.0183%
(0,δ2)(0,\delta_2)0%1.412.500%

The concentration near the origin is not specific to MERU. With the original cones, the texts, i.e., the parents of the text$\to$image entailment term, lie at or inside the boundary of the infeasible region in all three models (96%, 100%, and 90% of the texts). These texts carry most of the text$\to$image entailment loss (92%, 100%, and 83%). The mechanism is the shape of the original aperture ψK=arcsin⁡(1/s)\psi_K=\arcsin(1/s). For s<1s<1 the implementations clip it to a constant, so the loss has no gradient with respect to the radius of a parent through its aperture. Just outside s=1s=1, its derivative −1/(ss2−1)-1/(s\sqrt{s^2-1}) diverges and the cone narrows steeply. A parent therefore gains nothing from leaving the region and is pushed back when it does, so the configuration cannot be rearranged according to the hierarchy. This agrees with the audit of Kim et al. (2026), who find the text-side cones of MERU, HyCoCLIP, and PHyCLIP saturated at π/2\pi/2 and the parents ending near cρ=2K\sqrt{c}\rho=2K. This edge is exactly s=1s=1.

The proposed cones have an aperture below π/2\pi/2 with a finite derivative for every s>0s>0, so the gradient does not vanish. The effect is the same in all three models: the images move consistently outward (mean ss 1.99 →\to 3.27 in MERU, 1.01 →\to 1.43 in HyCoCLIP, 2.01 →\to 2.50 in UNCHA). We note that the texts do not move uniformly. In MERU and HyCoCLIP they stay at s<1s<1, where the proposed cones are feasible, while in UNCHA they move out to s≈1.4s\approx1.4. Since hyperbolic space near the origin is almost Euclidean, a larger radius means that the embeddings make more use of the hyperbolic geometry. The relaxation is thus needed, and useful, for every model that places the parents of its entailment terms near the origin, which all three models do.

Reviewer_2pBDOpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper addresses a known limitation of hyperbolic entailment cones (Ganea et al., 2018b). Near the origin, there is a region where cones cannot be defined, and making this region smaller forces the cones to be narrower. The paper makes three contributions. First, it proves this trade-off holds for every exactly transitive cone. Second, it proposes a relaxed notion of transitivity that allows a small, bounded amount of violation. Third, it derives closed-form cones under this relaxation that are defined everywhere except the origin and can directly replace the original aperture formula. The method is evaluated on hierarchical embedding benchmarks (WordNet nouns, MCG, Hearst), where it improves F1 over Ganea's cones in most settings, and in the vision-language model MERU, where accuracy is unchanged and hierarchy metrics improve slightly.

Critical strengths

S1. Showing the trade-off is unavoidable. The paper proves that any exactly transitive cone must have an infeasible region near the origin, with apertures no wider than Ganea's cones, with the same infeasible region (Prop. 3.2). The paper shows that a better transitive cone cannot remove the problem, and that transitivity itself has to be relaxed.

S2. A principled relaxation that gives practical new cones. The (ϵ\epsilon, δ\delta)-transitivity framework treats exact transitivity as a special case and makes the cost of relaxing it explicit through a bounded budget. Because only the aperture formula changes, the proposed cones can replace the original ones in any cone-based model. This makes it generalizable. They also let general concepts sit near the origin and more specific ones toward the boundary, a common way to model hierarchies in hyperbolic space (Nickel & Kiela, 2017). The gains are largest on hierarchies with many multi-parent nodes.

Critical weaknesses

W1. The gains seem to depend on the training setup. Under the Shadow Cones setup (App. L), a cone with Ganea's geometry already reaches 82.6 F1 on MCG and 86.5 on Hearst-25k (Table 7), higher than the proposed cones under the main setup (64.9 and 77.0; Table 2). With that setup, the proposed cones give smaller gains and are worse on WordNet nouns, as the authors already acknowledge. It is unclear how much of the gain comes from removing the infeasible region.

W2. The MERU result is weaker than it looks. The abstract's claim that the proposed cones give embeddings that better satisfy the entailment condition rests on the loss in Fig. 5, which is measured with the original aperture ψK\psi_K. Since the proposed cones are never wider than ψK\psi_K, the new model meets this condition almost automatically, so its lower loss is expected. With the hierarchy gains also small (Table 3), the evidence for a better embedding in MERU is limited.

Questions for authors

Clarification:

  1. Can you separate the effect of the infeasible region from the training protocol? Under the Shadow Cones protocol, a cone with the same geometry as the (ε,0)(\varepsilon,0) cones reaches 82.6 F1 on MCG (Table 7), compared with 46.6 under the main protocol (Table 2). Which part of the protocol (loss, initialization, or projection) explains this gap?
  2. Could you report precision and recall separately?

Minor Comments:

Notations:

  1. hmax⁡h_{\max} is used at Line 173 before its definition in Eq. (6).

Figures/Tables:

  1. Fig. 4 is too crowded. Perhaps a bigger figure with relaxed spacing can help visualize the node labels better. Apart from a few nodes near ε\varepsilon, the two panels look almost identical.
  2. Line 1603-1604: the threshold criterion is said to "improve monotonically (Table 5)", but on Hearst-25k the (ε,0)(\varepsilon,0) threshold F1 is not monotone (d=5d=5: 59.9 →\to 57.2 →\to 51.7 at 10/25/50%).
7 comments
AuthorsOpenReview ↗

Official Comment by Authors (1/5)

We thank the reviewer for the careful reading and the concrete questions. W1 and Q1 concern the same point, so we answer them together.

W1 and Q1. Separating the effect of the infeasible region from the training protocol

We implemented both protocols in a single code base and switched their differing components one at a time. Starting from the protocol of Ganea et al., each row of Table 12 of the revision (reproduced below) switches only one of nine components to its Shadow Cones counterpart, and the last row switches all of them. The components include the three the reviewer names (loss, initialization, and projection). Entries are test F1 under the containment criterion at the validation-selected KK and learning rate (50% non-basic edges, d=5d=5, one seed), with the change from the base row in parentheses.

One-at-a-time decomposition of the protocol (Table 12 of the revision)

MCG

(ε,0)(\varepsilon,0)(0,δ1)(0,\delta_1)(0,δ2)(0,\delta_2)
base (Ganea et al.)43.266.866.8
init: warm start →\to cold + burn-in 20 ep65.1 (+21.9+21.9)78.6 (+11.8+11.8)76.8 (+10.0+10.0)
loss: hinge + angle →\to CE + distance76.4 (+33.2+33.2)75.1 (+8.3+8.3)74.3 (+7.5+7.5)
margin: 0.01 →\to 0.00140.9 (−2.3-2.3)58.7 (−8.1-8.1)53.6 (−13.1-13.1)
optimizer: RSGD →\to RAdam84.6 (+41.4+41.4)83.7 (+16.9+16.9)82.9 (+16.2+16.2)
epochs: 200 →\to 50066.3 (+23.1+23.1)72.6 (+5.8+5.8)72.9 (+6.2+6.2)
batch size: 10 →\to 1647.8 (+4.6+4.6)67.1 (+0.3+0.3)63.6 (−3.2-3.2)
directions: forward →\to both45.2 (+2.0+2.0)64.9 (−1.9-1.9)57.2 (−9.6-9.6)
negatives: resampled →\to sampled once43.2 (+0.0+0.0)67.2 (+0.4+0.4)64.2 (−2.6-2.6)
projection shell: ε+10−5\varepsilon+10^{-5} →\to ε+10−2\varepsilon+10^{-2}74.4 (+31.1+31.1)––
all switched (Shadow Cones protocol)84.9 (+41.7+41.7)86.4 (+19.6+19.6)84.7 (+17.9+17.9)

Hearst-25k

(ε,0)(\varepsilon,0)(0,δ1)(0,\delta_1)(0,δ2)(0,\delta_2)
base (Ganea et al.)59.776.277.6
init: warm start →\to cold + burn-in 20 ep69.3 (+9.6+9.6)85.9 (+9.7+9.7)85.3 (+7.6+7.6)
loss: hinge + angle →\to CE + distance82.0 (+22.3+22.3)76.9 (+0.7+0.7)77.4 (−0.2-0.2)
margin: 0.01 →\to 0.00159.5 (−0.2-0.2)68.6 (−7.6-7.6)58.3 (−19.3-19.3)
optimizer: RSGD →\to RAdam88.7 (+29.0+29.0)88.9 (+12.7+12.7)88.3 (+10.7+10.7)
epochs: 200 →\to 50079.7 (+20.0+20.0)78.4 (+2.3+2.3)80.3 (+2.6+2.6)
batch size: 10 →\to 1664.6 (+4.9+4.9)78.3 (+2.1+2.1)78.5 (+0.8+0.8)
directions: forward →\to both61.4 (+1.7+1.7)72.5 (−3.6-3.6)73.3 (−4.3-4.3)
negatives: resampled →\to sampled once60.3 (+0.6+0.6)77.0 (+0.8+0.8)77.1 (−0.5-0.5)
projection shell: ε+10−5\varepsilon+10^{-5} →\to ε+10−2\varepsilon+10^{-2}83.8 (+24.1+24.1)––
all switched (Shadow Cones protocol)88.5 (+28.8+28.8)89.2 (+13.0+13.0)88.8 (+11.1+11.1)

WordNet nouns

(ε,0)(\varepsilon,0)(0,δ1)(0,\delta_1)(0,δ2)(0,\delta_2)
base (Ganea et al.)90.889.292.2
init: warm start →\to cold + burn-in 20 ep32.9 (−57.8-57.8)81.2 (−8.0-8.0)81.6 (−10.6-10.6)
loss: hinge + angle →\to CE + distance79.0 (−11.7-11.7)85.7 (−3.5-3.5)84.0 (−8.2-8.2)
margin: 0.01 →\to 0.00187.5 (−3.2-3.2)86.7 (−2.5-2.5)86.9 (−5.2-5.2)
optimizer: RSGD →\to RAdam94.4 (+3.7+3.7)94.3 (+5.1+5.1)94.9 (+2.8+2.8)
epochs: 200 →\to 40092.8 (+2.0+2.0)90.9 (+1.7+1.7)92.9 (+0.8+0.8)
batch size: 10 →\to 1691.8 (+1.1+1.1)90.2 (+1.0+1.0)92.2 (+0.1+0.1)
directions: forward →\to both89.9 (−0.9-0.9)90.0 (+0.8+0.8)91.5 (−0.7-0.7)
negatives: resampled →\to sampled once89.9 (−0.9-0.9)90.2 (+1.0+1.0)92.1 (−0.0-0.0)
projection shell: ε+10−5\varepsilon+10^{-5} →\to ε+10−2\varepsilon+10^{-2}87.3 (−3.4-3.4)––
all switched (Shadow Cones protocol)87.5 (−3.3-3.3)85.2 (−4.0-4.0)82.9 (−9.3-9.3)
AuthorsOpenReview ↗

Official Comment by Authors (2/5)

(W1 and Q1 continued)

Which component explains the gap. For the original (ε,0)(\varepsilon,0) cones on MCG, the gap between the two protocols (43.2 versus 84.9) is almost entirely reproduced by the optimizer alone: changing RSGD to RAdam gives 84.6. The three components the reviewer names also help individually (loss +33.2+33.2, initialization +21.9+21.9, projection shell +31.1+31.1), and so does training longer (+23.1+23.1), but none of them reaches the optimizer. The same holds on Hearst-25k. The gap is thus an optimization issue of the original cones under RSGD, not a property of one specific loss or initialization.

How much of the gain comes from removing the infeasible region. The table also shows that the original cones are far more sensitive to every one of these components than the proposed cones. On MCG, the four changes above move the original cones by +22+22 to +41+41 points but the proposed cones by only +6+6 to +17+17. In particular, moving the projection shell away from the boundary of the infeasible region, a component that exists only because of that region, improves the original cones by 31.1 points on MCG and 24.1 on Hearst-25k. This is direct evidence that the low base value of the original cones comes from how the embeddings near the infeasible region are handled. The proposed cones have no such region and need no projection.

We then repeated the comparison of the two optimizers alone with 10 seeds, keeping all other settings of the protocol of Ganea et al. (Table 13 of the revision, reproduced below).

RSGD versus RAdam (containment-criterion test F1, 10 seeds, mean ±\pm std; Table 13 of the revision)

WordNet nounsMCGHearst-25k
(ε,0)(\varepsilon,0), RSGD90.6 ±\pm 0.748.1 ±\pm 4.453.3 ±\pm 10.4
(0,δ1)(0,\delta_1), RSGD89.7 ±\pm 0.767.7 ±\pm 1.276.5 ±\pm 1.8
(0,δ2)(0,\delta_2), RSGD91.8 ±\pm 1.065.4 ±\pm 2.777.0 ±\pm 1.4
(ε,0)(\varepsilon,0), RAdam93.9 ±\pm 0.783.9 ±\pm 0.689.0 ±\pm 0.5
(0,δ1)(0,\delta_1), RAdam94.0 ±\pm 0.683.2 ±\pm 0.489.1 ±\pm 0.6
(0,δ2)(0,\delta_2), RAdam94.4 ±\pm 0.982.6 ±\pm 0.488.7 ±\pm 0.5

With RAdam the three cones reach similar final F1, and we agree with the reviewer that the large gains of the main table are specific to the protocol of Ganea et al. We state this explicitly in the revision. What remains with RAdam is a difference in convergence (Figure 13 of the revision: training loss and validation F1 over the epochs, 10 seeds). On MCG and Hearst-25k the loss of the proposed cones is lower at every epoch, although their apertures are never wider than ψK\psi_K. At epoch 20 it is 0.90 / 0.36 versus 1.46 on MCG and 0.67 / 0.55 versus 1.62 on Hearst-25k. Their validation F1 at epoch 20 is 61.5 / 73.2 on MCG and 74.7 / 77.6 on Hearst-25k, compared with 53.5 and 52.2 for the original cones. On WordNet nouns the original cones start lower in loss, but their loss almost stops decreasing after epoch 60. Their validation F1 peaks at epoch 120 and then declines, whereas the proposed cones keep improving and end higher (93.7 / 94.3 versus 91.9).

In summary, the gain of the main table decomposes as follows. Under RSGD, the original cones are hard to optimize because of the infeasible region (the projection shell alone accounts for +24+24 to +31+31 points), and the proposed cones remove this difficulty. With an adaptive optimizer the final F1 becomes comparable, and the remaining advantage of the proposed cones is faster and more stable convergence, without any projection. In this sense, we conclude that removing the infeasible region improves the optimization conditions. With an optimizer suited to the data, the proposed cones converge faster than the original cones on all three datasets. With RSGD they are far less sensitive to the protocol. This is also the conclusion stated in the revision, where these results are added to the appendix (Tables 12–13 and Figure 13).

AuthorsOpenReview ↗

Official Comment by Authors (3/5)

W2. Evidence in vision-language models

We thank the reviewer for this point, which made us re-examine the claim. We agree that a comparison of loss levels under ψK\psi_K needs care, and we have removed the phrase "better satisfy the entailment condition" from the abstract, the introduction, and the conclusion. We do not think, however, that the low loss follows automatically from the proposed cones being never wider than ψK\psi_K. The revised Sections 6.3–6.4, which now cover MERU, HyCoCLIP, and UNCHA, also show improvements that are not visible from accuracy.

Narrower cones do not make the loss under ψK\psi_K low by themselves. The inclusion only says that if a model satisfies its own (narrower) entailment condition, it also satisfies the one under ψK\psi_K. Whether it does is a matter of optimization, and the experiments show that it is not automatic.

  • The original cones are trained with ψK\psi_K itself, yet their loss under ψK\psi_K does not decrease and rises over training in all three models (text$\to$image term: MERU 3×10−3→2×10−23\times10^{-3}\to2\times10^{-2}, HyCoCLIP 0.04→0.110.04\to0.11, UNCHA 0.07→0.100.07\to0.10; Figure 5).
  • The (0,δ1)(0,\delta_1) cones are narrower than the (0,δ2)(0,\delta_2) cones. Yet in MERU their loss under ψK\psi_K starts to increase after about 30,000 iterations and merges with that of the original cones (Figure 17). That of the (0,δ2)(0,\delta_2) cones stays below 10−310^{-3}.
  • In HyCoCLIP and UNCHA, the box-level terms of the proposed cones are higher under ψK\psi_K than those of the original cones, and in UNCHA so is the text$\to$image term (Figure 5). We report this openly in the revision.

A low loss under ψK\psi_K is therefore not implied by the width of the cones. It is obtained in MERU and in the text$\to$image term of HyCoCLIP, which decreases from 0.05 to 0.03 while that of the original cones rises. There it reflects that the entailment loss is actually being optimized, in line with the improved optimization conditions discussed in W1.

Improvements that accuracy does not show. Sections 6.3–6.4 give evidence that does not use ψK\psi_K as a yardstick.

  • Where the embeddings are. With the (ε,0)(\varepsilon,0) cones, the texts, i.e., the parents of the text$\to$image term, lie at or inside the boundary of the infeasible region in all three models (96%, 100%, and 90% of the texts). These texts carry most of the text$\to$image loss (92%, 100%, and 83%; Figure 6 and Table 3). There the clipped aperture is constant, so the loss has no gradient through the aperture. This agrees with the audit of Kim et al. (2026), who find saturated text-side cones and parents ending at exactly this boundary.
  • What changes. With the proposed cones the infeasible region is removed, and the embeddings, in particular the images, move consistently outward in all three models (Figure 6). The mean normalized radius of the images changes from 1.99 to 3.27 in MERU, from 1.01 to 1.43 in HyCoCLIP, and from 2.01 to 2.50 in UNCHA. They use a wider part of hyperbolic space away from the origin, where its geometry differs from the Euclidean one.

On the size of the hierarchy gains. We agree that they are small, and we now say so. With the (0,δ2)(0,\delta_2) cones, classification and retrieval stay within about one point, and the hierarchy metrics improve slightly for MERU and HyCoCLIP (13 of 15 cells over three seeds each) but not for UNCHA. Section 6.4 discusses why the improvements above do not translate into accuracy. First, the entailment loss appears to hardly function in these models in the first place, so accuracy is carried by the contrastive loss. Second, all models remain in a near-Euclidean regime that the aperture formula does not change. The results are nevertheless not only negative. The outward shift is largest in MERU, the simplest of the three models (mean uu of the images from 0.40 to 0.65). There the hierarchy metrics also improve consistently over three seeds. We regard this as evidence that the aperture formula is one obstacle to effective entailment learning in image-text models.

Reviewer_wwfBOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

Entailment cones cannot be defined in a ball around the origin, and the size of this infeasible region trades off against the width of the cones. This paper argues that the trade-off is unavoidable for any cones satisfying exact transitivity. To overcome it, the authors relax transitivity with an allowance for violations, and construct cones that are defined everywhere except the origin, concentrate the violations in the former infeasible region, and recover the original cones far from the origin.

Critical strengths

The paper gives a clean formal account of a well-known limitation of entailment cones. The relaxation of transitivity is natural, it generalizes the original formulation, and the proposed cones are a simple drop-in replacement for the aperture.

Critical weaknesses

The original aperture problem is rendered obsolete by Shadow Cones (Yu et al., 2024). Yu et al. already identified the infeasible region of Ganea's cones and removed it with their half-space formulations, which place the light source at infinity. Umbral-half-space cones in particular are exactly transitive, have no infeasible region, and substantially outperform the original Ganea formulation. The paper nonetheless cites Yu et al. as evidence that resolving this trade-off "has remained open." The authors do run umbral-half-space, but only in Appendix L, as a reference outside the "arena" of the paper. There it is the best method in most settings, often by a wide margin (e.g., 87.9 vs. 85.6 on MCG and 91.8 vs. 87.4 on Hearst-25k at 50% non-basic edges, against the best proposed cones under the same protocol). Since this paper aims to characterize a fundamental trade-off between capacity and feasibility, I think the authors should include the state-of-the-art method in the analysis, both theoretically and in the main experiments. In particular, umbral-half-space shows that the trade-off is not a consequence of transitivity itself, as the abstract claims, but of the specific class of cones considered (axially symmetric around the origin of the ball). This is not a big ask, since Shadow Cones are not novel either: they are at least two years old.

Below are some minor criticisms: The gains depend on a weak baseline. The large improvements in the main table are obtained under the protocol of Ganea et al. Under the protocol of Shadow Cones, the gains largely shrink, and on WordNet nouns the original cones win at 25% non-basic edges or more (Appendix L). This suggests that much of the headline improvement comes from a training setup in which the original cones perform poorly, rather than from the aperture itself.

The downstream benefit is limited. In MERU, zero-shot classification and retrieval show no consistent trend, and the hierarchy metrics improve only slightly, over three seeds. The drop-in argument for vision-language models is therefore not yet supported by a practical gain.

Questions for authors

How do the authors position their theory relative to umbral-half-space cones, which avoid the infeasible region while satisfying exact transitivity?

7 comments
AuthorsOpenReview ↗

Official Comment by Authors (1/5)

We thank the reviewer for the careful reading and for pointing out the relation to Shadow Cones, which we agree must be made explicit. Since the question and the main weakness concern the same point, we answer them together.

W1 and Q. Relation to Shadow Cones (umbral-half-space cones)

We agree with the reviewer that umbral-half-space cones avoid the infeasible region while being exactly transitive, and that our trade-off is therefore not a consequence of transitivity alone. Our Proposition 3.2 holds for the class of cones whose axis points away from the origin and whose aperture depends only on the norm, i.e. cones that are rotationally symmetric about the origin of the ball. Umbral-half-space cones escape the trade-off by leaving this class: their light source is an ideal point, so in the Poincaré ball all cones are oriented toward one boundary point instead of the origin. We have stated this scope in the abstract (Section 3 already defines the cones by axial symmetry and rotation invariance about the origin, which is exactly this class), and revised the introduction to say that the trade-off has been resolved only partly, by formulations that change the geometry of the cones, and remains open for cones in the Poincaré and Lorentz models.

We nevertheless think this class is the one that matters for applications. The hyperbolic vision-language models built on entailment cones (MERU, HyCoCLIP, UNCHA) all assume that the origin is the generic side and that specificity grows with the distance from the origin; for example, generic texts are placed near the origin. Umbral-half-space cones put the root at an ideal point, so this premise does not hold, and the cones cannot be substituted into these models as they are. Our cones, in contrast, are a drop-in replacement of the aperture in any such model (as Reviewer 2pBD also notes, they keep general concepts near the origin).

We tested this directly by replacing the entailment cones of MERU with umbral-half-space cones of three radii (ViT-S/16, GRIT, seed 1; bold marks the best in each column).

Zero-shot classification accuracy (%) (Table 9 of the revision)

ImageNetFood-101CIFAR-10CIFAR-100CUBSUN397CarsAircraftDTDPetsCaltech-101FlowersSTL-10EuroSATRESISC45Country211Avg.
(ε,0)(\varepsilon,0)33.845.460.134.016.546.05.92.017.643.172.523.689.835.938.34.435.6
(0,δ1)(0,\delta_1)34.846.264.037.615.046.16.12.919.542.072.218.991.233.937.74.335.8
(0,δ2)(0,\delta_2)35.145.165.837.315.147.06.12.718.946.673.120.089.647.740.64.037.2
umbral-HS, r=0.1r=0.133.445.460.036.412.544.56.01.615.741.372.219.589.636.336.33.834.7
umbral-HS, r=0.5r=0.533.445.964.337.812.645.55.43.117.745.571.522.290.131.739.63.735.6
umbral-HS, r=2.0r=2.031.848.565.635.310.145.25.81.816.046.371.620.289.638.633.64.035.2

Image-text retrieval Recall (%) (Table 10 of the revision)

COCO T$\to$I@5COCO I$\to$T@5COCO T$\to$I@10COCO I$\to$T@10Flickr T$\to$I@5Flickr I$\to$T@5Flickr T$\to$I@10Flickr I$\to$T@10Avg.
(ε,0)(\varepsilon,0)52.567.364.277.979.589.686.894.276.5
(0,δ1)(0,\delta_1)52.066.663.776.777.989.086.193.975.7
(0,δ2)(0,\delta_2)52.567.564.177.878.387.286.192.675.8
umbral-HS, r=0.1r=0.152.366.363.877.477.988.485.893.475.7
umbral-HS, r=0.5r=0.552.066.163.977.278.788.686.593.375.8
umbral-HS, r=2.0r=2.052.366.764.177.579.288.685.893.676.0

ImageNet hierarchy metrics (Table 10 of the revision)

TIE ↓\downarrowLCA ↓\downarrowJaccard ↑\uparrowHier. prec. ↑\uparrowHier. rec. ↑\uparrow
(ε,0)(\varepsilon,0)4.2752.4620.7410.8230.820
(0,δ1)(0,\delta_1)4.1282.3990.7510.8300.828
(0,δ2)(0,\delta_2)4.1342.4430.7510.8280.830
umbral-HS, r=0.1r=0.14.3682.5180.7360.8190.816
umbral-HS, r=0.5r=0.54.3622.5090.7360.8180.816
umbral-HS, r=2.0r=2.04.5732.6080.7230.8080.806
AuthorsOpenReview ↗

Official Comment by Authors (2/5)

(W1 and Q continued)

With umbral-half-space cones, all five hierarchy metrics are worse than with the original cones for every radius, and worse than with both proposed cones; the classification average is also lower than with the proposed cones (34.7–35.6 vs. 35.8 / 37.2), while retrieval is similar for all models. In addition, umbral cones are not geodesically convex because of their hypercycle boundary (Yu et al., 2024, Theorem 3.3), whereas the cones in our class are. For these reasons we think that developing cones in the Poincaré and Lorentz models remains meaningful, and our contribution is to remove the trade-off within this class.

We thank the reviewer for this question, which has made the difference between our cones and umbral-half-space cones much clearer. We have added a new appendix section (Appendix M, "Relation to Umbral Half-Space Cones") that discusses this difference, covering the class of cones addressed by our theory, the premise of origin-centred models, and the experiment above. Given these results, we think that umbral-half-space cones, which lie outside this class and cannot be substituted into origin-centred models, do not need to be included in the main table, which compares cones within this class, and that keeping them as a reference in Appendix L together with the new section is appropriate. We would appreciate the reviewer's view on this.

AuthorsOpenReview ↗

Official Comment by Authors (3/5)

W2. Dependence on the training protocol

To separate the effect of the protocol from that of the aperture, we first ran a one-at-a-time ablation on WordNet nouns, MCG and Hearst-25k (50% non-basic edges, d=5d=5, one seed). Starting from the protocol of Ganea et al., each row of Table 12 of the revision (reproduced below) switches only one of nine components (initialization, loss, margin, optimizer, number of epochs, batch size, training directions, negative resampling, and the projection shell) to its Shadow Cones counterpart, and the last row switches all of them. Entries are test F1 under the containment criterion at the validation-selected KK and learning rate, with the change from the base row in parentheses.

WordNet nouns (50%, d=5d=5, seed 1)

(ε,0)(\varepsilon,0)(0,δ1)(0,\delta_1)(0,δ2)(0,\delta_2)
base (Ganea et al.)90.889.292.2
init: warm start →\to cold + burn-in 20 ep32.9 (−57.8-57.8)81.2 (−8.0-8.0)81.6 (−10.6-10.6)
loss: hinge + angle →\to CE + distance79.0 (−11.7-11.7)85.7 (−3.5-3.5)84.0 (−8.2-8.2)
margin: 0.01 →\to 0.00187.5 (−3.2-3.2)86.7 (−2.5-2.5)86.9 (−5.2-5.2)
optimizer: RSGD →\to RAdam94.4 (+3.7+3.7)94.3 (+5.1+5.1)94.9 (+2.8+2.8)
epochs: 200 →\to 40092.8 (+2.0+2.0)90.9 (+1.7+1.7)92.9 (+0.8+0.8)
batch size: 10 →\to 1691.8 (+1.1+1.1)90.2 (+1.0+1.0)92.2 (+0.1+0.1)
directions: forward →\to both89.9 (−0.9-0.9)90.0 (+0.8+0.8)91.5 (−0.7-0.7)
negatives: resampled →\to sampled once89.9 (−0.9-0.9)90.2 (+1.0+1.0)92.1 (−0.0-0.0)
shell: ε+10−5→ε+0.01\varepsilon+10^{-5}\to\varepsilon+0.0187.3 (−3.4-3.4)––
all switched (Shadow Cones protocol)87.5 (−3.3-3.3)85.2 (−4.0-4.0)82.9 (−9.3-9.3)

MCG (50%, d=5d=5, seed 1)

(ε,0)(\varepsilon,0)(0,δ1)(0,\delta_1)(0,δ2)(0,\delta_2)
base (Ganea et al.)43.266.866.8
init: warm start →\to cold + burn-in 20 ep65.1 (+21.9+21.9)78.6 (+11.8+11.8)76.8 (+10.0+10.0)
loss: hinge + angle →\to CE + distance76.4 (+33.2+33.2)75.1 (+8.3+8.3)74.3 (+7.5+7.5)
margin: 0.01 →\to 0.00140.9 (−2.3-2.3)58.7 (−8.1-8.1)53.6 (−13.1-13.1)
optimizer: RSGD →\to RAdam84.6 (+41.4+41.4)83.7 (+16.9+16.9)82.9 (+16.2+16.2)
epochs: 200 →\to 50066.3 (+23.1+23.1)72.6 (+5.8+5.8)72.9 (+6.2+6.2)
batch size: 10 →\to 1647.8 (+4.6+4.6)67.1 (+0.3+0.3)63.6 (−3.2-3.2)
directions: forward →\to both45.2 (+2.0+2.0)64.9 (−1.9-1.9)57.2 (−9.6-9.6)
negatives: resampled →\to sampled once43.2 (+0.0+0.0)67.2 (+0.4+0.4)64.2 (−2.6-2.6)
shell: ε+10−5→ε+0.01\varepsilon+10^{-5}\to\varepsilon+0.0174.4 (+31.1+31.1)––
all switched (Shadow Cones protocol)84.9 (+41.7+41.7)86.4 (+19.6+19.6)84.7 (+17.9+17.9)

Hearst-25k (50%, d=5d=5, seed 1)

(ε,0)(\varepsilon,0)(0,δ1)(0,\delta_1)(0,δ2)(0,\delta_2)
base (Ganea et al.)59.776.277.6
init: warm start →\to cold + burn-in 20 ep69.3 (+9.6+9.6)85.9 (+9.7+9.7)85.3 (+7.6+7.6)
loss: hinge + angle →\to CE + distance82.0 (+22.3+22.3)76.9 (+0.7+0.7)77.4 (−0.2-0.2)
margin: 0.01 →\to 0.00159.5 (−0.2-0.2)68.6 (−7.6-7.6)58.3 (−19.3-19.3)
optimizer: RSGD →\to RAdam88.7 (+29.0+29.0)88.9 (+12.7+12.7)88.3 (+10.7+10.7)
epochs: 200 →\to 50079.7 (+20.0+20.0)78.4 (+2.3+2.3)80.3 (+2.6+2.6)
batch size: 10 →\to 1664.6 (+4.9+4.9)78.3 (+2.1+2.1)78.5 (+0.8+0.8)
directions: forward →\to both61.4 (+1.7+1.7)72.5 (−3.6-3.6)73.3 (−4.3-4.3)
negatives: resampled →\to sampled once60.3 (+0.6+0.6)77.0 (+0.8+0.8)77.1 (−0.5-0.5)
shell: ε+10−5→ε+0.01\varepsilon+10^{-5}\to\varepsilon+0.0183.8 (+24.1+24.1)––
all switched (Shadow Cones protocol)88.5 (+28.8+28.8)89.2 (+13.0+13.0)88.8 (+11.1+11.1)

Among the nine components, changing the optimizer from RSGD to RAdam has the largest effect for every cone on every dataset (e.g., +41.4+41.4 / +16.9+16.9 / +16.2+16.2 on MCG and +29.0+29.0 / +12.7+12.7 / +10.7+10.7 on Hearst-25k for (ε,0)(\varepsilon,0) / (0,δ1)(0,\delta_1) / (0,δ2)(0,\delta_2)).

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.