acceptodds
Withdrawn from ICLR 2027

Task Entropy Must Be Paid: When Negative Curvature Helps Representation Learning

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

When does negative curvature become a useful representation resource rather than an architectural preference? A hierarchy alone does not decide this: the same input can induce a fine task that must preserve exponentially many distinctions or a coarse task that collapses most of them. We formalize this task-side demand through task entropy, the packing growth of a loss- or evaluation-induced pseudometric. For packings whose evaluation contains an -way margin-identification restriction, we prove an exact capacity law: with unrestricted coordinatewise -Lipschitz scores, a radius- representation supports exactly identities at margin . This yields a packing-restricted margin-error bottleneck, Euclidean resource tradeoffs, and a hyperbolic curvature threshold. On a regular tree, Euclidean obstruction and hyperbolic realization are governed by the same exponential leaf-growth rate, while a coarse task on the same hierarchy has zero task entropy and a bounded Euclidean realization. On a controlled 800-node WordNet intervention, Poincar\'e embeddings reduce hierarchy-task stress in all five seeds (); frozen-embedding margin-error curves provide a post-training identification diagnostic. Five real networks retain both positive and negative cases, probing task dependence and boundary cases highlighted by the capacity analysis rather than a universal optimizer-performance law.

Primary area:
learning theory
Keywords:
dependency graph, Negative Curvature, Representation Learning

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_oKguOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposed a metric Task Entropy to evaluate the task-side representational demand. For an N-way identification task, the authors showed that the number of classes that can be separated with a fixed margin is determined by packing capacity of the representation space. They then use the concept to compare the euclidean and hyperbolic spaces and empirically supported their results on WordNet data.

Critical strengths
  • The perspective of using task demand to re-visit the representation space capacity is interesting.

  • The main capacity results are clean and easy to interpret.

  • The authors are careful about the interpretation of the empirical results.

Critical weaknesses
  • The main theorem rely on the assumption that each task state has its own score coordinate and that success can be formulated as N-way margin identification. It is unclear how relevant the real representation learning objectives satisfy this.

  • The WordNet results do not clearly show that higher task demand gives a larger hyperbolic advantage. The hierarchy-versus-coarse comparison is seed-sensitive and reverses under the K=22 task definition, and the experiment uses metric stress rather than the identification objective assumed by the theorem.

  • The task metric, the resolution, and the scale parameter can all change the packing rate. It is unclear to what extent task entropy is an intrinsic property of the task itself.

Overall, these concerns limit the generality and empirical support of the proposed framework, which is why I lean toward weak rejection despite finding the core idea interesting.

Questions for authors
  • How sensitive is task entropy to the choice and normalization of the task pseudometric?

  • What role does the tree structure actually play in Theorem 9 beyond providing exponentially many leaves?

Reviewer_fgkcOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

The paper is about learning tasks, entropy, and negative curvature of (representation) spaces. Aside from this, the paper is not human-readable, so an LLM should be consulted to obtain a proper summary. I decided to refrain from using LLMs, as recommended by the ICLR reviewing policy.

Critical strengths

None

Critical weaknesses

The paper is not written for humans to read. I can't rule out that an LLM would find it interesting or significant, but I have not inquired.

For example, the introduction proposes the following core question:

When do task-required distinctions exceed Euclidean capacity under controlled dimension, radius, margin, and readout regularity—and when can negative curvature close that deficit?

What is a distinction? What does it mean for it to “exceed Euclidean capacity”? What does it mean for dimension, radius, margin, and readout regularity to be controlled? What margin are we talking about? What is readout regularity?

Overall, the introduction essentially fails to introduce the problem and the results to the (human) reader (or, at least, to me). Section 3 (Problem Setup and Task Entropy) also fails to introduce the problem. For instance, its opening sentence is:

The purpose of the formalism is simple: specify which target distinctions matter, then compare their demand with the number of margin-separated states a representation can support.

What formalism? Is the paper introducing a new formalism, or are we talking about an existing formalism that is used to solve some problem? What is a target distinction? What is its demand? What are margin-separated states? What does it mean for a representation to support them?

Skipping essentially all text (since it is not readable), one can read through the formulas alone. (1) introduces a sequence of finite pseudometric spaces, all subsets of a common ambient set χ\chi. (2) introduces the packing number of a pseudometric space (number of spheres that can be packed into the given space). (3) and (4) introduce the “task entropy”, which is a limit that exists finite when the sequence of spaces in (1) has a packing number that more or less grows exponentially. One could then continue; the math seems reasonably simple, and one could use the equations to deduce what the paper’s narrative ought to be, but I don’t think a reader should be required to do such a deduction exercise.

Overall, the writing style of the entire paper is that of poorly-written AI-generated text, with its annoying tendency to pack information without explaining what things actually mean.

Questions for authors

I would like to see a human-written and human-understandable paragraph that explains what is being done in this paper.

Reviewer_sTLKOpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper investigates when hyperbolic representations provide an advantage over Euclidean representations by considering the number of distinctions a downstream task must preserve, rather than relying only on the hierarchical structure of the input. The authors introduce task entropy, defined through the growth of packing numbers under a task-induced pseudometric, and derive an exact relationship between representation-space packing capacity and margin-based identification under Lipschitz-constrained readouts.

Using this framework, the authors establish resource tradeoffs between representation dimension, radius, margin, and readout smoothness, along with a curvature threshold under which hyperbolic space can accommodate exponentially growing identification demands. They illustrate these results theoretically on regular trees and empirically through controlled WordNet experiments, five graph datasets, and additional synthetic identification experiments. The results suggest that hyperbolic representations can be advantageous for fine-grained identification tasks, while coarser tasks on the same inputs may not require negative curvature.

Critical strengths

S1. The paper makes an important distinction between the hierarchical structure of input data and the distinctions required by a downstream task. By introducing task entropy, the authors formalize why the same hierarchical input can create very different representation-capacity requirements depending on the evaluation objective. The resulting framework provides a useful way to reason about when negative curvature may be beneficial rather than treating hyperbolic geometry as universally preferable for hierarchical data.

S2. Theorem 3 establishes an exact relationship between metric packing and margin-identification capacity under coordinatewise Lipschitz readouts. The subsequent results characterize how dimension, radius, margin, and curvature affect the ability to preserve task identities.

Critical weaknesses

W1. Limited theoretical novelty relative to existing work on hyperbolic representation capacity.

  • W1.1. Theorem 3 follows from standard packing and Lipschitz-separation arguments, with the converse obtained using distance-to-prototype scores. Similarly, the Euclidean–hyperbolic separation relies on classical differences in packing growth. Prior work, including Sala et al. (2018), has established stronger tree-embedding guarantees and representation tradeoffs.

  • W1.2. Rawal and Vishwanath (2026) establish an exponential Euclidean–hyperbolic sample-complexity separation for hierarchical learning under Lipschitz regularization. While the present paper considers a different, deterministic identification-capacity setting, it remains unclear what substantial new insight its results provide beyond these established geometric and statistical separations.

W2. The empirical evaluation does not convincingly establish the proposed framework as a useful principle for selecting representation geometry.

  • W2.1. The main theorems concern margin-identification capacity under explicit dimension, radius, margin, and Lipschitz constraints. However, the real-data experiments primarily evaluate distance-fitting stress and retrieval performance without matching these theoretical budgets across geometries.

  • W2.2. On WordNet, Poincaré embeddings improve hierarchy-task stress by 4.47%, but the difference between hierarchy and coarse-task gains is not robust across seeds. An alternative coarse-task construction reverses the ordering. The five additional graph experiments also produce mixed results.

  • W2.3. Although Appendix D provides controlled synthetic evidence for the capacity separation, the real-data results do not establish that task entropy reliably predicts when negative curvature improves learned representations.

Questions for authors

Q1: Can the authors clarify which theoretical conclusions are not already implied by established metric-packing arguments and previous Euclidean–hyperbolic representation bounds, particularly Sala et al. (2018) and Rawal and Vishwanath (2026)?

Q2: How should the reported sensitivity of the WordNet results to the coarse-task construction be interpreted? Can the authors identify conditions under which differences in task packing translate into consistent differences in empirical geometry performance, rather than simply differences in theoretical capacity?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.