acceptodds
Under review as a conference paper at ICLR 2027

The Intrinsic Geometry of a Task

Abstract

Understanding why deep learning models generalize on a task requires explaining how their internal representations encode information relevant to that task. Although empirical studies have revealed regularities in representation geometry, a theory connecting generalization to this geometry remains incomplete. We study this connection using the Fisher–Rao geometry induced by cross-entropy: a task already has a geometry, and that geometry is the reference a representation must match. A representation is sufficient if it recovers the task's conditional laws, and canonical if it is also minimal. Sufficiency implies isometry to this geometry; local deviations from a sufficient predictor determine excess loss. Pulling the same geometry back to parameter space yields local loss bounds and a criterion for pruning. Experiments confirm the theorems in the regime they assume and show that pullback residual length predicts pointwise excess loss. We then operationalize this geometry through pruning experiments.

Primary area:
interpretability and explainable AI
Keywords:
interpretability, deep learning theory, learning theory, representation

Reviews

Among:
Shape:
1234this paper, Bottom 1%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.00 over 3 reviews, above 0% of the others.
Reviewer_rbtXOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

The paper is trying to study the geometry of classification task in a very general way.

Critical strengths
  • Working on latent geometry is important and useful.
Critical weaknesses
  • The paper is short (only 8 pages of the main submission). I assume the authors mistakenly thought that this is the limit, nevertheless, the extra page could be helpful in several places like elaborate on the motivation more in the intro, more accurate definitions on the mathematical side, extra care when variables are introduced. The most low covered region is the Related Work section where I think that much more has been studied in the core of latent geometry, even when aiming to be task-oriented. The most prominent missing citation for me is this [1], but the authors should read and place their work more precisely within the already done research.

  • Writing level which is not up to top-tier conference level. The abstract is too general, talking generally on learned systems where the literature is focusing on specific regimes, making it vague regarding which type of machine learning methods the method is dealing. It dives fast to the details without explaining the motivation and rationale, sentences are seem to me like they out of context "Sufficiency implies isometry to this geometry; local deviations from a sufficient predictor determine excess loss". Some are generally vague for example: "Because a model is compositional, generalization at its output depends on its internal processes." - which model? Every model? or this: "we take a normative approach to this problem" - what a normative approach is? another example: ".Write PX for the marginal of X" - what does it mean? do you mean that you term P_x as the marginal of X?

  • The math formulations lack explanation for each variable. Walk over each definition and make sure you first introduce it before using it, moreover it would be appreciated if you would add a dimension for main variables.

  • The paper claims that it is solving a very general tasks, but at the end, as far as I understand it - is limited specifically to a certain loss (logarithmic), under specific constraints.

refs:

[1] INTO THE RABBIT HULL: FROM TASK-RELEVANT CONCEPTS IN DINO TO MINKOWSKI GEOMETRY. Fel et al.

Questions for authors
  • Classification is recovered by taking n=1. Do you mean by argmax?

  • "First we place a predictor exactly on the task, and just off it. Then we ask where the loss of a trained model comes from." - I dont understand nothing from this sentences, what do you mean by "just off it"? what is the second sentence even mean? what is the loss that the trained model comes from? I dont get it at all.

I feel like the paper is not aim for people but to AI agents. I didnt manage to follow majorities of the authors intentions - where I think the paper is lack of any intuition behind things, the textual non-mathematical parts are at the most not readable, the math derivations are not coherent and full, where it has holes that I dont know to fill. I think that the paper need a major revision. Sorry about that.

Reviewer_mHUHOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
1
Soundness
2
Read review
Paper summary

The paper defines a task's geometry as its image τT(X)⊂Δ(Yn)\tau_T(\mathcal X)\subset\Delta(\mathcal Y^n) of conditional laws, equipped with the KL divergence and the Fisher–Rao metric.

They define canonical representation ϕ\phi if some readout π\pi gives π∘ϕ=τT\pi\circ\phi=\tau_T and ϕ\phi is constant on the fibers of τT\tau_T.

I have very limited understanding of the following theory due to poor writing.

Experiments use a synthetic hidden semi-Markov language with known conditionals (GRU/Transformer/MLP), ChaosNLI-M and CIFAR-10H with human label distributions, and one-shot structured pruning.

Critical strengths

The question is well posed: what model-independent reference should internal representations be compared against?

The experiments are sensible. The synthetic language has exact conditionals, and ChaosNLI-M and CIFAR-10H supply human label distributions.

Critical weaknesses

I think the paper is poorly written. It is full of jargons, many without clear definition. As authors claimed, this paper was drafted by AI. I don't think it has been carefully polished for human reading.

I give some examples here:

For example, in introduction: I don't understand "post-hoc analysis uses the model itself to investigate the task". It is completely unclear to the reader what it means by "use a model to investigate the task".

Line 72 use sequence simplex, which seems to be Δ(Yn)\Delta(\mathcal{Y}^n). I don't get the sequence part. Is it because n>1n > 1. Why is it a sequence

Line 77: The loss ll equips the statistical manifold with a divergence between conditional laws. What is a conditional law? What do we mean by "divergence between conditional laws"

In Definition 3, JJ is never defined. I am guessing it is Jacobian.

Definition 1 about geometry is the core of the paper, but it only gets a very informal definition without rigorous math language.

In general, the writing is very poor that makes it extremely challenging to follow.

Based on my limited reading, the main results either follow directly from the definitions or are standard.

The experiments are very toy.

Questions for authors
  1. Task definition has a loss. I don't understand why we have to use KL for generalization instead of using the task's own loss.

  2. I don't see how canonicity, as opposed to sufficiency, used in any result. Why do we define it?

Reviewer_enBAOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

The paper argues that a classifier (or sequence classifier output) task defines a geometry in the output space simplex (product of simplices) and the geometry can be locally related to the Fisher geometry in weight space. The geometry is thought to help understand generalization and model optimization.

Critical strengths

Problem is highly relevant, would be very interesting to have simple geometric mechanisms explaining generalisation

Critical weaknesses

The link between the local geometry at a perfect solution and generalisation is not established convincingly. Writing is very brief and it would have been useful to use the full set of 9 pages allowed.

Questions for authors

Please clarify the contribution and its significance for generalization

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.