acceptodds
Under review as a conference paper at ICLR 2027

Sparse Prototype Code Underlies Classification and Prediction Across Modalities

Abstract

Neural representations have become a central tool for studying the internal mechanisms of modern AI models, yet their complex high-dimensional structure makes it difficult to identify the geometric features that determine predictive performance. We identify a universal representational geometry shared across state-of-the-art models in vision, audio, and language, spanning tasks from object recognition to next-token prediction. Its defining feature is that within-class variability is strongly structured: its task-relevant component is aligned with the class’s own centroid and the centroids of competing classes. Building on this observation, we derive an analytical mean-field theory governed by the statistics of projections along true-class and rival-class centroid directions, together with a global renormalization of the class radius that compensates for the non-Gaussianity of real representations. The resulting theory accurately predicts per-class accuracy across tasks, architectures, and modalities. The relevant geometric quantities improve systematically with model scale, revealing how larger models achieve higher accuracy even though their overall within-class variability often increases. A striking feature of this framework is its sparsity: accurate predictions require only a small set of centroid coordinates associated with the true class and its strongest rivals. These rivals are often semantically related to the true class, suggesting a connection to sparse-feature extraction approaches such as sparse autoencoders. Together, these results provide a parsimonious predictive theory of neural representations and suggest that classification and prediction in deep networks are governed by a sparse, centroid-aligned structure embedded within the full high-dimensional representation space.

Primary area:
learning theory
Keywords:
representational geometry, neural manifolds, prototype classification, mean-field theory, sparse coding, statistical mechanics, deep learning

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 4 reviews, above 85% of the others.
Reviewer_1oN5OpenReview ↗
Rating
4(+2)
Confidence
2
Informativeness
3
Presentation
3
Soundness
3(+1)
Read review
Paper summary

The paper exposes geometric structures in trained representations that are relevant to classification and can predict classification accuracy. Specifically, residuals (representation minus centroid) are largely aligned with centroid directions. The most prominent centroids correspond to the true class as well as a few rival classes that are semantically related. This property of the residuals is highly pronounced as a random rotation about the true centroid axis destroys confusion and yields 99.999% classification accuracy. This is tested in the paper across training tasks, modalities, models, and model sizes. The paper provides solid math insight as it starts with a naive isotropic model and its properties, and then shows how a real model significantly differs in behavior step by step, revealing clear properties about the real model and its representations. The formulated theory according to grounded observations can predict the classification accuracy given the representations from the geometric perspective alone. No source code is released.

Critical strengths
  • The writing is incredibly clear. The paper masterfully organizes various related nontrivial concepts into an easy-to-follow path.
  • Proposing the isotropic model to compare with real models is an elegant approach. All but one of the perturbations in Section 4 and their interpretations are sound. The resulting observations are salient, insightful, and general.
Critical weaknesses
  • Missing source code! The perspectives and experiment results strongly imply a sound research process but the source code is not released. Source code is important for this kind of paper, so not releasing code significantly undermines its contribution. The submission would otherwise warrant my Strong Accept, assuming a sound source code is released, and the reviewers' questions are addressed.
Questions for authors
  • It will be very interesting to see the distribution of residuals' projection onto the true centroid, especially their sign. Are most positive? Fig 2b implies that most are positive. This would mean the centroid of any class tends to undershoot the cluster of correctly classified samples. This would be intuitive but there has not been any documentation of this phenomenon.
  • More padding needed below Figures.
  • Should "class variability" be "within-class variability"? For outsiders, "class variability" may look like the variance in the class label.
  • L075 "The centroid-aligned correlations are therefore precisely the geometric source of classification errors." needs adjustment. The centroid-aligned correlations, which are the projections of residuals onto directions of other class centroids, are by definition the cause of misclassification. Even unaligned residuals follow this description. The authors' results showed that residuals are mostly aligned. Therefore the authors probably want to say "The geometric source of misclassification is not uniform divergence from the true centroid, but instead, targeted deviations towards rival class centroids".
  • L079 "see Fig. 1a, Table 2" should be "see Fig. 1a and Table 2".
  • L102 should be "P centroids using the ``Leave one out'' method (Lachenbruch & Mickey, 1968)." Parenthesis would imply the previously described method was the leave-one-out method.
  • Either move R = R_k before eq 2, or change eq 2 to use R_k instead of R and use R in the future. Both options are good. Same for \sigma.
  • The same operation is sometimes written as EkE_k and sometimes Ex∈MkE_{x \in M_k}. Select one and commit. (Recommendation: use the longer, standard form, and remove the "in addition" on L157.) Also fix definition 5.2, which is even more problematic since the two xxs are invisibly different.
  • Most of the paper uses k for true label and \mu for rival label. Consider changing \mu in Def 2.1 to k for consistency. Then \mu does not need another definition and is fine to show up in c^μ\hat{c}_\mu (L152) as the first appearance since c hat has been defined to take a class subscript.
  • Table 1. If you still need to save space, consider removing the "Notation" header.
  • L232 Consider "Fig.2b blue line". This would contrast L257 "Fig.2b".
  • Fig 2b: What are the two vertical lines?
  • "Breaking residual–centroid correlations recovers the isotropic model." Indeed striking. Do you have a hypothesis as to why? What happened to the 39.2% points correctly classified before delta-rotation at large R*?
  • "Centroid-centroid correlations play a minor role" cannot be concluded. Since the first-stage delta-rotation already modified the real models' behavior into the isotropic model's behavior, further intervening in centroid-centroid correlations reveals nothing about the real models. Please point out my confusion if I'm wrong.
  • Which part in Sec.4 illustrated the sparseness of relevant centroids? This is a dependency of L269, and then more importantly eq 4.
  • Consider adding visualization of σμ2\sigma^2_\mu, which is supposedly sparse. Maybe a bar plot. Plot the isotropic model as baseline.
  • Eq 4: What is bold s?
  • L292 may benefit from calling e some kind of residual since e may look unfixed and confusing.
  • L072 is near the beginning of Introduction and should prefer "residual" over "variability" for outsider friendliness. Maybe "Its defining property is that within-class residuals are strongly aligned with the true-class centroid and the centroids of a few competing classes." Using the word "variability" before equations 2.x is risky because the word by itself is a quantity and a property at the same time.
  • L077: "... small set of rivals, which is sufficient to reproduce the predicted per-class accuracy" is linguistically problematic for the unclear "which" and the overcomplicated "reproduce prediction". Maybe "... small set of rivals. Using only a few rivals, our theory predicts per-class classification accuracy from the geometric perspective."
  • L061: "Our analytical lens is prototype-based classification." might be unnecessarily weak and might lead first time readers (including me) to think the experiments are on prototypical classifiers only. Maybe spell out "We use the lens of prototype-based metric(?) learning to study classifiers and language models." Similarly, L100 "We use" -> "We consider" may help.
  • L137: "In autoregressive models this is the embedding of the token preceding the word." This actually refers to the word embedding, which is much less exciting and not what you mean. Change it to "activations" or "hidden states".
  • Fig 2 caption: Consider "Per-class accuracy vs. per-class empirical radius R". During my first read I didn't realize the Rs were natural and instead kept asking "What's the intervention that changed R, that is not R*?"
  • Fig 2: Is the "null uncorrelated model" the isotropic model? Update legend?
7 comments

(1/3)

We thank the reviewer for their detailed review, for recognizing the value of our work, and for the kind words about the writing. The reviewer is absolutely right that source code is essential for this work. The code is now uploaded both to the submission, as a zip file in the supplementary material, and to this anonymous GitHub repository: https://anonymous.4open.science/r/sparse-prototype-code-8B54/. The code takes a representation matrix and a label vector, as numpy arrays or PyTorch tensors, and an optional list of class names, and produces the results of the paper for that representation: Fig. 1 (visualization), Fig. 2 (accuracy and geometric manipulations), Fig. 3 (sparsity and the statistics of s), Fig. 4 (theory and experiment comparison), Table 1 (top rivals), its row in Table 3 (geometric measures across scales), in Table 4 in the appendix (additional geometric measures) and in the theory table of Appendix I, the rival-selection tables of Appendix F, and the theory error against the number of rivals K (Appendix F.3). In addition, a standalone script simulates the synthetic isotropic model and generates Fig. 6a (accuracy against theory). The main logic of the code is in geometry.py, theory.py, manipulations.py and rivals.py, which we hope are sufficiently clear and well documented; we will gladly answer any questions about them. The code follows the notation of the paper as closely as possible.

Regarding the questions, we answer the main scientific ones below; the points about notation and corrections are already incorporated in the revision, and we are grateful for them.

It will be very interesting to see the distribution of residuals' projection onto the true centroid, especially their sign.

By definition, the residual projections always have zero mean. However, the reviewer's intuition is correct: their distribution is skewed, such that its peak is at a positive value (sk≈0.55s_k \approx 0.55 in DINOv3 ViT-7B, in units of its standard deviation; 59% of all examples project positively), balanced by a long negative tail (skewness −0.98-0.98). It is also true that for correctly classified examples the distribution has a positive mean, sk=0.22s_k = 0.22 in DINOv3 ViT-7B, and a negative mean for misclassified examples, sk=−1.34s_k = -1.34. This means that even though the centroid is, by construction, the "center of mass", in well-performing networks such as DINOv3 a large mass overshoots the centroid with positive projections, while a smaller mass of negative projections has a longer tail that balances them. The misclassified examples sit almost entirely in this tail: 90% of them have a negative projection, and 78% of the examples with sk<−2s_k < -2 are misclassified, against 2.4% of the examples with sk>0s_k > 0 (recall that the logit of the true class also contains the centroid term +1+1, against $g_\mu $ for a rival, so an example is misclassified only when its projection on a rival exceeds its projection on the true centroid by more than 1−gμ1-g_\mu). The same holds in Qwen3-32B, where 74% of the correct and 30% of the misclassified examples project positively. This analysis was added to Appendix G (new Fig. 9), and is referred to in the text in Section 4.

Should "class variability" be "within-class variability"? For outsiders, "class variability" may look like the variance in the class label.

Correct. We fixed it in every place where this phrase was not accurate.

L075 "The centroid-aligned correlations are therefore precisely the geometric source of classification errors." [...]

We gladly adopt the more accurate description suggested by the reviewer.

Fig 2b: What are the two vertical lines?

The two vertical lines mark the empirical radius range, R between 0.4 and 4.3, which is R/NR/\sqrt{N} between 0.007 and 0.067 on the axis of Fig. 2b. The point is to show that in this range the original geometry behaves similarly to the empirical accuracy, decreasing from 100% to about 60%, while the rotated data keep perfect accuracy. We added a clarification to the caption, thank you.

(2/3)

"Breaking residual-centroid correlations recovers the isotropic model." Indeed striking. Do you have a hypothesis as to why?

Our hypothesis is that once the residuals are isotropic, the centroid correlations barely matter at any radius, because they enter the accuracy only through factors that are close to 1 for almost all rivals. This can be checked directly, since a model with isotropic residuals and the measured, correlated centroids can be solved with the same techniques as the model in Section 3 (Appendix F.4). Let the P centroids be fixed, with their measured overlaps gμ=c^k⋅c^μg_\mu = \hat{c}_k \cdot \hat{c}_\mu, and let the residuals be isotropic with radius R (Eq. 6 with σk=σμ=1/N\sigma_k = \sigma_\mu = 1/\sqrt{N} and ρμ=gμ\rho_\mu = g_\mu, since isotropic projections on two centroids are correlated exactly by their overlap). The accuracy of this model is

Acc(R)=Ez∼N(0,1)[∏μ≠kΦ(1−gμ1+gμ(z+NR))]\mathrm{Acc}(R)=\mathbb{E}_{z\sim\mathcal{N}(0,1)}\left[\prod_{\mu\neq k}\Phi\left(\sqrt{\frac{1-g_\mu}{1+g_\mu}}\left(z+\frac{\sqrt{N}}{R}\right)\right)\right]

where z is the standardized projection of the residual on the true centroid and Φ\Phi is the standard normal distribution function. It differs from the uncorrelated isotropic model only through the factors aμ=(1−gμ)/(1+gμ)a_\mu = \sqrt{(1-g_\mu)/(1+g_\mu)}. In DINOv3 ViT-7B these factors are close to 1 for almost all rivals: over all pairs of classes, the overlaps have a median of −0.005-0.005 and a 99th percentile of 0.10, and the median class has only 7 of its 999 rivals with aμ<0.9a_\mu < 0.9. As a result, along the whole range of radii in Fig. 2b, this model differs from the uncorrelated isotropic model by at most 1.7 percentage points in accuracy (at R⋆/N=0.25R^\star/\sqrt{N} = 0.25). This is why the δ\delta-rotated data follow the isotropic model, and why breaking the centroid correlations as well (the full rotation) changes their accuracy very little. At the empirical radii, N/R\sqrt{N}/R is large, and both models are essentially perfect.

This is the corrected statement in Section 4.1: the centroid-centroid correlations act as a quantitative correction that shifts the decision boundaries, whereas the residual-centroid correlations change the scaling of the problem. They replace the isotropic projections of order 1/N1/\sqrt{N} by the measured σμ\sigma_\mu (0.12 for the strongest rival in DINOv3 ViT-7B, against 1/N=0.0161/\sqrt{N} = 0.016), which moves the decline of the accuracy from R of order N\sqrt{N} to R of order 1.

What happened to the 39.2% points correctly classified before delta-rotation at large R⋆R^\star?

As for the 39.2% of the examples that are correct at large R⋆R^\star in the original data, the random rotations indeed remove their large positive projections on the true centroid, which keep them correctly classified there. However, at the empirical radii they stay correct, like the other examples, because the true-centroid term +1+1 is larger than the overall random variability. Since they now rely only on this term, at very large R⋆R^\star these examples, like all the rest, are classified at chance, and the accuracy decays to 1/P, unlike in the original geometry.

(3/4)

"Centroid-centroid correlations play a minor role" cannot be concluded. Since the first-stage delta-rotation already modified the real models' behavior into the isotropic model's behavior, further intervening in centroid-centroid correlations reveals nothing about the real models.

The reviewer is right that this statement is too strong, and the same point was raised by other reviewers; we revised it accordingly (Section 4.1). Unfortunately, a direct model-free geometric manipulation that breaks only the centroid correlations is not possible, since a rotation around the origin also breaks all the residual-centroid correlations except those with the true centroid. We therefore compared the two factors in the empirical regime indirectly in three other ways.

First, the choice of rivals is the most direct empirical test. For each class, we restrict the competition to its K top-ranked rivals and find the K at which this K-way problem reproduces the full P-way accuracy to within 1%. If the errors were driven by the centroid-centroid correlations, ranking the rivals by centroid proximity (gμg_\mu) would identify the true competitors with the fewest rivals. Instead, ranking them by the residual-centroid correlations (σμ\sigma_\mu), as in the paper, needs fewer rivals in every modality, as already noted in lines 342-343 of the original manuscript ("In particular, it outperforms selection based on centroid proximity, providing additional evidence that centroid correlations [...]"). The table gives the median K over the backbones of each modality, with the fraction of P in parentheses (Table 10 of the revision):

ModalityBackbonesPCentroid proximity (gμg_\mu)Residual projections (σμ\sigma_\mu)
Vision15100076 (7.6%)47 (4.7%)
Autoregressive LM14320102 (31.9%)89 (27.8%)
Masked LM432048 (15.0%)42 (13.1%)
Audio331060 (19.4%)39 (12.6%)

The true competitors of a class are thus the classes along whose centroids its examples vary the most, rather than the classes whose centroids are closest.

Second, the role of each type of correlation can be tested in the theory (Eq. 6). Setting the overlaps gμg_\mu to zero raises the RMSE of the predicted per-class accuracy in DINOv3 ViT-7B from 0.043 to 0.081, so the centroid correlations are not negligible and are needed for an accurate quantitative prediction. Removing the residual-centroid correlations instead, as in the model with isotropic residuals above, breaks the theory completely: it requires λ=9.8\lambda = 9.8 against 1.32 for the full theory, which effectively compensates for the N\sqrt{N} scaling of R, and still leaves an RMSE of 0.121 (Table 12).

Third, we intervened on the classifier directly. Removing the centroid-centroid terms c^μ⋅ck\hat{c}_\mu \cdot c_k from the rival logits, with the residuals unchanged, raises the accuracy from 85.5% to 91.5% in DINOv3 ViT-7B, while the δ\delta-rotation, which keeps the centroid correlations, raises it to 100%.

We stress that this discussion concerns only the qualitative question of which effect dominates. Our analysis did not neglect the contribution of the centroid correlations: the theory includes them explicitly, through the measured overlaps gμg_\mu, so it requires no change. The full discussion is now a new appendix section (Appendix F.4).

Reviewer_pSbZOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper investigates how the geometry of neural representations determines classification accuracy in a prototype classification setting. The authors show that classification performance is largely determined by competition with a small number of rival prototypes (roughly 5–6 for a close approximation). They find that within-class variability tends to have substantial projections along directions that discriminate a class from its competitors, making classification more difficult. Randomly rotating the within-class residuals in a high-dimensional space improves prototype classification accuracy from approximately 86% to 100% in their leave-one-out evaluation. The authors develop an analytical theory relating class centroids and within-class variability to prototype classification accuracy. The theory predicts per-class accuracy remarkably well across 36 models spanning vision, language, and audio. However, while the study provides some insights, its practical utility for classification/recognition in a general setting is uncertain as the rotation is class-specific and requires knowing the true class label of each sample being tested.

Critical strengths
  1. The main strength is the analytical theory that quantitatively relates manifold geometry to prototype classification accuracy, accounting for competition among multiple rival classes. The predictions agree remarkably well with empirical results across 36 models.
  2. The finding that classification depends mainly on a small number of rival prototypes is interesting, although perhaps not entirely surprising.
  3. The observation that larger models can achieve better classification accuracy even when their overall within-class variability increases is interesting. It suggests that how variability is organized relative to discriminative directions matters more than its overall magnitude.
Critical weaknesses
  1. The random-rotation result is interesting but not particularly surprising. In a high-dimensional space, randomly rotating within-class residuals would naturally reduce their projections onto a small number of discriminative directions and therefore improve classification.
  2. The practical relevance of the rotation experiment is unclear. The rotation is class-specific and requires knowing the true class label of each sample. For an unseen example, how would we know which rotation to apply?
  3. It is unclear how general the proposed theory is beyond prototype classification. For example, an SVM explicitly finds decision boundaries that maximize classification margins and can potentially exploit the existing geometry without requiring rotations.
  4. Although the authors show that prototype classification accuracy correlates with SVM accuracy, this does not establish that their geometric theory can also predict SVM performance.
  5. More generally, it is not clear whether the findings reveal a general organizing principle of learned representations or are largely consequences of the particular prototype classifier used in the analysis.
Questions for authors
  1. Can the same geometric theory quantitatively predict the classification accuracy of an SVM or other trained linear readouts?
  2. Could the authors repeat the class-specific random-rotation experiment using an SVM classifier? Would the rotation still improve accuracy substantially when the classification boundaries are already optimized?
  3. Is there a practical way to take advantage of the proposed geometric insights to improve classification? Since the random rotation requires knowing the correct class of each sample, how could something similar be implemented for unseen inputs?

Post-rebuttal note: I thank the reviewers for addressing my questions. Overall, I think the new results substantially strengthen the paper and would encourage the authors to include the generalized theory and SVM experiments prominently in the revised manuscript. It could potentially be a diagnostic tool. The proposed applications to training regularization and distillation are interesting, although their practical benefits remain to be demonstrated. But that is not a major issue. I increase the score to 3.

3 comments

(1/2)

We thank the reviewer for appreciating our analytical theory and the scale analysis.

Weakness 1

The random-rotation result is interesting but not particularly surprising. In a high-dimensional space, randomly rotating within-class residuals would naturally reduce their projections onto a small number of discriminative directions and therefore improve classification.

We are glad that the reviewer finds the random-rotation result unsurprising; this is the conclusion we hoped the text would lead to. We emphasized the experiment because of the classic representational-geometry literature (e.g. Chung et al. 2018, "Classification and Geometry of General Perceptual Manifolds"), in which the variability is random and the accuracy is set by its size (the radius), its dimension, and the number of classes relative to the dimension (α=P/N\alpha = P/N). None of these enter our theory: the accuracy depends on a few rivals rather than on all P classes, the radius appears only through the projections on the rival directions, and the dimension plays no role at all. The δ\delta-rotation is a model-free test of this difference: it keeps the radius, the dimension and P fixed and changes only the alignment of the variability with the centroids, yet the accuracy rises from 85.5% to 100% in DINOv3 ViT-7B.

Weakness 2

The practical relevance of the rotation experiment is unclear. The rotation is class-specific and requires knowing the true class label of each sample.

As stated in lines 242-243, the rotation uses the true label and is therefore a diagnostic tool, not a procedure for unlabeled examples. What it identifies, however, is actionable: the correlations between the variability and the rival centroids (the σμ\sigma_\mu) can be reduced directly during training, instead of targeting the overall variability, which matters less for the accuracy, or only the class distances, as some contrastive losses do indirectly. Section 6 shows that larger and better models already move in this direction: the strongest rival's σμ\sigma_\mu drops from 0.172 in DINOv3 ViT-S to 0.120 in ViT-7B, and from 0.241 in Qwen3-0.6B to 0.175 in Qwen3-32B (Table 3), along with their accuracy gains. We describe concrete ways to use this under Question 3.

Weaknesses 3-5 and Question 1

Can the same geometric theory quantitatively predict the classification accuracy of an SVM or other trained linear readouts?

Following the reviewer's suggestion, we generalize our theory to any linear readout, including an SVM. Replace the centroids by readout vectors wμw_\mu with biases bμb_\mu, and let Tk=wk⋅ck+bkT_k = w_k \cdot c_k + b_k be the readout of the true centroid. The centroid-readout correlations are then gμ=(wμ⋅ck+bμ)/Tkg_\mu = (w_\mu \cdot c_k + b_\mu)/T_k, the residual-readout correlations σμ\sigma_\mu are the standard deviations of wμ⋅δxw_\mu \cdot \delta x over the class divided by RTkR T_k, and sμs_\mu and ρμ\rho_\mu are defined as before from wμ⋅δxw_\mu \cdot \delta x. With these definitions the accuracy has exactly the form of Eq. 6, which it recovers for wμ=c^μw_\mu = \hat{c}_\mu and bμ=0b_\mu = 0. We measured these quantities on the held-out examples, as the paper does for the prototype classifier. The theory then predicts the held-out per-class accuracy of the SVM, with λ\lambda close to 1, as well as it predicts that of the prototype classifier on the same examples. The table compares the two readouts, with the Pearson r, slope and RMSE of the per-class theory (K = 20 rivals) against the held-out per-class accuracy, and the fitted λ\lambda:

BackboneReadoutHeld-out accuracyPearson rSlopeRMSEλ\lambda
DINOv3 ViT-7BSVM0.8860.9521.050.0351.09
Prototype0.8560.9460.970.0451.30
Qwen3-32BSVM0.5830.9470.950.0601.06
Prototype0.5640.9491.020.0701.11

The results and a figure are in a new subsection at the end of Appendix J. Together with the rotation experiments below, this shows that the theory governs trained linear readouts as well. The structure we describe is therefore not a result of a specific classifier, but of the organized geometry of the class clusters, whose within-class variability is strongly correlated with a few rival directions.

(2/2)

Question 2

Could the authors repeat the class-specific random-rotation experiment using an SVM classifier? Would the rotation still improve accuracy substantially when the classification boundaries are already optimized?

Yes, and the rotation still improves the SVM substantially (Appendix B.2, Table 5). We train the SVM on the δ\delta-rotated data, on 90% of the examples, and evaluate it on the rotated held-out 10%. The test accuracy is 100% in both DINOv3 ViT-7B and Qwen3-32B, against 88.6% and 58.4% when the SVM is trained and evaluated on the original data. The same holds when the held-out data are rotated by an independent rotation, so the SVM does not memorize the rotation. Isotropizing the within-class variability therefore removes the errors of the linear readout exactly as it does those of the prototype classifier, making the data easy to learn and generalize.

As an additional check, we apply the SVM trained on the unrotated data to the rotated held-out data. This unchanged SVM already benefits significantly, rising from 88.6% to 97.4% in DINOv3 ViT-7B and from 58.4% to 69.5% in Qwen3-32B. If we rotate each class around the SVM's own point instead of its centroid (the readout direction of the class at the norm of its centroid), even the unchanged SVM becomes essentially perfect (100% in both). More broadly, the errors of a linear readout come from the correlations between the variability and the readout vectors; once these are broken, the correlations among the readout vectors themselves do not prevent perfect accuracy. These results are not in Appendix B.2 .

Question 3

Is there a practical way to take advantage of the proposed geometric insights to improve classification?

We see two practical, actionable items, which we are working on and outline in the Discussion. The first is to penalize these correlations during training as a regularizer. This requires care, to avoid simply overfitting (placing each class in its own dimensions) at the expense of the real semantic structure of the data. The second, more appealing in our view, is distillation: a large teacher network whose geometry is known to generalize defines target values for these correlations, so that the student learns the teacher's semantic structure beyond its logits, without aligning the whole representation through CKA, which can be a difficult training objective. As Section 6 shows, in the DINOv3 family, whose smaller models are distilled from ViT-7B, the geometric measures of the students are not aligned with those of the teacher, and the theory relates these differences to their accuracy; matching them could help close the gap. This is still a hypothesis that we plan to test soon, but it is directly actionable.

We thank the reviewer again for the constructive questions, which led to the new important SVM analyses, and we are happy to answer any further questions.

For your convenience, the new results on the SVM readout that you asked about are collected, with their captions, in a single PDF: https://anonymous.4open.science/api/repo/sparse-prototype-code-8B54/file/revision_figures/reviewer_3/reviewer_3.pdf. It contains Table 18 and Figure 15 (Appendix J.1), where the theory predicts the held-out per-class accuracy of the SVM, and Table 5 (Appendix B.2), with the class-specific rotation applied to a trained SVM.

We would be grateful to hear from you soon to know whether these results answer your questions, and we remain available to discuss any remaining concerns.

Reviewer_MEodOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

This paper investigates the underlying geometric properties of neural representations in deep learning classification and prediction models. The authors propose the existence of a “universal geometry”, a sparse, centroid-aligned structure that consistently captures intra-class variability across diverse model architectures, training regimes, and data modalities. Leveraging this key empirical insight, the paper formulates a mean-field theory capable of accurately predicting both per-class performance variations and overall generalization trends as models scale in capacity and data. Through rigorous theoretical analysis and extensive experiments, the work provides strong evidence for a sparse prototype code governing high-dimensional representation space, offering deep insights into representation learning mechanisms.

Critical strengths
  1. Addresses a fundamental question in mechanistic interpretability by modeling the high-dimensional geometric structure of neural representations.

  2. Introduces a novel approach that uses intra-class variability to uncover shared representation geometry across different models and tasks.

  3. Provides valuable theoretical insights explaining per-class accuracy variations and scaling-driven generalization improvements.

  4. Includes comprehensive empirical validation covering major neural architectures (DinoV3, Qwen3, Roberta, Llama) and data modalities.

Critical weaknesses
  1. The manuscript requires substantial revision to improve readability and clarity; the methodology section, in particular, is currently difficult to follow due to dense notation and imprecise definitions of key experimental setups.

  2. The theoretical motivations lack clear intuitive framing and could be far better integrated with the paper's core empirical research questions and hypotheses.

  3. Experimental results would benefit from a more logical structure and narrative arc to improve presentation, flow, and contextualization relative to prior work.

  4. Figure labels, text descriptions, legends, and visual annotations are currently hard to read and require enhancement for clarity and visual accessibility.

Questions for authors
  1. How is this work related to existing sparse coding structures in neural representation analysis literature?

  2. What is the definition of 'rival classes' in your formulation? In the current manuscript, it is not defined at all. Please clarify this in the methodology section.

  3. Can these classification-based sparse prototype code findings generalize to tasks such as multi-label classification and regression?

6 comments

(1/2)

We thank the reviewer for recognizing the broader scope of our work in relation to mechanistic interpretability, and the breadth of the experiments.

The manuscript requires substantial revision to improve readability and clarity; the methodology section, in particular, is currently difficult to follow due to dense notation and imprecise definitions of key experimental setups.

We are sorry that the manuscript was difficult to read. We revised the presentation in the new version and hope that it is now clearer; we will gladly fix any further concrete point that interferes with the flow of reading. The notation, with the meaning and definition of every symbol, is summarized in Table 2 of the revised manuscript, to help follow it along the paper. The full experimental setup (datasets, models and the extraction of the representations) is described in Appendix C. In addition, we uploaded our source code, as a zip file in the supplementary material and as an anonymous GitHub repository, https://anonymous.4open.science/r/sparse-prototype-code-8B54/. The reviewer can load representations and labels of their choice and run the full analysis, which reproduces the main results of the paper.

The theoretical motivations lack clear intuitive framing and could be far better integrated with the paper's core empirical research questions and hypotheses.

For clarity, we summarize the picture in intuitive terms. Mechanistic interpretability extracts sparse, interpretable features from network activations, but it does not explain quantitatively how the geometry of a representation determines how well it supports a task. Our results identify this geometry for classification. Each class is represented by its center (its centroid), and an example is assigned to the class whose center it is most aligned with. The examples of a class do not spread randomly around their center: the spread is organized along a few directions, toward the class's own center and toward the centers of a few competing classes (its rivals). An example is misclassified when it deviates toward a rival's center more than toward its own. The theory turns this picture into a prediction: the accuracy of each class follows from a few measured quantities, namely the size of the spread, the fraction of it that points toward its own center and toward each rival, and the overlaps between the centers. With these quantities, it predicts the per-class accuracy across 36 networks in vision, language, and audio.

Experimental results would benefit from a more logical structure and narrative arc to improve presentation, flow, and contextualization relative to prior work.

The experimental results follow this logic. We start from the simplest assumption, that the spread within each class is random (the isotropic model, Section 3). Its predictions clearly do not match the data (Section 4): it predicts nearly perfect accuracy over the whole range of spreads that real classes have, while in real representations the accuracy varies widely across classes within this range. Simple manipulations of the data then reveal what the random model misses. When we keep everything about each class fixed but randomly re-orient its spread around the class center, all the errors disappear, so the errors come from the directions in which the classes spread, toward the centroids of competing classes. The model of Section 5 is built on these observations, and we measure the two properties it needs: each class competes with only a few other classes (Fig. 3a), and large deviations are more frequent than for Gaussian variables (Fig. 3b), which the theory absorbs in a single correction factor per model. The resulting theory predicts the accuracy of each class across 36 networks (Fig. 4 and Appendix I), and Section 6 uses it to compare models of different sizes.

(2/2)

Figure labels, text descriptions, legends, and visual annotations are currently hard to read and require enhancement for clarity and visual accessibility.

Following the reviewer's advice, we improved the readability of the figures in the revised version: Figs. 1 and 4 were redrawn with larger text, Figs. 2 and 3 were enlarged, and all the captions now sit below their figures.

How is this work related to existing sparse coding structures in neural representation analysis literature?

Sparse coding, in its modern form in mechanistic interpretability (sparse autoencoders and dictionary learning; Makhzani & Frey, 2013; Elhage et al., 2022; Bricken et al., 2023; Huben et al., 2024; Templeton et al., 2026; Gao et al., 2025), learns a dictionary of directions using an autoencoder network, such that each input activates only a few elements of the dictionary at a time, and shows that these directions often carry semantically meaningful concepts. Our results identify a complementary sparse structure in a supervised dictionary, the class centroids: for each class, the classification-relevant part of the within-class variability lies along only a few dictionary elements, the true centroid and a small set of rival centroids, and these rivals are enough to reproduce the full accuracy of the model. We also show that often these rivals are semantically related to the true class (Table 1). This is the sense of the "sparse prototype code" in the title. Unlike the directions of sparse autoencoders, which are found by reconstruction, the functional role of these directions is directly tied to classification, which is what allows the theory to predict the accuracy from them.

What is the definition of 'rival classes' in your formulation? In the current manuscript, it is not defined at all. Please clarify this in the methodology section.

The rivals of a class are the other classes it competes with. Definition 5.1 makes this precise: the rival set of a class consists of the few classes toward whose centroids its examples spread the most (the K classes with the largest variance fraction, Table 2). In the revision, we followed the reviewer's advice and defined the term at its first use, already in Section 2. Appendix F compares several ways of choosing the rivals, and shows that our choice is preferable in terms of both simplicity and performance.

Can these classification-based sparse prototype code findings generalize to tasks such as multi-label classification and regression?

The framework extends to multi-label data, where each object carries a set of labels, once the task of the multi-label classification is specified. Two natural tasks both can be matched to the single-label setting of the paper.

Recovering one of the many labels. The prediction is correct if the top-scoring logit is one of the object's labels. Each label defines a manifold made of all the objects that carry it, similar to the paper. Since only one label can have the top score, for an image labeled {for example, dog, frisbee} the probabilities that "dog" wins and that "frisbee" wins add, and the overall accuracy is the sum of the theory's per-label accuracies, weighted by the labels' frequencies.

Recovering the full label set. The prediction is correct only if all the labels the model predicts are correct. Then, each label combination that occurs in the data defines its own manifold, so {dog}, {frisbee} and {dog, frisbee} are three classes. This is again a single-label problem, and the theory applies without modification, provided each combination has enough examples.

Regression is a harder generalization, because the method is built for discrete categories that can be clustered, which covers many realistic settings, including language models. It is, however, an interesting direction, and we note it as future work.

We are grateful for this feedback, which improved the presentation of the paper, and we remain available for any further questions.

For your convenience, the result of the revision related to your question on sparse coding, Table 1 with the semantically related rivals (including the examples added in the revision), is available with its caption at https://anonymous.4open.science/api/repo/sparse-prototype-code-8B54/file/revision_figures/reviewer_2/reviewer_2.pdf. The redrawn figures, with larger text and the captions below them, are in the revised paper.

We would appreciate hearing from you soon whether our responses address your concerns, and we would be glad to clarify anything that remains unclear.

Reviewer_WoKuOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies how the geometry of neural representations determines multiclass classification accuracy, using a prototype classifier on frozen representations from pretrained models across vision, audio, and language. The authors find that within-class variability is not isotropic but aligned with the true-class centroid and a small set of rival centroids. Geometric manipulations suggest that this alignment, rather than the class radius or the arrangement of centroids, is the main source of classification errors. Based on this structure, they derive an analytical mean-field theory that predicts per-class accuracy from a few rival directions, with a single global rescaling that compensates for heavy-tailed statistics. They further use the theory to analyze how representational geometry changes with model scale, showing that accuracy gains are not explained by reduced within-class variability.

Critical strengths

Identifies a geometric structure of class representations shared across models and modalities. The paper shows that within-class variability in learned representations is not random. Instead, it is aligned with the true-class centroid and a small set of rival centroids, which are often semantically related. This structure appears consistently across architectures, training objectives, and modalities. Compared with prior pairwise analyses, the paper extends this view to the multiclass setting and shows that competition is concentrated on only a few rivals.

Provides an analytical theory that both predicts accuracy and serves as an interpretive tool. Building on the identified structure, the paper derives a simple theory that links representational geometry directly to per-class accuracy. This addresses the realistic regime in which substantial within-class variability remains, a regime only partially covered by existing geometric frameworks. The theory is also useful beyond prediction: applied across model scales, it shows that accuracy gains are not explained by reduced within-class variability. Instead, they accompany stronger alignment with the true-class centroid and weaker correlation with the strongest rivals.

Critical weaknesses

The causal interpretation of the geometric manipulations is stronger than the evidence supports. The paper states that centroid-aligned correlations are precisely the geometric source of classification errors and presents the δ-rotation as causal evidence for this claim. However, for a prototype classifier, the scores depend only on the residual components along centroid directions and the overlaps between centroids, so errors can only arise from these components by definition. Moreover, randomly rotating residuals in high dimensions removes these components for any representation, so near-perfect accuracy after the δ-rotation is expected regardless of the data. Similarly, the full-rotation experiment is applied only after the δ-rotation, so the minor role of centroid overlaps is shown only when residuals are no longer aligned with any centroid and at radii far beyond the empirical range. The empirical findings themselves remain valuable: residuals are aligned with a few centroid directions far more strongly than expected by chance. However, the claims about the underlying mechanism should be stated more carefully.

(minor) Formatting. Several elements do not follow the ICLR 2027 formatting instructions. Figures 1, 2, and 4 have no spacing between the caption and the following text, and the heading of Section 6 directly follows the caption of Figure 4. The caption of Figure 3 is placed beside the figure rather than below it. The text in Figures 1 and 4 is also difficult to read at print size. Separately, Equation 3 overlaps with the preceding line. I encourage the authors to fix these issues in the revision.

Questions for authors

On the relative roles of centroid overlaps and rival-aligned variability. The conclusion that centroid–centroid correlations play only a minor role is based on the full-rotation experiment. Could the authors compare the two factors in the empirical regime, for example, by using Eq. 6 to remove either the centroid overlaps (g_μ → 0) or the rival-aligned variability (σ_μ → 0) and comparing the resulting changes in accuracy?

On a control with untrained representations. All analyses are performed on trained models, so it is unclear whether the identified structure is a product of learning or a generic property of class means and prototype classification. Could the authors repeat the main analyses on a randomly initialized network with the same architecture? Specifically, do the centroid alignment, the sparsity of rivals, and the accuracy of the theory also appear there?

(minor) On the meaning of "sparse prototype code." The term in the title is not defined in the main text, and it seems to mean that each class is described by only a few centroid directions. However, "sparse code" usually refers to a representation in which each individual input activates only a few components. Could the authors clarify which notion of sparsity the title refers to?

4 comments

(1/2)

We thank the reviewer for recognizing the importance of the analytical theory and the universality of the claim. The reviewer raises important issues, which we address below.

1. The role of the centroid-centroid correlations

Similarly, the full-rotation experiment is applied only after the δ-rotation, so the minor role of centroid overlaps is shown only when residuals are no longer aligned with any centroid and at radii far beyond the empirical range. [...] Could the authors compare the two factors in the empirical regime, for example, by using Eq. 6 to remove either the centroid overlaps (gμg_\mu → 0) or the rival-aligned variability (σμ\sigma_\mu → 0) and comparing the resulting changes in accuracy?

We are grateful to the reviewer for raising the issue of the centroid-centroid correlations. We agree that "minor role" is not nuanced enough for this effect, and we changed the phrasing in Section 4: the centroid-centroid correlations are now described as a correction on top of the main structure captured by the residual-centroid correlations. Unfortunately, a direct geometric manipulation that targets the centroid correlations while keeping everything else intact is not possible. A rotation around the origin without the additional δ-rotation breaks the centroid-centroid correlations, but it also breaks all the other residual-centroid correlations; only the correlations between the residuals and the true centroid stay intact. However, we have several less direct lines of evidence for the smaller role of the centroid correlations.

The first line of evidence follows the reviewer's excellent suggestion to use the theory directly. Removing the centroid correlations, i.e. setting the overlaps gμ=c^k⋅c^μg_\mu = \hat{c}_k \cdot \hat{c}_\mu between the true and rival unit centroids to zero, does affect the accuracy of the model: in DINOv3 ViT-7B the RMSE of the predicted per-class accuracy rises from 0.043 to 0.081. This increase is somewhat tamed when the representations are first whitened, which decreases the centroid correlations further: the RMSE then rises from 0.035 to 0.061. Removing the residual-centroid correlations, however, breaks the model completely. A model in which the residuals are random but the centroids remain correlated can be solved with techniques similar to those of the model in Section 3. Let the P centroids be fixed, with their measured overlaps gμg_\mu, and let the residuals be isotropic with radius R. The accuracy of this model is

Acc(R)=Ez∼N(0,1)[∏μ≠kΦ(1−gμ1+gμ(z+NR))]\mathrm{Acc}(R)=\mathbb{E}_{z\sim\mathcal{N}(0,1)}\left[\prod_{\mu\neq k}\Phi\left(\sqrt{\frac{1-g_\mu}{1+g_\mu}}\left(z+\frac{\sqrt{N}}{R}\right)\right)\right]

This model, however, has two problems. First, as in the uncorrelated model, the accuracy declines only at R of order N\sqrt{N}, which is far from the behavior observed in the data. The only exception would be extremely correlated centroids (1−gμ1-g_\mu of order 1/N), a regime that is likewise not observed in the data. Second, we tested on DINOv3 ViT-7B whether this formula could predict the accuracy when λ is used to renormalize R, as in the model presented in the paper. The fit requires an enormous λ (9.8, compared to 1.32 in the paper's model with the same 20 rivals per class), which effectively compensates for the N\sqrt{N} scaling of R. It also yields an RMSE of 0.121, compared to 0.043 for the model in the paper, so its error is almost 3 times larger. This makes the centroid-proximity model significantly inferior to the centroid-aligned variability model in predicting real data.

Another line of evidence comes from the choice of rivals. If errors were driven by centroid clustering, then ranking a class's candidate rivals by centroid proximity would be the natural and best way to identify its true competitors. For each class we restrict the competition to its K top-ranked rivals and measure how large K must be for this K-way problem to reproduce the full P-way accuracy to within 1%. We compare ranking the rivals by centroid proximity and by their correlations with the residuals, as in the paper (median K over the backbones of each modality, Appendix F of the revision):

ModalityBackbonesPCentroid proximityResidual projections
Vision15100076 (7.6%)47 (4.7%)
Autoregressive LM14320102 (31.9%)89 (27.8%)
Masked LM432048 (15.0%)42 (13.1%)
Audio331060 (19.4%)39 (12.6%)

Proximity is consistently a worse method to select rivals than residual projections.

(2/2)

A more detailed discussion of the role of the centroid correlations is now in the revised appendix (Appendix F.4). The wording was indeed too strong and is changed in the revision, but we hope that the reviewer finds the scientific evidence convincing that the residual correlations are the dominant effect. We stress that this discussion concerns only the qualitative question of which effect dominates. Our analysis did not neglect the contribution of the centroid correlations: the theory includes them explicitly, through the measured overlaps gμg_\mu, so it requires no change.

2. A control with untrained representations

Could the authors repeat the main analyses on a randomly initialized network with the same architecture? Specifically, do the centroid alignment, the sparsity of rivals, and the accuracy of the theory also appear there?

We are very happy to adopt the suggested control, together with a similar one: the trained network with random labels. The results strengthen our claim that the findings reflect a true organization of trained representations. With both random weights and random labels, the accuracy is close to chance (exactly chance with random labels), the own-centroid alignment disappears, the competition is not sparse, and the theory no longer tracks the per-class accuracy. The table shows DINOv3 ViT-7B on ImageNet, with the same analysis as in the paper:

TrainedRandom weightsRandom labels
Prototype accuracy (chance 0.1%)85.5%1.9%0.1%
Mean class radius R1.507.4235.8
Classes whose own centroid is among their 3 strongest directions99.4%2.4%1.0%
Rivals that are the strongest competitor for 90% of the examples14177540
Rivals needed to recover the accuracy to within 1%37270n/a

Neither control is sparse: the strongest competitor of 90% of the examples is spread over 177 rivals with random weights and 540 with random labels, against 14 for the trained network, and recovering the accuracy to within 1% requires 270 rivals with random weights, against 37. With random weights the accuracy is close to chance for most classes (median 0.2%); the mean of 1.9% comes from a few classes that are distinguishable by low-level features such as color and texture, for example jack-o'-lantern, scuba diver, jellyfish and rapeseed (33-38%). With random labels the accuracy is at chance, so a 1% tolerance is ten times the accuracy itself and the count of rivals is not meaningful. The randomly initialized Qwen3-32B gives the same picture (accuracy 8.4% against 56.6%, and 115 rivals that are the strongest competitor for 90% of the examples, against 36). The accuracy, radius and sparsity measures of both controls are now reported in the revised appendix (Appendix B.3).

3. The meaning of "sparse prototype code"

Could the authors clarify which notion of sparsity the title refers to?

The term is an analogy to the sparse coding literature. In standard sparse coding, each input vector is reconstructed from a small number of active components. Here, we reconstruct the accuracy of a class manifold from a small number of centroid directions. In DINOv3 ViT-7B, for example, the true centroid and 37 rival centroids per class, out of 1000 classes, suffice to recover the accuracy to within 1%. The introduction now includes this clarification.

4. The causal interpretation of the δ-rotation

However, for a prototype classifier, the scores depend only on the residual components along centroid directions and the overlaps between centroids, so errors can only arise from these components by definition.

This is a fair point, and we softened the causal wording throughout the paper. The δ-rotation is best seen as a diagnostic. It keeps the radii and the centroid overlaps at their empirical values, so it shows that at the observed radii these alone produce essentially no errors. The errors therefore come from the alignment of the residuals with a few centroid directions, which, as the reviewer notes, is far stronger than chance. Near-perfect accuracy after the δ-rotation is also not guaranteed for every representation. In the random-labels control above, whose radii are 24 times larger (mean R of 35.8, against 1.5), it remains essentially at chance (0.2%).

Formatting. Thanks to the reviewer's careful reading, all the formatting issues are fixed in the revision: the spacing around the captions of Figs. 1, 2 and 4, the heading of Section 6, the caption of Fig. 3 (now below the figure), the text size in Figs. 1 and 4, and the overlap in Eq. 3.

We thank the reviewer again for the insightful comments, which improved the paper, and we would be glad to discuss any remaining concerns.

For your convenience, the results of the revision related to your questions are collected, with their captions, in a single PDF: https://anonymous.4open.science/api/repo/sparse-prototype-code-8B54/file/revision_figures/reviewer_1/reviewer_1.pdf. It contains Table 12 (Appendix F.4), which compares the roles of the centroid-centroid and the residual-centroid correlations in the theory, and Table 7 (Appendix B.3), with the random-weights and random-labels controls you suggested.

We would be grateful to hear from you soon to know whether our responses address your concerns, and we would be happy to clarify anything that remains open.

Reviewer_WoKuOpenReview ↗

Official comment by Reviewer WoKu

Thanks for your response. My overall assessment stays the same.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.