- Rating
- 4(+2)
- Confidence
- 2
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 3(+1)
Read review
Paper summary
The paper exposes geometric structures in trained representations that are relevant to classification and can predict classification accuracy. Specifically, residuals (representation minus centroid) are largely aligned with centroid directions. The most prominent centroids correspond to the true class as well as a few rival classes that are semantically related. This property of the residuals is highly pronounced as a random rotation about the true centroid axis destroys confusion and yields 99.999% classification accuracy. This is tested in the paper across training tasks, modalities, models, and model sizes. The paper provides solid math insight as it starts with a naive isotropic model and its properties, and then shows how a real model significantly differs in behavior step by step, revealing clear properties about the real model and its representations. The formulated theory according to grounded observations can predict the classification accuracy given the representations from the geometric perspective alone. No source code is released.
Critical strengths
- The writing is incredibly clear. The paper masterfully organizes various related nontrivial concepts into an easy-to-follow path.
- Proposing the isotropic model to compare with real models is an elegant approach. All but one of the perturbations in Section 4 and their interpretations are sound. The resulting observations are salient, insightful, and general.
Critical weaknesses
- Missing source code! The perspectives and experiment results strongly imply a sound research process but the source code is not released. Source code is important for this kind of paper, so not releasing code significantly undermines its contribution. The submission would otherwise warrant my Strong Accept, assuming a sound source code is released, and the reviewers' questions are addressed.
Questions for authors
- It will be very interesting to see the distribution of residuals' projection onto the true centroid, especially their sign. Are most positive? Fig 2b implies that most are positive. This would mean the centroid of any class tends to undershoot the cluster of correctly classified samples. This would be intuitive but there has not been any documentation of this phenomenon.
- More padding needed below Figures.
- Should "class variability" be "within-class variability"? For outsiders, "class variability" may look like the variance in the class label.
- L075 "The centroid-aligned correlations are therefore precisely the geometric source of classification errors." needs adjustment. The centroid-aligned correlations, which are the projections of residuals onto directions of other class centroids, are by definition the cause of misclassification. Even unaligned residuals follow this description. The authors' results showed that residuals are mostly aligned. Therefore the authors probably want to say "The geometric source of misclassification is not uniform divergence from the true centroid, but instead, targeted deviations towards rival class centroids".
- L079 "see Fig. 1a, Table 2" should be "see Fig. 1a and Table 2".
- L102 should be "P centroids using the ``Leave one out'' method (Lachenbruch & Mickey, 1968)." Parenthesis would imply the previously described method was the leave-one-out method.
- Either move R = R_k before eq 2, or change eq 2 to use R_k instead of R and use R in the future. Both options are good. Same for \sigma.
- The same operation is sometimes written as and sometimes . Select one and commit. (Recommendation: use the longer, standard form, and remove the "in addition" on L157.) Also fix definition 5.2, which is even more problematic since the two s are invisibly different.
- Most of the paper uses k for true label and \mu for rival label. Consider changing \mu in Def 2.1 to k for consistency. Then \mu does not need another definition and is fine to show up in (L152) as the first appearance since c hat has been defined to take a class subscript.
- Table 1. If you still need to save space, consider removing the "Notation" header.
- L232 Consider "Fig.2b blue line". This would contrast L257 "Fig.2b".
- Fig 2b: What are the two vertical lines?
- "Breaking residual–centroid correlations recovers the isotropic model." Indeed striking. Do you have a hypothesis as to why? What happened to the 39.2% points correctly classified before delta-rotation at large R*?
- "Centroid-centroid correlations play a minor role" cannot be concluded. Since the first-stage delta-rotation already modified the real models' behavior into the isotropic model's behavior, further intervening in centroid-centroid correlations reveals nothing about the real models. Please point out my confusion if I'm wrong.
- Which part in Sec.4 illustrated the sparseness of relevant centroids? This is a dependency of L269, and then more importantly eq 4.
- Consider adding visualization of , which is supposedly sparse. Maybe a bar plot. Plot the isotropic model as baseline.
- Eq 4: What is bold s?
- L292 may benefit from calling e some kind of residual since e may look unfixed and confusing.
- L072 is near the beginning of Introduction and should prefer "residual" over "variability" for outsider friendliness. Maybe "Its defining property is that within-class residuals are strongly aligned with the true-class centroid and the centroids of a few competing classes." Using the word "variability" before equations 2.x is risky because the word by itself is a quantity and a property at the same time.
- L077: "... small set of rivals, which is sufficient to reproduce the predicted per-class accuracy" is linguistically problematic for the unclear "which" and the overcomplicated "reproduce prediction". Maybe "... small set of rivals. Using only a few rivals, our theory predicts per-class classification accuracy from the geometric perspective."
- L061: "Our analytical lens is prototype-based classification." might be unnecessarily weak and might lead first time readers (including me) to think the experiments are on prototypical classifiers only. Maybe spell out "We use the lens of prototype-based metric(?) learning to study classifiers and language models." Similarly, L100 "We use" -> "We consider" may help.
- L137: "In autoregressive models this is the embedding of the token preceding the word." This actually refers to the word embedding, which is much less exciting and not what you mean. Change it to "activations" or "hidden states".
- Fig 2 caption: Consider "Per-class accuracy vs. per-class empirical radius R". During my first read I didn't realize the Rs were natural and instead kept asking "What's the intervention that changed R, that is not R*?"
- Fig 2: Is the "null uncorrelated model" the isotropic model? Update legend?
7 comments
(2/3)
"Breaking residual-centroid correlations recovers the isotropic model." Indeed striking. Do you have a hypothesis as to why?
Our hypothesis is that once the residuals are isotropic, the centroid correlations barely matter at any radius, because they enter the accuracy only through factors that are close to 1 for almost all rivals. This can be checked directly, since a model with isotropic residuals and the measured, correlated centroids can be solved with the same techniques as the model in Section 3 (Appendix F.4). Let the P centroids be fixed, with their measured overlaps , and let the residuals be isotropic with radius R (Eq. 6 with and , since isotropic projections on two centroids are correlated exactly by their overlap). The accuracy of this model is
where z is the standardized projection of the residual on the true centroid and is the standard normal distribution function. It differs from the uncorrelated isotropic model only through the factors . In DINOv3 ViT-7B these factors are close to 1 for almost all rivals: over all pairs of classes, the overlaps have a median of and a 99th percentile of 0.10, and the median class has only 7 of its 999 rivals with . As a result, along the whole range of radii in Fig. 2b, this model differs from the uncorrelated isotropic model by at most 1.7 percentage points in accuracy (at ). This is why the -rotated data follow the isotropic model, and why breaking the centroid correlations as well (the full rotation) changes their accuracy very little. At the empirical radii, is large, and both models are essentially perfect.
This is the corrected statement in Section 4.1: the centroid-centroid correlations act as a quantitative correction that shifts the decision boundaries, whereas the residual-centroid correlations change the scaling of the problem. They replace the isotropic projections of order by the measured (0.12 for the strongest rival in DINOv3 ViT-7B, against ), which moves the decline of the accuracy from R of order to R of order 1.
What happened to the 39.2% points correctly classified before delta-rotation at large ?
As for the 39.2% of the examples that are correct at large in the original data, the random rotations indeed remove their large positive projections on the true centroid, which keep them correctly classified there. However, at the empirical radii they stay correct, like the other examples, because the true-centroid term is larger than the overall random variability. Since they now rely only on this term, at very large these examples, like all the rest, are classified at chance, and the accuracy decays to 1/P, unlike in the original geometry.
(3/4)
"Centroid-centroid correlations play a minor role" cannot be concluded. Since the first-stage delta-rotation already modified the real models' behavior into the isotropic model's behavior, further intervening in centroid-centroid correlations reveals nothing about the real models.
The reviewer is right that this statement is too strong, and the same point was raised by other reviewers; we revised it accordingly (Section 4.1). Unfortunately, a direct model-free geometric manipulation that breaks only the centroid correlations is not possible, since a rotation around the origin also breaks all the residual-centroid correlations except those with the true centroid. We therefore compared the two factors in the empirical regime indirectly in three other ways.
First, the choice of rivals is the most direct empirical test. For each class, we restrict the competition to its K top-ranked rivals and find the K at which this K-way problem reproduces the full P-way accuracy to within 1%. If the errors were driven by the centroid-centroid correlations, ranking the rivals by centroid proximity () would identify the true competitors with the fewest rivals. Instead, ranking them by the residual-centroid correlations (), as in the paper, needs fewer rivals in every modality, as already noted in lines 342-343 of the original manuscript ("In particular, it outperforms selection based on centroid proximity, providing additional evidence that centroid correlations [...]"). The table gives the median K over the backbones of each modality, with the fraction of P in parentheses (Table 10 of the revision):
| Modality | Backbones | P | Centroid proximity () | Residual projections () |
|---|---|---|---|---|
| Vision | 15 | 1000 | 76 (7.6%) | 47 (4.7%) |
| Autoregressive LM | 14 | 320 | 102 (31.9%) | 89 (27.8%) |
| Masked LM | 4 | 320 | 48 (15.0%) | 42 (13.1%) |
| Audio | 3 | 310 | 60 (19.4%) | 39 (12.6%) |
The true competitors of a class are thus the classes along whose centroids its examples vary the most, rather than the classes whose centroids are closest.
Second, the role of each type of correlation can be tested in the theory (Eq. 6). Setting the overlaps to zero raises the RMSE of the predicted per-class accuracy in DINOv3 ViT-7B from 0.043 to 0.081, so the centroid correlations are not negligible and are needed for an accurate quantitative prediction. Removing the residual-centroid correlations instead, as in the model with isotropic residuals above, breaks the theory completely: it requires against 1.32 for the full theory, which effectively compensates for the scaling of R, and still leaves an RMSE of 0.121 (Table 12).
Third, we intervened on the classifier directly. Removing the centroid-centroid terms from the rival logits, with the residuals unchanged, raises the accuracy from 85.5% to 91.5% in DINOv3 ViT-7B, while the -rotation, which keeps the centroid correlations, raises it to 100%.
We stress that this discussion concerns only the qualitative question of which effect dominates. Our analysis did not neglect the contribution of the centroid correlations: the theory includes them explicitly, through the measured overlaps , so it requires no change. The full discussion is now a new appendix section (Appendix F.4).
(1/3)
We thank the reviewer for their detailed review, for recognizing the value of our work, and for the kind words about the writing. The reviewer is absolutely right that source code is essential for this work. The code is now uploaded both to the submission, as a zip file in the supplementary material, and to this anonymous GitHub repository: https://anonymous.4open.science/r/sparse-prototype-code-8B54/. The code takes a representation matrix and a label vector, as numpy arrays or PyTorch tensors, and an optional list of class names, and produces the results of the paper for that representation: Fig. 1 (visualization), Fig. 2 (accuracy and geometric manipulations), Fig. 3 (sparsity and the statistics of s), Fig. 4 (theory and experiment comparison), Table 1 (top rivals), its row in Table 3 (geometric measures across scales), in Table 4 in the appendix (additional geometric measures) and in the theory table of Appendix I, the rival-selection tables of Appendix F, and the theory error against the number of rivals K (Appendix F.3). In addition, a standalone script simulates the synthetic isotropic model and generates Fig. 6a (accuracy against theory). The main logic of the code is in
geometry.py,theory.py,manipulations.pyandrivals.py, which we hope are sufficiently clear and well documented; we will gladly answer any questions about them. The code follows the notation of the paper as closely as possible.Regarding the questions, we answer the main scientific ones below; the points about notation and corrections are already incorporated in the revision, and we are grateful for them.
By definition, the residual projections always have zero mean. However, the reviewer's intuition is correct: their distribution is skewed, such that its peak is at a positive value (sk≈0.55 in DINOv3 ViT-7B, in units of its standard deviation; 59% of all examples project positively), balanced by a long negative tail (skewness −0.98). It is also true that for correctly classified examples the distribution has a positive mean, sk=0.22 in DINOv3 ViT-7B, and a negative mean for misclassified examples, sk=−1.34. This means that even though the centroid is, by construction, the "center of mass", in well-performing networks such as DINOv3 a large mass overshoots the centroid with positive projections, while a smaller mass of negative projections has a longer tail that balances them. The misclassified examples sit almost entirely in this tail: 90% of them have a negative projection, and 78% of the examples with sk<−2 are misclassified, against 2.4% of the examples with sk>0 (recall that the logit of the true class also contains the centroid term +1, against $g_\mu $ for a rival, so an example is misclassified only when its projection on a rival exceeds its projection on the true centroid by more than 1−gμ). The same holds in Qwen3-32B, where 74% of the correct and 30% of the misclassified examples project positively. This analysis was added to Appendix G (new Fig. 9), and is referred to in the text in Section 4.
Correct. We fixed it in every place where this phrase was not accurate.
We gladly adopt the more accurate description suggested by the reviewer.
The two vertical lines mark the empirical radius range, R between 0.4 and 4.3, which is R/N between 0.007 and 0.067 on the axis of Fig. 2b. The point is to show that in this range the original geometry behaves similarly to the empirical accuracy, decreasing from 100% to about 60%, while the rotated data keep perfect accuracy. We added a clarification to the caption, thank you.