acceptodds
Under review as a conference paper at ICLR 2027

Detecting overfitting in Neural Networks during long-horizon grokking using Random Matrix Theory

Abstract

Training neural networks without overfitting is difficult; detecting that overfitting is difficult as well. We present a data-free Random Matrix Theory method for detecting the onset of overfitting from the trained weights alone. For each layer, we shuffle the weight matrix element-wise, , fit the shuffled matrix's empirical spectral distribution with a Marchenko–Pastur law, and identify large outliers beyond the finite-size right edge. We call these outliers Correlation Traps. In long-horizon grokking, traps form and grow in number and scale during anti-grokking, a post-generalization phase in which training accuracy remains high while test accuracy declines. Across an MNIST MLP, a modular-addition transformer, and a GPT2-style composition model, Correlation Traps distinguish anti-grokking from pre-grokking despite similar train and test behavior. Following Li and Sonthalia's spiked-regression account, a separated spike may be benign or harmful depending on its alignment with the learned function. We estimate this alignment without task data by passing random inputs through the trained model and measuring the Jensen–Shannon divergence between output logit distributions before and after suppressing each detected mode. The resulting scores recover the behavioral ranking obtained with in-distribution inputs, while continued suppression of the detected modes improves generalization more than an equal-count, equal-update-energy largest-weight edit. We also find Correlation Traps in foundation-scale language models, where their effect on generalization remains to be tested. These results identify randomized weight spectra as a data-free signature of anti-grokking and a practical screen for potential harmful overfitting.

Primary area:
learning theory
Keywords:
Anti-grokking, Random Matrix Theory, Marchenko-Pastur, Grokking, overfitting, Correlation Traps, Data-Free Metrics, Generalization, Memorization

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_7s4ROpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

The authors consider three model/task combinations that are known to show grokking and train these models for long enough that the test accuracy decreases again (denoted as anti-grokking by the authors). The authors then define a metric, by counting the Marchenko-Pastur outliers of weight matrix spectra after shuffling their entries, and show that these outliers form essentially only in the anti-grokking phase. Feeding random data through the model and suppressing these spike directions, the authors identify spike directions relevant to the outputs of the models. Suppressing these directions continuously during training improves generalization in the anti-grokking phase compared to the unsuppressed baseline.

Critical strengths

In the examples presented by the authors, there is a strong agreement between formation of the "correlation traps" and the anti-grokking phase, which, if a convincing relation to overfitting in a more general setting was shown, would allow a detection of overfitting from weight matrices alone. The suppression analysis indicates that these spike directions in the shuffled weights may indeed be related to the loss in generalization performance in the anti-grokking phase.

Critical weaknesses
  1. The manuscript overall feels unfinished, poorly written, and it is very hard to follow. I list some examples that makes me come to this conclusion below:

a. Figures: All figures seem unfinished. The text is extremely small and the legends not legible (at any reasonable zoom level).

  • Fig. 1: The legend mentions a gray region, but in the figure it seems to be drawn in blue. The axis labels are tiny.

  • Fig. 2: There are no axis labels. The caption says λtrap=106.5\lambda_{\rm trap}=10^{6.5} but the outlier in panel b seems to be at λ≈6.3\lambda \approx 6.3. The panel titles contain Rmp\mathcal{R}_{mp} which seems to be undefined. The vertical, dashed, red line is not explained.

  • Fig. 3: The curves seem to be way smoother despite it is supposed to show the same data as in Fig. 1. The text claims the "test loss reaches a minimum at some point. This is the regime (green) we call anti-grokking", but the test loss in panel a is minimal in the middle of the yellow (grokking) regime.

  • Fig. 4: nn is undefined. It is not clear what the points represent or how they are obtained. The text says: "Figure 4 validates random-probe JSD against the task-based consequence of the same intervention. [...] Within each model family, the ranking is strong. The results support JSD as a data-free prioritization score for behaviorally consequential traps", but I really do not see how this is supported by what is shown in the figure.

  • Fig. 5: The y-axis label is missing. It is described in the main text by the incomplete sentence: "The practical implication that one can detect layers for follow-up behavioral testing when only weights are available".

b. There are many dense sentences such as: "A Correlation Trap is a separated right-edge mode relative to an empirical MP/TW null fitted after entry-wise randomization of a layer weight matrix".

c. The appendices are not well motivated and integrated into the paper. For example the proofs of Appendix A are not clearly motivated in the paper. The paragraph starting on line 266 is unclear. It is hard to see how the proved theorem is related to the main text at all.

d. Table 2: The reported KS-test p-values are only either tiny or 1.0, which indicated wrong underlying statistics (if drawn according to the test distribution, the p-values should be uniform in [0,1] by definition).

e. The abbreviations JSD (I assume Jensen-Shannon divergence) is used all over the manuscript, but it is never defined, neither what it stands for nor how it is computed.

f. Lines 291-298 are completely ununderstandable if one is not an expert in the GrokkedTransformer of Wang et al. (2024).

g. On line 258 it is unclear what the authors mean by "fitted-MP distance".

h. Line 261 mentions some point or section "A" that is not mentioned before.

i. Line 303: The parameter αinit\alpha_{\rm init} is not defined or motivated and requires a reader to know Liu et al. (2023).

j. Line 362: It is unclear what "same Frobenius correction budget at the trap edit" means.

k. Line 385: It is not clear what "a matched Gaussian replacement for the mapped FC1 direction" means.

 

  1. The abstract makes overstated claims about overfitting detection from weights alone that are not convincingly demonstrated in the paper, which the authors also acknowledge in the limitations section, which is, however, rather hidden in appendix H. I was also irritated by the sentence "We do not claim that this archive alone is an end-to-end executable release" seeming to indicate that the code may not have been used to run the experiments and that the data may not be reproducible?

 

  1. References and the related work are weak. In the RMT paragraph of the related work all citations involve the same author, Charles H. Martin, despite there are many works on RMT of network weights in the literature (this is 20% of the only 25 references for this author only). Another example, in the footnote 1 on page 4, the citations do not establish that neural network bulk vectors follow Porter-Thomas statistics, but that the vectors of random matrices do.
Questions for authors
  1. How consistent are the results over different training runs? How are error bars computed in Figure 1 (both for accuracies and for spike outliers)? Not all curves seem to have error bars, or are they too small to be visible?
  2. In which sense does the modular addition task in Fig. 1b show grokking, both test and train seem to rise together?
  3. The authors discuss that the number of outliers is quite stable between reshuffling. How do the outlier compare to those of the unshuffled weights in the different training phases? In which phase do MP outliers form in the unshuffled weights?
  4. The text corresponding to Fig. 2 says it is for a "well behaved layer". How does a typical layer look like?
  5. The examples of global ℓ2\ell_2 norm in line 375 shows strong changes in mean and fluctuations between anti-grokking and the other regimes (where the norms are very similar), while mean trap count has error bars almost as large as the mean itself. Why do the authors conclude that the trap count is a better metric than the norm?
  6. In what sense are spike directions (i.e. the singular vectors) stable between different shuffles?
  7. Does Fig. 5 show the mean outlier count over different shuffles? How large are the fluctuations?
3 comments
AuthorsOpenReview ↗

Response to 7s4R pt 1 addressing critical weaknesses

We thank you for your time reviewing this paper

1. Presentation, Appendix A, KS p-values, and JSD

We have now added more detail to the figure labels to prevent ambiguity and will shortly upload the updated manuscript. The dense sentences resulted from trying to include this information within the page limit. We have revised these passages and moved material to the appendix to ensure clarity, but would also like to clarify several important points in the review.

Interpretation of KS p-values. We respectfully disagree with the inference that these values indicate “wrong underlying statistics.” Uniformity applies to correctly calibrated continuous p-values under a true null, not to an arbitrary collection of tests on separate phases. Table 2 compares selected training phases, including anti-grokking, where departure from the MP baseline is precisely the phenomenon being investigated. There is no requirement that these three selected values appear uniformly distributed since these are separate tests on separate phases/distributions , and their failure to do so clearly does not establish that the statistics are wrong.

Definition and interpretation of JSD. JSD is already explicitly identified as Jensen–Shannon divergence in Section 5, lines 393–394. Equation (7) defines our score as the expected divergence between the original and intervened models’ softmax output distributions; lines 396–398 specify the probes and temperature. Its meaning and use are therefore already described in the manuscript.

Regarding what the JSD figures support, the idea is to quantify the effect of each trap. One way is to pass data through the original model, remove the trap, and measure the change in predictions. Another way and what our JSD test does, is to pass Gaussian probes through the original and intervened models and measure the change in their softmax output distributions. The plots show that the effects measured using data inputs and Gaussian probes are highly correlated, suggesting that the JSD test can identify traps with greater influence on predictions.

Modular addition does show delayed generalization, which is characteristic of grokking. Training and test accuracy do not converge at the same time. Training accuracy reaches 100% around (7\times10^2) steps, whereas test accuracy reaches 100% around (10^4) steps : a roughly 14-fold difference in convergence time.

One important point here is that the axis is logarithmic. This can make the delay easy to miss visually unlike the linear scale shown by Nanda et al. etc , but is essential to show the long training period required for anti-grokking.

Motivation for Appendix A. Appendix A connects spectral structure to generalization in a tractable Gaussian regression setting. Theorem A.1 quantifies how spike strength and target alignment affect excess test risk despite exact training fit. Proposition A.5 and Lemma A.6 then motivate assessing individual traps through their effects on predictions: JSD measures output sensitivity, while labeled interventions determine whether performance improves or worsens. This is the connection to the main text, and we will make it more explicit in the paragraph beginning at line 266.

2. Checkpoint availability and reproducibility

That sentence refers specifically to the GPT-2 checkpoints used for the anti-grokking analysis. They were excluded from the ZIP file because of their size and have been uploaded to Hugging Face; the link was withheld to preserve anonymity. All experiments are reproducible from scratch, and the tool is fully open source and available for others to test.

3. Porter–Thomas statistics and references

Martin and Mahoney (2021, Section 3.3) explicitly discuss Porter–Thomas statistics for bulk eigenvectors and examine eigenvector localization in trained networks in Figure 19.

Additionally, Thamm et al. (2022, Figures 4 and 8) directly test singular-vector components of trained neural-network weight matrices against Porter–Thomas predictions, finding broad agreement in the random-like bulk and deviations primarily among vectors associated with large singular values. We are happy to add Thamm et al. and other relevant references suggested by the reviewer.

AuthorsOpenReview ↗

Response to 7s4R pt 2 addressing questions

1. Training runs, shuffles, and error bars

For the MLP in Figure 1(a), the accuracy curves summarize 10 independent training runs with different random seeds. The solid curves are the pointwise means, and the shaded regions are ±1 sample standard deviation across the 10 runs.

At each analyzed checkpoint for each run, every layer is independently shuffled 50 times. The blue curve is the mean trap count over these shuffles, and its faint band is ±1 sample standard deviation over the shuffles.

The main modular-addition and GPT-2 trajectories are individual long-horizon runs. They required nearly half a year of GPU training to reach anti-grokking (due to the fact that this occurs only on the long horizon) , so Figure 1 does not report training-run error bars for these panels.

We will revise the caption to state (n=10) for the MLP accuracy bands, (n=50) shuffles for the spectral bands, identify the bands as sample standard deviations, and explicitly mark the single-run panels. We will also clarify that Figure 3 shows the mean of two training runs ((n=2)). We have not included training-run error bars in Figure 3, since estimates of run-to-run variability based on only two runs are highly uncertain; we will state this limitation explicitly in the caption.

2. Grokking in Figure 1(b)

Figure 1(b) does show delayed generalization, which is characteristic of grokking. Training and test accuracy do not converge at the same time. Training accuracy reaches 100% around 7×1027 \times 10^2 steps, whereas test accuracy reaches 100% around 10410^4 steps: a roughly 14-fold difference in convergence time.

One important point here is that the axis is logarithmic, unlike in Nanda et al. This can make the delay easy to miss visually, but is essential to show the long training period required for anti-grokking.

3. Randomized versus original weight spectra

We thank the reviewer for raising this distinction. Our diagnostic deliberately operates on the entry-wise randomized weight spectra, rather than the spectra of the original trained weight matrices.

Martin and Mahoney (2019, 2021) show that trained weight matrices can develop strong correlations and therefore non-Marchenko–Pastur ESDs, including heavy-tailed ESD regimes ( as seen in the alpha metric in their paper) in which a well-defined separation between an MP bulk and isolated outliers is absent.

Consequently, directly applying an MP/TW criterion to unshuffled weights would not generally have the same statistical interpretation. Our approach instead constructs a conditional randomized reference by permuting weight entries and evaluating the resulting spectrum against a fitted MP/TW baseline. The purpose of randomization is therefore to disrupt the learned correlations associated with the original arrangement of the weights while preserving their empirical distribution.

4. Meaning of a “typical layer”

By a “typical layer,” we mean one whose distribution of weight entries is compatible with our randomized MP/TW baseline: after entry-wise shuffling, its covariance spectrum is well described by the fitted MP bulk, without persistent large outliers beyond the finite-size right-edge threshold. This is the behavior described as a “well-behaved layer” in Section 3.2. Shuffling preserves the distribution of the weight entries while removing their learned arrangement, allowing us to assess this compatibility directly. The Gaussian-like and Laplace-like cases discussed in the paper are both relevant to the MP law.

5. Comparison with the L2 norm and interpretation of the reported spread

Consider a thought experiment: scale the same initial weights using init=10 versus init=100. The initial L2 norm differs tenfold by definition, whereas the MP/TW outlier count remains unchanged under this rescaling because the eigenvalues and fitted threshold scale together. Thus, there is no scale-independent absolute L2 cutoff: the norm can be modulated simply by changing the initialization scale. Our detector measures spectral atypicality independently of this overall scale, because the eigenvalues and the fitted MP/TW threshold scale together, leaving the trap count unchanged under uniform rescaling. Consistently, our experiments with init=4,8 show zero traps during pre-grokking and grokking.

The reported (4.56 + /- 3.03) represents the mean and sample standard deviation across layers, not a standard error. The large spread reflects differences in layer characteristics and mean trap counts 7.00, 1.17, and 5.50 while all three layers exhibit nonzero mean counts during anti-grokking versus zero earlier.

Furthermore, unlike a single global norm, our method identifies individual traps whose influence on predictions can be characterized using JSD. Labeled interventions can then determine whether these effects are beneficial or harmful.

AuthorsOpenReview ↗

Response to 7s4R pt 2 addressing questions contd.

6. Stability of trap directions across shuffles .

We thank the reviewer for suggesting this experiment, as it led to very interesting results.

We tested 100 independent entry-wise shuffles at the final MNIST MLP checkpoint without weight decay. For each shuffle, we reconstructed the leading 20 detected traps separately and undid the permutation to return them to the original weight coordinates. Specifically, if PP denotes the entry permutation and σkukvk⊤\sigma_k u_k v_k^\top is a trap component of the shuffled matrix, its contribution in the original coordinates is Ak=P−1(σkukvk⊤)A_k = P^{-1}(\sigma_k u_k v_k^\top). We compared the subspaces spanned by these reconstructed matrices across shuffles, so the comparison does not depend on their ordering or the choice of basis within each subspace. We also compared their sums, ΔW=∑k=120Ak\Delta W = \sum_{k=1}^{20} A_k, using cosine similarity. This sum is exactly the change to the weights when all 20 traps are removed: the edited layer is W−ΔWW-\Delta W. Thus, we compared both the subspaces identified by different shuffles and the direction of the combined weight change produced by removing the detected traps. We repeated both comparisons using 20 bulk singular-vector pairs as a control.

Comparison after undoing the shuffle20 traps20 bulk singular-vector pairs
Backprojected subspace similarity0.09850.000506
Same subspace comparison, excluding shared alignment with the original weights0.09010.000179
Trap-removal matrix similarity0.6160.087
Same matrix comparison, excluding shared alignment with the original weights0.4050.031

Values are means over all 4,950 shuffle pairs, in the original weight coordinates. Subspace similarity is 1 for identical subspaces and 0 for orthogonal subspaces; matrix similarity is the cosine between the reconstructed matrices.

The backprojected trap subspaces overlap substantially more than the bulk control.

Most interestingly, we examined 31 WeightWatcher-detected traps at this checkpoint and measured each trap’s stability as its mean squared projection onto trap subspaces from the same 100 shuffles, after undoing the permutations and retaining alignment with the original weights. We found a strong positive correlation between trap stability and output JSD on 8,192 Gaussian probes: Spearman ρ=0.793\rho = 0.793 for the Gaussian replacement of the trap vector.

So, in this experiment, more stable directions seemed to be related to traps that had greater functional influence on the predictions under these interventions: a very intriguing result!

7. GPT-OSS shuffle count

This is the mean count over five shuffles, with fluctuations of around 7%. Because of GPT-OSS’s size and our limited compute, we could shuffle and compute the spectra only five times, compared with 50 for the other main spectral analyses. We will make this explicit.

Reviewer_UtipOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The manuscript identifies a post-grokking mode that the authors call anti-grokking: after a model has already generalized, continued optimization can degrade held-out performance while training accuracy remains near perfect. To diagnose this regime, the authors propose an entry-wise randomized spectral diagnostic based on the Marchenko–Pastur (MP) law and Tracy–Widom edge corrections, implemented through WeightWatcher. Separated outliers beyond the fitted MP edge are termed Correlation Traps. The paper further introduces a Jensen–Shannon divergence (JSD) score with random probes to rank the behavioral influence of detected traps, and supports the analysis with ablation and continued-training interventions. Experiments cover an MNIST MLP, modular addition, a GPT2-style composition task, and exploratory layer-wise profiles for GPT-OSS models.

Critical strengths

The main idea of the paper is quite interesting. It draws attention to long-horizon training behavior after grokking, which is under-explored relative to the memorization-to-generalization transition itself. It also proposes to utilize the spectral characteristics of the weight covariance matrix to infer whether the training is in the stage of overfitting. In addition, The paper empirically demonstrates the existence of specific spectral features in several typical network models, e.g. an MLP, a modular-addition transformer, and a GPT2-style composition task.

Critical weaknesses

The phenomenon is interesting. The diagnostic is potentially useful. However, the causal and mechanistic claims are not yet sufficiently established. In particular, the authors of the paper failed to provide any intuitive and analytical explanation on why the spectral characteristics can serve as a signature for the overfitting stage.

Questions for authors

This paper has potential influences in terms of theory, but unfortunately, its theoretical value has not yet been fully clarified.

According to my current understanding, the current state of the methodology in this paper has very limited value in terms of application, because the actual trainers of the model can directly use the test error to control overfitting. So, could the author provide a clearer explanation of the practical significance of the methods employed in the paper?

1 comment
AuthorsOpenReview ↗

Response to Utip: Practical significance

We thank the reviewer for the thoughtful and constructive review, and for recognizing both the interest of studying long-horizon behavior after grokking and the potential usefulness of our diagnostic. We appreciate the questions about its theoretical interpretation and practical significance, and will make these points clearer in the revised manuscript.

The practical value is precisely that downstream users must evaluate models they did not train. Hugging Face hosts over one million models, often with limited training-data transparency. Benchmark contamination is a well-documented problem (Deng et al., 2024; Li et al., 2024), and reported test scores alone may not reliably establish generalization.

Our method gives users an initial, data-free diagnostic directly from the weights. It identifies spectral warning signs, localizes specific layers and modes, and uses random-probe JSD to measure their influence on predictions. This provides actionable evidence for deciding where to focus expensive, task-specific evaluation.

Regarding the intuitive and analytical explanation, Appendix A shows how perfect training fit can coexist with errors that a large eigenvalue makes particularly costly on new inputs. In the Gaussian regression setting, coefficient errors orthogonal to the training inputs do not affect training predictions, but can still affect test predictions. A large covariance eigenvalue increases the contribution of errors along its eigenvector (a simple regression analogue of the trap vector) . Corollary A.2 identifies an explicit range where the corresponding sample eigenvalue separates from the MP bulk and test risk exceeds that of the setting without the covariance spike, despite exact training fit.

For the neural networks, our evidence goes beyond correlation: removing detected traps during continued training achieved 64.46% final test accuracy, versus 50.75% for ordinary continued training and 58.92% for matched largest-weight edits.

Reviewer_hcznOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This submission considers the phenomenon of "anti-grokking", namely grokking followed by overfitting (thus test accuracy belatedly increases but then decreases again) in three scenarios: MNIST classification using an MLP, modular arithmetic using a small transformer architecture and the GrokkedTransformer GPT2 model. The paper proposes a data-free method for detecting the point at which anti-grokking occurs. The method consists, in a nutshell, in identifying outliers in the ESD of permutations of the network's trained weight matrices. By sampling permutations of a weight matrix, the author(s) obtain samples of weight matrices that share all the statistical properties of the reference matrix (since they all have the same CDF), and this allows a statistically meaningful identification of outliers in the ESD of these matrices. These outliers - the part of the spectrum not captured by the MP distribution - are called correlation traps. The paper shows that these correlation traps start occurring at the onset of anti-grokking.
The paper then considers the impact of these correlation traps by ablation or replacement by noise. The author(s) measure (a) the influence of the traps via the JSD between pre- and post-intervention predictions and (b) the net contribution of the traps via the task effect delta.

Critical strengths
  • It was nice to review a paper that was written so clearly and with the evident objective of sharing and explaining findings to a human audience. This alone makes the paper stand out amongst everything else I've reviewed.
  • The methodology (sampling from permutations, ablation/random replacement, JSD) is sound and the experiments cover a suitably large set of architectures and scales.
  • The appendix is rich is additional details, including an analytically tractable toy example demonstrating that spikes (correlation trap proxies) can have a range of influences on learning error, from helping to hindering via non-interference.
  • The anti-grokking setup is a very nice and very rich scenario from a learning dynamics perspective.
Critical weaknesses
  • I'm generally sceptical of data-free methods, and this has not changed my mind. Have you considered what happens if the networks are initialised from a law that will exhibit "correlation traps" from the onset of training? For example, a power law with no variance or no kurtosis?
  • The connection with the heavy-tailed extensions of MP theory is a little unclear to me, in particular why do you adopt the MP+TW perspective on the ESD rather than Martin & Mahoney's moderately/very heavy-tailed regime where there is no bulk edge, but the entire ESD needs to be modelled with a power law? This seems like a potentially better fit for situations like Figure 2 (c) ... or maybe not ... but I'd like to know why you chose one option over the other.
  • I'm not sure sec 6 adds much. I can't tell from Figure 5 if the bulk has been removed and if we're only looking at the TW/outliers part of the spectra.
  • What about general overfitting? I.e. overfitting that may not follow grokking? It seems like your correlation traps might show up there too, and it feels like a very natural thing to consider. As nice as I find the anti-grokking scenario from an intellectual perspective, why do you restrict yourself only to that scenario when overfitting is a much more general concern?

Minor comments: l64: "(gray)", did you mean "(blue)"? l329: table 5 is only in the Appendix l432-l436: what experiment are you talking about here? l471-462: this sentence does not make grammatical sense

Questions for authors

My main question is: can correlation traps be used to identify overfitting beyond the anti-grokking special case? If yes, do the conclusions of the paper hold more generally for overfitting? If yes, why isn't this discussed, i.e. why the focus on anti-grokking?

2 comments
AuthorsOpenReview ↗

Response to hczn: Scope, initialization, and foundation-model relevance

We thank the reviewer for the thoughtful and encouraging review, particularly the comments on the clarity of the paper, the methodology, and the interest of the anti-grokking setting. The questions about initialization and applicability beyond grokking are important, and have motivated additional experiments and clarifications that we will include in the updated manuscript.

Why MP/TW? Prior empirical studies report trained weight-entry distributions well approximated by a Laplace distribution. Although broader than Gaussian, Laplace distributions have exponentially decaying tails and belong to the light-tailed setting underlying our MP baseline. This motivates our expectation of an MP-like randomized bulk under standard assumptions. The distinction is that the original trained matrix can have a heavy-tailed spectrum due to correlations even when its individual entries have a light-tailed distribution. We in particular study the randomized ESD as compared to the unrandomized ESD studied by Martin & Mahoney. The reason is that the thresholds may be easier to observe with MP , shuffling breaks the initial correlations present and any left over departure from MP indicates a sort of atypicality.

Foundation-model relevance. We believe the unusually large number of correlation traps in gpt-oss-20b and gpt-oss-120b suggests some form of overfitting to their training data. In our observations, other foundation-scale models exhibit substantially fewer traps. These two models therefore stand out as concerning cases, suggesting potentially pathological weight structure that warrants targeted investigation.

1. Do Correlation Traps occur beyond anti-grokking, and why focus on this setting?

Correlation Traps observed outside the anti-grokking setting. SETOL (Martin and Hinrichs, 2025, Section 6.5, Figure 25) compares the MLP3 at 16× and 32× the baseline learning rate, corresponding to 0.16 and 0.32. At 16×, the largest eigenvalue after entry-wise randomization of the weight matrix lies near the MP bulk edge. At 32×, traps start to separate from the bulk, coinciding with worse training and test accuracy. The learning-rate experiments use five random seeds. This provides an example of inducing a Correlation Trap through large learning rates, without requiring a preceding grokking transition.

Our focus on anti-grokking lets us compare a model before it generalizes, after it generalizes, and after that generalization is lost. Pre-grokking and anti-grokking provide a controlled setting for asking whether the diagnostic distinguishes failure to acquire generalization from the loss of an already-acquired generalizing solution.

We will make this connection to the high-learning-rate results explicit in the revision.

2. What happens when traps are present from initialization?

To examine whether a network can learn while such traps remain present, we constructed an FC1 initialization containing 33 detected traps, using rescaled contributions from an MNIST-trained donor matrix (from the anti-grokking state) together with a random remainder. This is a controlled trap-containing initialization.

Step 1. Identify the 33 traps in the initialized FC1.

Let W0W_0 be the initial weight matrix and PP the fixed shuffle of its entries. We decomposed

A0=P(W0)=UTΣTVT⊤⏟33 traps+UBΣBVB⊤.A_0 = P(W_0) = \underbrace{U_T \Sigma_T V_T^{\top}}_{\text{33 traps}} + U_B \Sigma_B V_B^{\top}.

We saved UT,ΣT,VTU_T, \Sigma_T, V_T and the bulk singular values ΣB\Sigma_B. These were identified in the initialized network, because adding the random remainder can rotate the original donor's trap vectors.

Step 2. Remove proposed updates that would change the trap vectors.

After an ordinary AdamW step proposes W~\widetilde{W}, form A~=P(W~)\widetilde{A} = P(\widetilde{W}). Let UCU_C span the orthogonal complement of UTU_T. Compute

B=UC⊤A~(I−VTVT⊤).B = U_C^{\top} \widetilde{A} \left(I - V_T V_T^{\top}\right).

This confines the trainable remainder to directions orthogonal to the fixed trap vectors on both sides.

Step 3. Restore the bulk singular values and reinsert the unchanged traps.

Compute B=LDR⊤B = L D R^{\top}, replace DD with the original ΣB\Sigma_B, and set

Wnext=P−1!(UTΣTVT⊤+UCLΣBR⊤).\boxed{W_{\mathrm{next}} = P^{-1}\\!\left(U_T \Sigma_T V_T^{\top} + U_C L \Sigma_B R^{\top}\right).}

Thus, the trap contribution remains exactly fixed. The bulk vectors can change, but its singular values cannot grow.

Step 4. Continue supervised training normally for everything else.

FC1's bulk vectors, its biases, and all later-layer weights and biases remain trainable. Forward and backward passes use the weights in their original coordinates.

We then trained on the full MNIST training set for 30 epochs, or 9,000 updates per run, across three seeds. After every AdamW update, we restored the 33 trap vectors and their singular values. We also fixed the remaining singular values so that the bulk could not grow and make the traps disappear relative to the fitted threshold. The bulk vectors, biases, and later layers remained trainable. The matched control imposed the same singular-value restriction but allowed the trap vectors to change.

The final results were:

ConditionTrain accuracyTest accuracy
Ordinary initialization99.93% ± 0.02%98.03% ± 0.10%
33 original FC1 traps fixed99.63% ± 0.21%51.44% ± 0.28%
All singular values fixed; trap vectors allowed to change98.78% ± 0.25%97.48% ± 0.33%

Values are means ± sample standard deviations across three seeds. All 273 detector checks returned exactly 33 traps under the fixed defining shuffle. Other shuffles were not constrained to return the same count.

Minor comments

We will clarify the Figure 1 color descriptions, explicitly identify Table 5 as an appendix table, connect lines 432–436 to the continued-training experiment in Appendix C.4, and correct the grammatical issue. We will also incorporate the additional experiments and the distinctions above into the updated manuscript, which we will upload shortly.

Reference

Martin, C. H., and Hinrichs, C. (2025). SETOL: A Semi-Empirical Theory of (Deep) Learning. arXiv:2507.17912, Section 6.5, Figure 25.

Thank you again for your constructive feedback. We hope these explanations address your questions and would be happy to clarify any remaining points or continue the discussion.

Comments on the paper

1 comment
AuthorsOpenReview ↗

KS clarification pt 3 - investigating MP

3. Whole-matrix Monte Carlo calibration and checkpoint results.

We used whole-matrix Monte Carlo calibration to account for eigenvalue dependence and the fitting procedure. For each checkpoint, we generated 1,000 dimension-matched Gaussian reference matrices and 1,000 matrices by iid resampling of its empirical weights. All matrices underwent the same centering, standardization, shuffle procedure, and WeightWatcher fitting and cutoff settings.

Our statistic was the mean full-spectrum KS discrepancy across shuffles after minimizing over MP scale. Each base matrix contributed one independent reference statistic. We computed Gaussian-reference Monte Carlo p-values using the upper-tail rank [3]

pMC=1+b1001,p_{\mathrm{MC}}=\frac{1+b}{1001},

where bb counts Gaussian reference statistics at least as large as the observed value.

CheckpointTraining stepMean KS discrepancyGaussian-reference pMCp_{\mathrm{MC}}
Pre-grokking1401.191%0.112
Grokking66,6671.130%0.941
Anti-grokking10,000,00013.516%≈0.000999\approx0.000999

Here 1/1001≈0.0009991/1001\approx0.000999 is the minimum attainable Monte Carlo p-value. An independent 1,000-matrix Gaussian validation bank gave a median calibrated value of 0.480, with 54/1,000 below 0.05.

Anti-grokking shows a substantial departure that persists after removing detected outliers and refitting the conditional bulk: discrepancy 7.616%, with pMC=1/1001p_{\mathrm{MC}}=1/1001.

4. What these comparisons establish wrt MP.

Pre-grokking and grokking show excellent agreement with MP, with discrepancies of just 1.191% and 1.130%, respectively, and no significant departure under whole-matrix Gaussian calibration (p=0.112p=0.112 and 0.9410.941).

In contrast, anti-grokking exhibits a substantial and statistically significant departure from MP, as we saw before, with a discrepancy of 13.516% (p=0.000999p=0.000999). Importantly, this departure persists even after removing detected spectral outliers and refitting the bulk (7.616%, p=0.000999p=0.000999), indicating that the deviation is not confined to isolated eigenvalues.

References

[1] Götze and Tikhomirov (2014). On the Rate of Convergence to the Marchenko–Pastur Distribution, version 3, Theorem 1.1.

[2] Simard and L’Ecuyer (2011). Computing the Two-Sided Kolmogorov–Smirnov Distribution.

[3] Phipson and Smyth (2010). Permutation P-values Should Never Be Zero.

[4] Yaskov (2016). A short proof of the Marchenko–Pastur theorem, Theorem 2.1 and Proposition 2.1.

[5] Bao and He (2022). On Cramér–von Mises statistic for the spectral distribution of random matrices, Section 1.1.

[6] Claeys, Fahs, Lambert, and Webb (2021). How much can the eigenvalues of a random Hermitian matrix fluctuate?, Theorem 1.3 and Remark 1.1 in the linked preprint.

[7] Landa, Zhang, and Kluger (2022). Biwhitening Reveals the Rank of a Count Matrix, Section 5.1 and Table 1.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.