- Rating
- 2
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper studies whether representation geometry can provide useful observables of grokking. The authors introduce M_int, a label-free scalar statistic that measures pairwise overlap between hidden features. They relate M_int to participation-ratio effective rank and introduce target-subspace alignment to formalize whether a representation captures task-relevant directions.
Theoretically, the paper establishes an unobservability result showing that the complete history of M_int, and more generally statistics based only on the feature-side Gram matrix, is insufficient in general to determine the target-relevant geometric transition. Empirically, the authors study the evolution of M_int during grokking on modular tasks, construct an M_int-based detector, and investigate interventions that explicitly penalize representation overlap. The interventions can advance the initial onset of generalization in the tested settings, although this comes with reduced retention and final test performance.
Critical strengths
-
The general question of whether representation geometry provides useful observables of learning progress during grokking is interesting. If one could identify an internal geometric quantity that reliably precedes generalization, this could potentially provide both mechanistic understanding and a practical diagnostic of learning dynamics.
-
The paper contains a substantial theoretical component. In particular, the unobservability result clearly characterizes an important limitation of feature-overlap statistics: identical overlap dynamics can in principle correspond to different target-relevant representation dynamics.
-
I appreciate that the paper includes several controls rather than simply reporting a correlation between M_int and test accuracy. In particular, the experiments investigate cases in which representation overlap changes without successful generalization, as well as interventions that directly modify overlap.
-
The intervention experiments are interesting. Penalizing representation overlap can substantially change the onset of generalization, suggesting that the measured geometric quantity is not merely a passive statistic of training dynamics.
Critical weaknesses
My main concern is that I do not yet understand the relationship between M_int and grokking, despite this being the central framing of the paper.
First, the different characteristic times associated with M_int do not appear to track the grokking time particularly well. Already in the schematic in Figure 1(b), t_peak, t_detect, and t_grok are substantially separated. This separation is even clearer in the experiments: the M_int peak can occur hundreds of epochs before grokking, and changing hyperparameters can substantially shift the grokking time without producing a comparable shift in the detector time. I therefore do not understand what quantitative relationship is expected between t_peak, t_detect, and t_grok.
More fundamentally, this makes it unclear in what sense M_int should be considered an observable "of grokking", rather than simply an observable of representation dynamics. The theoretical results show that even the complete M_int history is insufficient in general to determine the target-relevant transition, while the empirical detector can also trigger for runs that never grok. If M_int neither reliably determines whether grokking occurs nor predicts when it occurs, I think the paper needs a sharper explanation of what information about grokking M_int actually provides.
Second, I found the geometric meaning of M_int difficult to build intuition for from the current presentation. Figure 1(a) explains how the scalar is computed from the representation matrix, but does not give an intuitive geometric picture of what high versus low M_int representations actually look like. I think simple low-dimensional visualizations would help substantially. For example, the authors could show configurations of feature vectors or representation points with different M_int values, and snapshots of the geometry before, around, and after the M_int peak.
Third, the empirical scope is relatively narrow, and it is unclear how the proposed phenomenon transfers to other settings. Most of the evidence comes from modular tasks and a shallow MLP. The Transformer experiment also seems to behave differently from the characteristic MLP picture. It would strengthen the paper considerably to demonstrate that M_int captures a meaningful phenomenon in deeper networks, different architectures, or tasks beyond the standard modular grokking setting.
Finally, M_int is exactly related to participation-ratio effective rank for fixed feature dimension. This raises a novelty question: if M_int is essentially a monotonic reparameterization of an existing effective-rank statistic, what new information does M_int provide as a representation diagnostic? This question becomes particularly important if the main conclusion is that M_int is an informative descriptor of representation dynamics rather than a reliable predictor of grokking.
Questions for authors
-
What exactly is the expected relationship between t_peak, t_detect, and t_grok? Is there a quantitative correlation between these quantities across seeds, hyperparameters, architectures, or tasks? In the current experiments, they appear to respond quite differently.
-
In what precise sense should M_int be considered an observable "of grokking"? The theoretical results suggest that the complete M_int trajectory cannot generally determine the target-relevant transition, and empirically the detector neither determines whether grokking occurs nor closely tracks when it occurs. What information about grokking does M_int provide beyond indicating that representation geometry is changing?
-
Could the authors provide a more intuitive geometric visualization of M_int? For example, it would be helpful to show simple low-dimensional feature configurations corresponding to high and low M_int, and ideally how this geometry evolves during training.
-
Since M_int is exactly related to participation-ratio effective rank for fixed K, what is fundamentally new about M_int compared with directly monitoring effective rank? Are there settings in which M_int provides information that participation ratio does not?
-
How broadly do the empirical observations generalize? Do similar M_int dynamics appear in deeper networks, other architectures, non-modular tasks, or more realistic datasets? The Transformer results seem to already suggest that the characteristic MLP dynamics are not universal.
-
What is the intended practical use of monitoring M_int? If its peak or detector time does not reliably predict t_grok, what decision could a practitioner make based on observing M_int? The intervention experiments provide one possible application, but advancing the onset also appears to reduce retention and final performance.
2 comments
## To Reviewer V9jj (part 2)
Q4 — Since Mint equals PR for fixed K, what new information does it provide?
Thank you for this question about informational content. For fixed and the same column-normalized matrix, Mint provides no additional scalar information beyond PR — they are exact reparameterizations via Eq. (2), and we should not suggest otherwise.
The value of the Mint formulation is operational, not informational: the cosine formulation makes a natural differentiable penalty that enters the loss function directly, while PR is more natural for interpreting effective dimensionality. They are the same number; the choice is one of convenience for the application at hand. Mint's specific contribution is the dual role described in our response to W1 of Reviewer UpSV: the same quantity serves as both the diagnostic analyzed theoretically (§3, Theorem 1) and the penalty used for intervention (§4, Table 2). When changes during training (e.g., as neurons die), it must be tracked alongside Mint — this is a normalization shift, not additional information content.
To alleviate this concern, we will state the PR equivalence and this operational distinction explicitly in the revision.
Q5 — How broadly do these observations generalize?
Thank you for this question about scope. Our experiments cover a two-layer ReLU MLP on modular addition and multiplication, and a one-layer Transformer on modular addition. The Transformer results show a boundary peak (all 10/10 baselines peak at epoch 0), which does not support the MLP's internal-peak phenomenon. This is an honest scope boundary, not a minor caveat.
We have not tested deeper MLPs, larger Transformers, or non-algorithmic tasks. Whether the onset–retention trade-off and the information limit transfer to these settings are genuine open questions that we cannot resolve with a disclaimer.
To alleviate this concern, we will add an explicit scope paragraph stating what architectures and tasks are covered, what the Transformer boundary peak means for generality, and what remains untested.
Q6 — Practical use of Mint; how to address the onset–retention cost
Thank you for this question about practical implications. In the submitted paper, we do not demonstrate a validated practical policy. The onset–retention trade-off is the paper's main empirical finding: β = 0.17 halves the onset time (from ~800 to ~400 epochs) but cuts retention from 1.000 to 0.047 and final accuracy from 1.000 to 0.885. These costs are central to the result, not secondary caveats.
Our preliminary experiments (not in the submitted version) suggest a path toward resolution: applying the penalty only until generalization onset, then removing it, preserves the earlier onset (epoch ~405 vs. baseline ~840) with retention restored to 1.000 and final accuracy to 1.000 (tested on 2–3 seeds in a local environment). If confirmed at larger scale, this would mean that temporal scheduling of the Mint penalty — apply early, remove after onset — could provide the acceleration without the retention cost.
To alleviate this concern, we will describe this penalty-scheduling direction as future work with appropriate caveats in the revised conclusion, clearly distinguishing it from the submitted results.
## To Reviewer V9jj (part 1)
W1 — Core relationship between Mint and grokking is unclear; time relationships are not stable
Thank you for this concern about the central framing. The reviewer is right that Mint's characteristic times do not reliably track grokking time. The data show this clearly: the raw Mint peak occurs at epoch ~40, the detector alarm at epoch ~60, and robust onset at epoch ~800 ± 29. Under hyperparameter changes, onset shifts by thousands of epochs while the alarm barely moves (e.g., lr=1e-4: onset ~2440, alarm ~148; wd=0.03: onset ~2200, alarm ~54). There is no stable proportionality.
The relationship between Mint and grokking should be stated as a boundary, not a prediction. Mint measures feature overlap — it does not specify which target directions the representation captures. Theorem 1 formalizes this: two trajectories can share identical overlap histories while differing in whether they capture a fixed target. The negative controls confirm this empirically: overlap changes occur without true-label generalization. The detector is therefore better described as an early representation-change detector, not a grokking predictor.
To alleviate this concern, we will rewrite the introduction to frame Mint as a bounded observable (information limit) rather than a grokking predictor, and remove overstrong positive-observability language from the abstract.
Q1 — Expected relationship between t_peak, t_detect, and t_grok; quantitative correlation
Thank you for this question about quantitative relationships. No stable quantitative relationship exists in our data. The peak-to-onset gap ranges from ~760 epochs (baseline) to ~2290 epochs (lr=1e-4), with no consistent ratio or proportionality. At the current logging resolution (every 10 epochs), the detector alarm shows zero seed-level variation in the baseline (all seeds at epoch 60), so there is no within-condition correlation to report either.
To alleviate this concern, we will provide a per-run table of all three times across conditions and seeds, and state explicitly that no predictive relationship is established.
Q2 — In what sense is Mint an "observable of grokking"?
Thank you for this question about the meaning of observability. Strictly, Mint is not shown to be an observable of behavioral grokking in the predictive sense. Theorem 1 establishes an information limitation: the complete Mint trajectory cannot in general determine whether generalization has occurred. The negative controls confirm this empirically — alarms fire for non-grokking runs. The detector does not predict grokking; it detects representation change.
What Mint does provide beyond indicating geometry is changing: (1) a formal information limit on what any feature-overlap monitoring can achieve (Theorem 1), regardless of monitoring frequency or cleverness; and (2) a controllable intervention handle — when used as β⋅Mint2 in the loss, it changes onset and retention in a structured, measurable way (Table 2).
To alleviate this concern, we will correct the abstract's implication of positive observability and reframe the paper around the information limit.
Q3 — Geometric visualization of high/low Mint
Thank you for this suggestion. We provide a new mathematical schematic (attached as Figure R1:
geometry_schematic_v2.png).Panel (a) plots Mint=∣cosθ∣ and rPR=2/(1+cos2θ) continuously as the angle θ between two feature columns varies from 0° to 90°. Three key points are annotated: parallel (θ=0°, Mint = 1, fully redundant), θ=45° (Mint ≈ 0.71), and orthogonal (θ=90°, Mint = 0, fully independent). The PR curve on the right axis confirms the exact one-to-one equivalence.
Panels (b) and (c) illustrate the information limit in 3D: two pairs of orthogonal features (both with Mint = 0) span different planes. In panel (b), the target direction lies outside the feature span — the representation cannot capture it. In panel (c), the target lies inside the span — fully captured. Same Mint value, different target alignment. This is the geometric content of Theorem 1: overlap alone does not determine target capture.
All vectors represent feature columns across probe samples, not data points in hidden-coordinate space. The figure is clearly labeled as a mathematical schematic, not experimental data. Showing real training-time geometry snapshots would require recovering the original checkpoint representations and is noted as additional work.