- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
The authors define spectral shape residual (SSR) as a distance between the normalized singular-value distributions of the query and key projection matrices. In small transformers trained on modular arithmetic, SSR declines near grokking. The paper also identifies declining trends in a chosen set of heads from one OLMoE run and examines differences across layers and model pairs in published language-model checkpoints.
Critical strengths
- SSR is straightforward to calculate and reproduce without a set of evaluation examples (Section 3). The OLMoE analysis includes every one of the six head-trend groups, notably the 74 heads whose values keep diverging (Table 2), rather than restricting the report to favorable cases.
- The small-scale training studies and the Pythia null comparison offer a basis for investigating when changes in spectra coincide with shifts in training. They are useful observations even though they fall short of validating prediction.
Critical weaknesses
- Functionally identical attention can have different SSR values. Under the appropriate matrix convention, an invertible change of basis A permits WQ to become A WQ and WK to become A^{-T} WK without changing QK^T; the individual singular spectra, and hence SSR, can nevertheless change. The authors should examine this symmetry and assess whether SSR carries information beyond the geometry of a chosen parameterization. The present statistic alone cannot establish the interpretations of “Q/K coordination” or “structural health.”
- The warning-signal argument rests on little data and on choices made after observing trends. Q2 includes only three seeds. In Q3, 36 of 192 heads are chosen by their behavior before grokking, and the median of those heads is assessed using a 40% cutoff after a 50% cutoff was not attained (pp. 7–8). In addition, 74 heads continue to diverge, and five Pythia runs do not show a general decline. The authors should define both the cutoff and head-selection procedure in advance, then evaluate them on independent runs with and without grokking.
Questions for authors
- How does SSR change under a basis transformation that preserves attention outputs? After controlling for a functionally equivalent parameterization, do the reported relationships with generalization remain?
- With the rule fixed before evaluation, can SSR anticipate grokking across new models or seeds, and what false-positive rate does it produce on runs without grokking? This result would affect my view of its monitoring value.
Response to the Parameterization Symmetry Concern
We agree with the reviewer that SSR is not invariant under all reparameterizations that preserve the attention function. In particular, under the transformation
WQ′=AWQ,WK′=A−TWK,
the product WQ⊤WK remains unchanged, whereas the individual singular-value spectra of WQ′ and WK′ generally differ from those of the original matrices. Consequently, SSR can change even when the represented attention function remains unchanged.
This is an important property of SSR, but it does not invalidate the quantity we define or compute. SSR is intended as a parameterization-dependent, paired weight-space spectral descriptor, not as an invariant of the attention function. Its purpose is to quantify the discrepancy between the normalized singular-value spectra of the query and key projection weights in a given model parameterization.
We do not claim that SSR uniquely characterizes the underlying attention function, nor that a low or high SSR value alone establishes functional coordination or structural health. We will clarify this interpretation and its limitations in the revised manuscript. The parameterization dependence limits the scope of what SSR can represent, but does not prevent it from serving as a descriptive statistic for analyzing the spectral structure of a given checkpoint and its evolution under the training parameterization being studied.
Response to the Pythia Comparison
The reviewer raises a valid concern about the interpretation of our Pythia results. We analyzed five Pythia model scales (160M, 410M, 1B, 1.4B, and 2.8B), using their publicly available training checkpoints. These are five model sizes, rather than five independent training runs or random-seed replications.
Our analysis found monotonically increasing SSR trajectories and no Q/K-aligning heads in these five models under our measurement and classification procedure. However, we agree that this observation alone does not establish that the models did not undergo grokking. In particular, our Pythia analysis was not designed to independently identify grokking transitions through task-specific generalization measurements.
We therefore agree that our original statements were too strong. Specifically, the claims that this null result establishes SSR convergence as a grokking-specific signal and confirms that SSR-leading behavior requires a grokking-type phase transition go beyond what the Pythia experiment can support.
We will revise this discussion to present Pythia as a descriptive comparison: in the five examined model scales, SSR increased throughout the available checkpoints, in contrast to the aligning-head behavior observed in our OLMoE analysis. This contrast is an interesting observation, but it should not be interpreted as evidence that SSR convergence is exclusive to grokking or that the absence of SSR-leading behavior rules out grokking.
The intended conclusion is narrower: SSR exhibits different temporal patterns across the examined training trajectories, motivating further investigation into how these patterns relate to independently established generalization transitions.