acceptodds
Under review as a conference paper at ICLR 2027

Geometric precursors of learning transitions

Abstract

A model can begin reorganizing internally before a new capability becomes meaningfully detectable in its output behavior. After that point, intervention may require expensive rollbacks during training, so efficient steering requires cheap early detection methods. To this end, we develop a geometric framework for understanding and detecting such changes by tracking latent representations on relevant inputs. By constructing Gram matrices out of latent representations, we are able to give an invariant decomposition of motion during learning into growth, spectral allocation, internal rotation, and support transport. Our central theoretical result relates the amplitude of an emerging response to the rate of its geometric motion, explaining when normalized geometry can change before amplitude-dependent measurements reveal it. Using this description, we derive scalar early warning signals of learning. We then test this method in three settings (deep linear networks, induction heads, and the Pythia suite), each with decreasing prior knowledge about future learning events. In deep linear networks, support transport predicts the next acquired mode. For induction, alarms based on representations alone precede behavioral expression and remain silent on controls. Across Pythia scales, we identify recurring internal reorganizations and a shift from early growth to later internal rotation.

Primary area:
learning theory
Keywords:
learning dynamics, representation geometry, early warning, phase transitions, representation learning

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_iWr2OpenReview ↗
Rating
4
Confidence
3
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

This paper investigates whether monitoring changes in the geometry of internal representations during training (the Gram matrix and its support subspace) can detect key events in model learning in advance, or locate them automatically.

Critical strengths

The paper provides a clear theoretical explanation for why internal learning can precede behavioral expression. I find this distinction particularly interesting and useful for understanding apparently sudden learning transitions. (1) The problem is meaningful. A geometric method for monitoring training dynamics that does not rely on labels for the target capability is valuable for understanding emergent abilities and for the interpretability of the training process. (2) The experiments build on each other step by step, moving from "known learning target" to "known capability, unknown mechanism," and then to "both capability and mechanism unknown." The logic is clear.

Critical weaknesses

Points that may need further clarification: (1) What is the detector's false positive rate? For example, how many events does it trigger when the checkpoint order is shuffled, or when random or unstructured probe data is used? (2) On Pythia, do the geometric events reliably come before the corresponding changes in behavior or metrics? (3) How is the detection threshold chosen? I suggest adding a sensitivity analysis of the detection threshold to show that the conclusions do not depend on one particular set of settings.

Questions for authors

Show the four motion components separately for the different observation views and layers. Testing alternative probe distributions would further clarify which findings reflect a general learning progression.

1 comment
AuthorsOpenReview ↗

Response to Reviewer iWr2

Thank you for the careful review. We address your points below.

False positives, timing, and thresholds

  • False positives. We use random-token prefixes for induction detection and find no alarms in the 12 control runs, while detecting all 28 measured transitions in advance. The controls include training without repetition and slower-learning runs. We use these controls because shuffling checkpoints would itself introduce abrupt changes into the trajectory.

  • Timing relative to behavior. Some Pythia alarms precede substantial changes in the measured behavior, including the late 1B event shown in the paper. Other events overlap the measured changes or have no identified behavioral counterpart. We therefore find advance warnings in specific cases and can locate learning changes more generally. We will distinguish these outcomes more clearly in the text.

  • Threshold choice. We first compare each geometric departure with the size of ordinary fluctuations in its recent history. We then require the resulting score to exceed a fixed cutoff at two consecutive observations. We chose the cutoffs empirically during development: 4.5 for induction and 3 for Pythia. We kept the Pythia cutoff when moving from the smaller model to larger models. Raising the induction cutoff from 4.5 to 6 preserves all 28 pre-transition detections and zero alarms in the 12 controls. The Pythia 1B catalogue is stable over a narrower local range. We will clarify the selection procedure and report these sensitivity checks in the text.

Motion by layer and observation view

In response Figure 2, we show the four motion components across all stored layers and observation views in Pythia 1B. We find that internal rotation has the largest median share of later motion throughout the transformer layers, while growth remains dominant at the embeddings. The zero-transport panels reflect full resolved support in the hidden-state views, not missing data or an absence of learning; attention also shows transport across the spectral cutoff. We also checked different subsets of the existing probes, but not a new probe distribution.

The two response figures and their captions are in the anonymous figure supplement.

Reviewer_W844OpenReview ↗
Rating
2
Confidence
2(−1)
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes a new method to monitor capability creation in a model using Gram Matrices, motivated by the idea that can begin reorganizing representations internally before any capability signal is present in the output. Overall idea is to give model a fixed set of inputs called probes at every checkpoint and record how similar the model things each pair of inputs is, this gives a model snapshot for that checkpoint powered by Gram Matrices. They measure change in these snapshots against a fixed point which doesn't change throughout the checkpoints.

Any change is composed of four overlapping pieces i.e Growth (everything gets stronger together), allocation (weight shifts among existing directions), internal rotation (existing directions rotate among themselves) and support transport (model starts using new direction previously not used). Key claim is that, a new skill starts small, so its strength grows slowly but its geometry can finalize much faster, indicating that internal structure can change way before behaviour does. To convert this signal into alarm, they define what "no relevant change" means and that acts as a fixed point, deviation from which, for several checkpoints can be flagged.

To support the theoretical claims, they test in four experimental settings, each with less prior knowledge than the previous one.

Critical strengths
  1. A principled framework that introduces a way of measuring change that follows from sensible choices rather than arbitrary ones. The four-way split is useful.
  2. Useful comparison against existing techniques like CKA and CCA showcasing partial capture of the four kinds of change in them.
  3. The experimental design is careful, the alarm used only past information present at any checkpoint, and stays silent on runs that cannot learn the target capability. Forecasts are calibrated and evaluated on separate runs.
  4. A fair comparison against the loss-fluctuation method of Hennick & Corlouer, 2026 on the same runs.
  5. Paper reports honestly about the limits in the Pythia results that shows when something changes, but unsure about what it means,.
  6. All the computations scales with the number of probes, so it is cheap to run irrespective of model size.
Critical weaknesses
  1. The “geometry before strength” argument works when you can isolate the tiny new feature by itself, which they do in the linear networks and in the repetition-conditioned induction view (Eq. 22). But that conditioned view is not blind, because it uses repeated sequences. The blind induction alarm and the Pythia observer instead watch the whole representation, where the new feature is a small addition on top of existing structure. Pythia’s repeated-minus-permuted views (Eq. 23) do not remove this structure, because they subtract between inputs at the same checkpoint rather than removing what is already learned. The blind alarm may still work, but probably for a different reason.
  2. The four way split defined does not drive the alarm, induction alarm uses total Gram velocity without projecting onto any of the four components. The decomposition is used as a detector in the linear networks and descriptively in Pythia.
  3. The forecasting claim doesn't hold up in the own re-run with the MAE of 38 same as the fixed training step where as the main text says their forecast is off by 30 steps, compared with 50 for fixed training step.
  4. Induction circuits are known to form in two stages, with a previous-token heads forming first and then induction heads. The blind alarm might be catching this change itself instead of a new early warning.
  5. For Pythia, there are 13 behavioural comparisons across 45 time windows, with that many comparisons, some might look special by luck. A comparison against how often a random comparison looks special, as well as elimination of check that events dont line up with learning rate schedules will definitely help.
  6. With 32 probes and feature dimensions far larger than that, the centered Gram matrix is generically full rank, so there are no unused directions, yet transport fractions are reported. Is there a cutoff for transport classification? If yes, what is that? as that can dictate the classification from transport or allocation.
  7. The main text is dense, hard to follow and relies heavily on the appendix. It should also point readers to where key claims are proved, such as the orthogonality proof for Eq 4.
  8. The paper is pitched as a tool for steering training early, but not experiment actually uses an alarm to intervene
Questions for authors
  1. Does blind alarm in induction heads coincide with forming of the previous-token heads ?
  2. Comparison against random chance for Pythia and lining up with learning rate schedules.
3 comments
AuthorsOpenReview ↗

Response to Reviewer W844 (1/2): theory, forecasting, and induction

Thank you for the careful review. We address your points below.

1. Weak responses on established structure

  • Connection to blind detection. We subtract motion predicted from recent history, while the Gram metric can remain sensitive to weak directions within the larger representation. We use these properties to connect the theory to blind detection without identifying a particular emerging feature. Repeated-minus-permuted inputs alone do not remove established structure.

  • What the timing theorem establishes. The theorem concerns a weak observed response, not an arbitrary full representation. In our new nonlinear experiment, we test its timing prediction for an emerging response on top of an established one. In the stored blind-alarm checks, we find departures from recent geometric motion, but cannot identify this timing mechanism as their cause. We will make that distinction explicit.

2. Decomposition and detection

  • Known mechanism. In DLN, we use the decomposition to select residual support transport as the detection signal.

  • Unknown mechanism. In induction and Pythia, we detect departures from the recent geometric path, not total speed alone. We need not select a motion component in advance. In Pythia, we then use the decomposition to describe what changed; we do not claim that selecting a component improves the alarms.

We will clarify these uses. The four components are orthogonal, not overlapping, and the blind detector compares with a prediction from recent history rather than a fixed snapshot.

3. Forecasting comparisons

The main text and appendix report different studies, which we should have distinguished more clearly.

  • Original comparison. Median absolute forecast error is 30 steps for the geometric method versus 50 for the fixed-time baseline. We reproduced this result from the saved individual predictions.

  • Later comparison. The appendix reports 38 steps for both methods. This is a tie, not a forecasting improvement.

  • What the warning adds. Both methods use earlier runs for calibration. The fixed-time baseline predicts a transition at a typical training step; our method first requires a geometric change in the current run. These representation-triggered warnings precede induction and remain silent in the tested controls. They therefore provide evidence of change in the current run even when the resulting timing forecast ties the baseline.

We will distinguish the two comparisons and separate early detection from accuracy of the predicted transition time. The original study shows a forecasting advantage; the later study shows equal median timing accuracy.

4. Previous-token attention

We find that the alarm precedes the measured sustained rise in previous-token attention in all 28 transition runs. At alarm time, that attention remains near the uniform reference (response Figure 1). The alarms therefore do not simply mark the appearance of strong previous-token attention under this measurement. We can establish the order of the measured changes, but not which circuit causes the alarm.

The two response figures and their captions are in the anonymous figure supplement.

We address points 5 to 8 in the second part of this response.

AuthorsOpenReview ↗

Response to Reviewer W844 (2/2): Pythia, cutoff, clarity, and steering

This continues part 1 of our response with points 5 to 8. Points 4 and 5 also address your two concluding questions.

5. Random windows and the learning-rate schedule

  • Random comparison. We compared all seven Pythia 1B event windows with random windows of the same number and lengths within the measured training periods. Our windows capture 50.7% of normalized squared change across internal and output measurements, versus a random mean of 42.7%. We average related measurements rather than select favorable event-measurement pairs. The advantage is smaller for output behavior alone. We use this exploratory comparison to test whether the windows locate learning changes.

  • Learning-rate schedule. We reconstructed the schedule and found that none of the seven later event cores coincides with warmup completion or arrival at the minimum learning rate. The 122k event occurs while the rate is constant at its minimum. The separate early propagation event occurs during warmup. We checked schedule boundaries, not all possible learning-rate effects.

6. Rank and the spectral cutoff

  • Definition. We resolve directions whose eigenvalues exceed τ=0.1tr⁡(K)/32\tau = 0.1 \operatorname{tr}(K)/32. Transport means motion between directions above and below this cutoff, not entry into an exact nullspace. We center all 96 probes before selecting 32-probe subsets, so these subsets can remain full rank.

  • Sensitivity check. In Pythia 1B, we halved and doubled the cutoff, checking motion fractions and alarms separately. Attention fractions change, but rotation remains the largest component at nearly every later checkpoint. Five of seven event cores persist; two are cutoff-sensitive. The alarm uses normalized shape and a smooth support measure, not the hard motion labels.

We will state the cutoff explicitly and call the below-cutoff block unresolved motion rather than imply exact rank birth.

7. Clarity and proof pointers

We will define the mathematical objects before using them and add a direct pointer from Eq. 4 to its orthogonality proof. For each experiment, we will state the question, measurement, and evaluation together, and bring the quantitative comparisons into one place.

8. Steering

We have not tested an alarm-triggered intervention. We test whether we can detect learning changes before substantial behavioral expression, which is a prerequisite for early steering. We will distinguish that evidence from successful steering and identify interventions during the warning window as future work.

I thank the authors for the detailed response and the additional analysis, and I acknowledge the errors in my summary; they were typos on my end. Several points are now clear, and I appreciate the clarifications of what the results do and do not show.

However, post-rebuttal, the contribution seems narrower than the submission suggests, particularly for the induction and Pythia results. The link between the theoretical framework and empirical results seems less direct than presented.

I am keeping my score, but I've slightly lowered my confidence after further reflection.

Reviewer_p6a9OpenReview ↗
Rating
3
Confidence
2
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

The paper provides a comprehensive and elegant description of the training dynamics of representations in four independent terms / flows. The flows can be used to detect changes in learning dynamics before they manifest in performance. While the main results are theoretical, several experiments are presented for illustration: 1) A deep linear networks trained against a simple 3D linear teacher; 2) small scale transformer learning of "induction"; 3) Experiments with the Pythia language model

Critical strengths

Important problem: Learning in large scale models is still very much wizardry, with limited understanding of the learning process. Typical learning proceeds with some kind of validation set based early stopping under various implicit regularizers. Better learning diagnostics can potentially lead to more sustainable AI.

Strong theoretical progress reported. The detailed decomposition of the learning process in terms of the latent space flow is interesting and likely to spawn new theoretical research as well as empirical approaches that can lead to new learning practices.

Critical weaknesses

The empirical results are not completely clear to me: While the first "linear learner on linear teacher" experiment is well defined, it is unclear how it examines the key assumptions of the theory. Challenging the design with weak non-linearities could be interesting.

The second and third experiments are qualitative illustrations and lack (as far as I understand - the details are scattered over several sections, appendices and references) baselines and concrete hypotheses that can be quantitatively tested.

Questions for authors

Could you provide a clear objective/hypothesis to test in the experiments, introduce intuitive baselines and report the evidence of improvement over baselines (tables incl uncertainty estimates) and draw quantitative conclusions?

2 comments
AuthorsOpenReview ↗

Response to Reviewer p6a9

Thank you for the careful review. We address your points below.

Theory and the nonlinear test

In the deep linear experiment, we test whether the representation moves toward the next teacher mode before meeting the teacher-subspace acquisition criterion. Knowing the teacher lets us check the direction of this change independently.

Following your suggestion, we also tested a weakly nonlinear teacher-student network with two hidden units and a smooth erf activation. Near a learning plateau, the units begin to specialize. Their hidden responses separate at first order, while their summed output changes only at second order. We verified a growing specialization direction, then measured residual geometry after removing the shared response recorded at the plateau. We found that geometric change preceded output improvement in all nine trajectories. Reducing the initial hidden-unit difference barely changed the geometric crossing time but delayed output improvement by the amount predicted from the local growth rate. With this experiment, we test the theory's separation between early geometric motion and delayed expression in a nonlinear learning problem. We will add this controlled test and clarify what the linear experiment examines.

Measurements and quantitative comparisons

We test whether geometric changes anticipate or locate independently measured learning changes. For a known transition, we measure warning lead, L=Ttransition−Talarm,L=T_{\mathrm{transition}}-T_{\mathrm{alarm}}, together with detection and control-alarm rates.

  • Deep linear networks: Does the remaining representation turn toward the next teacher mode before acquiring it? We compare residual-projector displacement with teacher-subspace acquisition and check the direction subsequently learned. Rank-one teachers provide controls with no further mode to acquire.

  • Weakly nonlinear networks: Does the separation in timing follow the theory? We vary the initial difference ϵ\epsilon between two hidden units. Geometric crossing times remain approximately fixed, while output-improvement times increase as λ−1log⁡(1/ϵ)\lambda^{-1}\log(1/\epsilon), consistent with the growth rate λ\lambda computed independently from the local dynamics.

  • Induction: Does a departure from the recent representation path precede induction? We compare alarms from random-prefix representations with the sustained induction-attention crossing, and evaluate warning lead and control alarms against raw geometric speed, SHC, participation ratio, and loss-based measures. We find warnings in all transition runs and none in the controls, with earlier median warnings than raw geometric speed and SHC.

  • Grokking: Can geometry distinguish successive learning stages? We compare the onset of shape-dominated motion and subsequent spectral-entropy decrease with memorization, first sustained above-chance test performance, and sustained 90% test accuracy. Controls without weight decay memorize but show neither geometric stage.

  • Pythia: Do detected windows contain more learning change than matched random windows? We normalize squared checkpoint changes in each evaluation measurement by their mean within the measured training period, then average within and across measurement families to obtain EiE_i. We compare C(W)=∑i∈WEi∑iEiC(W)=\frac{\sum_{i\in W}E_i}{\sum_i E_i} against random windows with the same number and lengths within each measured period.

The following comparison uses the appendix's retrained induction runs and the stored Pythia 1B event catalogue:

Experiment and quantityGeometric detectorBaseline
Induction: median warning lead, steps [IQR]330 [292, 362]270 [245, 298], raw geometric speed
Pythia 1B: fraction of measured change captured50.7%42.7%, matched random-window mean

For induction, brackets show run-to-run spread. Neither method alarms in the 12 controls; the exact binomial 95% interval for the control-alarm rate is 0--26.5%. For Pythia, the geometric result exceeds 95.7% of matched random placements. The reported Pythia score combines internal and output measurements. We also find an advantage for output measurements alone, although it is smaller. We test whether windows locate learning changes, rather than requiring every event to predict a named capability.

We will bring these objectives, definitions, and comparisons together so that each experimental claim has an explicit quantitative test.

Reviewer_p6a9OpenReview ↗

Positive impression confirmed

Thank you for added tests and insights, I think this approach is quite promising and maintain my positive score.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.