acceptodds
Under review as a conference paper at ICLR 2027

Mode Dynamics: A Framework for Neural Network Learning

Abstract

We introduce Mode Dynamics, a framework that connects coordinated parameter motion during neural network learning to changes in model outputs. Starting from a recorded parameter trajectory, we decompose its updates into collective modes and map each mode to its state-dependent output effect. The framework organizes this link through three mechanisms. Decomposition identifies collective modes in parameter motion and their time-varying amplitudes. Reconstruction tests whether the effects induced by those modes recover local and accumulated output change. Concentration quantifies how behavioral contribution is distributed across modes. Controlled scalar regression and next-token experiments directly test these mechanisms. They show that small sets of leading modes reconstruct behavior, induced effects depend on the training path, and modal complexity varies with task and data. We then evaluate the framework on held-out image and language-model outputs using CIFAR-100 autoencoding and a public Pythia-70M pretraining trajectory. Across these settings, the results connect compact modal subspaces of recorded parameter motion to evolving behavior.

Primary area:
interpretability and explainable AI
Keywords:
learning dynamics, neural network optimization, modal analysis, trajectory decomposition, behavioral reconstruction, low-dimensional subspaces, mechanistic interpretability

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_6KGiOpenReview ↗
Rating
2
Confidence
2
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

The paper suggests Mode Dynamics framework, which has three steps: step one is effectively SVD on the differences between model checkpoints ;
step 2 is taking checkpoint-dependent amplitudes (ΣWT\Sigma W^T) from step 1 as weights and multiply it with the Jacobian vector products to associate the modes in parameter space with the model input-output functions;
step 3 (section 3.4 and not 3.3) shows how much each detected mode influences the input-output functions

The paper tests 4 settings - scalar regression, controlled next-token prediction, CIFAR-100 autoencoding, and a public Pythia-70M pretraining trajectory to test if leading trajectory-derived modes reconstruct observed output change

Critical strengths
  1. The paper attemps to address an important problem in the interpretability and safety domain. Indeed as stated in the introduction - the object of interest is the function learnt by the neural network and understanding degree of freedom of this function and how this function is shaped could be crucial for interpretability and safety domains.
  2. The paper is relatively easy to follow and understand on a high level
  3. Control experiments and error bars are great - initialization frozen baseline and random subspace baselines are good controls to contextualize the proposed methods behaviour. (but better explnataions that signed-controls preserve length but not angles would be helpful for a naive reader)
Critical weaknesses

1.The biggest and most crucial criticism - scientific impact of the method: While the methodology seems to be simple and elegant, the results seems to be raw. Yes, we can perform SVD on the weight changes across different model types and see that we find most important directions - but what new information does it tell us? What can we learn about neural networks from these type of analysis?

To my impression, the claims currently done in the paper are known before:

  • we already know that learning dynamics are path-dependent and could not be approximated with a single tagent plane due to non-linear complex loss landscape - see references 3-4
  • we already know that more complex tasks require more capacity / circuits / principal components / modes in the same network (see references 1, 2).

The paper mentions some potential directions in future work but given the simplicity of methodology I would argue that one of the examples of such analysis should be provided in a paper, especially something involving causal interventions might be particularly useful, to prove that modifying uku_k during training changes model behaviour - otherwise the method is still close to the post-hoc analysis like the ones described in "mechanistic interpretability" section of related work.

2.Clarity / Notation strictness:
* Sometimes new letters in the mathematical notations are introduces without explictly saying what they mean, the meaning of the indexes are also not specified everywhere (like equations 10 and 11 - what are jj there? the jj also have different meanings in both formulas).
* In general, the text seems to be overly condensed, for example, 1st paragraph of the related works is unreadable for a reader, who is not accustomed for the jargons of the field (e.g. collective coordinates ) - hence related work paragraph becomes unhelpful / looses its purpose
* Neural tangent kernel (NTK) is the only baseline and in section 4.1 and could have been introduced at least a bit in the text (what is does / what are the limitations)

  1. Lack of baselines - what are the other methods which could compare things across checkpoints? why there are not compared against?

References:

  1. Wang, K., Variengien, A., Conmy, A., Shlegeris, B., & Steinhardt, J. (2022). Interpretability in the wild: a circuit for indirect object identification in GPT-2 small. arXiv preprint arXiv:2211.00593.
  2. Olsson, C., Elhage, N., Nanda, N., Joseph, N., DasSarma, N., Henighan, T., ... & Olah, C. (2022). In-context learning and induction heads. arXiv preprint arXiv:2209.11895.
  3. Chizat, L., Oyallon, E., & Bach, F. (2019). On lazy training in differentiable programming. Advances in neural information processing systems, 32.
  4. Song, M., Montanari, A., & Nguyen, P. (2018). A mean field view of the landscape of two-layers neural networks. Proceedings of the National Academy of Sciences, 115(33), E7665-E7671.
Questions for authors
  1. Please reply for criticism 1 from the previous fields
  2. How much does the modes depend on the initial initializations? Does their amount (e.g. rank rr from page 2) depends on the initializations?
  3. Why have you used only 100 test images for CIFAR and not the whole test dataset? How these 100 images were selected?
  4. Do you think that Pythia-70M results are valid, given that checkpoints are probably quite far apart from each other? How close the checkpoints should be to each other to make Mode Dynamics conclusions trustworthy and reliable?
  5. How many modes were considered here for the participation ratio for fig 3?
  6. Do I get it right that concentration in the context of the paper is roughly the effective rank / spectral sparsity and reconstruction fidelity is the quiality of the low-rank approximation ?
6 comments
AuthorsOpenReview ↗

Main insight, the causal intervention you suggested, and answers to each question

Thank you for the detailed review.

Main insight

Our main insight is that the functional change produced by learning is organized in collective modes of parameter-space velocity. This organization lies not in individual parameters or their separate velocities, but in how the velocities of many parameters are coordinated. Treating checkpoint updates as discrete velocities, we define a mode as a direction along which parameters across many neurons and tensors move in fixed proportion. SVD extracts these directions, but not which ones matter: the directions with the most motion need not change the function most. In a new comparison on held-out Pythia contexts, the five modes with the largest output effect give R2=0.728R^2=0.728, versus 0.1340.134 for the five with the most motion. The causal test below shows the same during training.

Path dependence and task capacity

We agree that path dependence (Chizat et al., 2019, Mei et al., 2018) and task-dependent capacity (Wang et al., 2022, Olsson et al., 2022) are known. Mode Dynamics recovers both, as consistency checks rather than new claims. Evaluated on the tangent plane at initialization, all 50 Pythia modes give held-out R2=−0.018R^2=-0.018, versus 0.99980.9998 along the path. More complex tasks also recruit more modes: PR is 3.8 for copying, versus 11.0 for induction and 13.0 for random labels.

New causal intervention

As you suggested, we modified the modes during training, fixing selected amplitudes at zero. In a one-layer transformer trained with full-batch SGD on an induction task, we found the modes in a reference run, then retrained from the same initialization with every gradient projected away from the selected directions. Separate runs suppress the Top-1/3/5 dominant modes, modes with matched parameter motion but less output change, or tensor-matched controls that shuffle the dominant modes within each tensor. Across five seeds and 216 held-out contexts, step-7,000 accuracy falls from 95.7% to 29.8%, 21.5%, and 38.6% under dominant-mode suppression, versus 86–89% for energy-matched modes and about 96% for tensor-matched controls (revised Figure 4). No dominant-mode run reaches 90% by step 10,000, unlike 13 of 15 energy-matched runs.

Initialization

In a new experiment, five runs differ only in initialization. Across them, PR is 1.99±0.521.99\pm0.52, Top-5 R2R^2 is 0.974±0.0150.974\pm0.015, and KeffK_{\rm eff}, the number of modes holding 90% of the contribution energy, ranges from 2 to 5. Numerical rank varies little, from 47 to 52. The algebraic rank rr is bounded by the number of recorded updates, so it is not our mode count. We do not claim that individual directions align across initializations.

CIFAR test images

For each seed, we drew one test image per class with a prespecified seed and froze the 100 indices before scoring, excluding them from training, ranking, and mode selection. One image per class guarantees class coverage at a cost of 7,840 full-RGB JVP evaluations per trajectory.

Checkpoint spacing

The reconstruction conclusions are valid at Pythia's spacing, but the mode counts grow with checkpoint density. Going from 51 to 101 checkpoints raises held-out R2R^2 at a matched 40% of modes from 0.9668 to 0.9877, while KeffK_{\rm eff} rises from 13 to 21. In a new analysis of five controlled trajectories, refining spacing from 1,000 to 100 steps keeps Top-5 R2R^2 above 0.96, while KeffK_{\rm eff} averages 2.0–2.2 down to 200-step spacing and 3.0 at 100 steps. Every tested spacing sufficed for reconstruction, but none gave converged counts.

Baselines

Section 4.1 uses three baselines. Dimension-matched random subspaces test whether trajectory-derived directions matter, and the initialization-frozen reconstruction tests path dependence. The NTK trains the model linearized at initialization, so it cannot capture a changing Jacobian. Methods that compare checkpoints credit change to training examples (TracIn, Pruthi et al., 2020) or to single parameters (loss change allocation, Lan et al., 2019), not to coordinated directions, so they give no comparable quantity.

Concentration and reconstruction

For Question 5, Figure 3 computes PR over all 1,000 modes, one per recorded update. For Question 6, roughly yes, with one difference each. PR is a standard effective-dimension formula, but we apply it to the energies of the output contributions gk,t(x)g_{k,t}(x), not to singular values. Top-KK reconstruction is not a fixed low-rank approximation of the outputs: it projects the parameter updates onto KK modes and re-evaluates their effects along the path.

Manuscript revisions (marked in blue)

  • Rewrote the introduction, discussion, and related work in plain language, defining terms such as collective coordinates and adding the works above.
  • Defined every symbol and index at first use, including in Equations 10–11.
  • Introduced the three baselines and explained that signed-permutation controls keep each mode's length but change its direction.
Reviewer_6KGiOpenReview ↗

Follow up on main insight and causal intervention

Thanks a lot for a brief, clear reply and integrating my suggestions.

Main insight

I still struggle to distill the main insight. Given the other reviews, I am clearly no alone in this struggle. What does the insight of "how the velocities of many parameters are coordinated" actually mean? E.g. does it give some better understanding of the learning process (beyond what was know before), some ideas on how to manipulate the learning procedure or as reviewer WNUJ mentioned - do we get some a stronger link between output change and model capabilities?

Causal intervention

I find it very surprising that on a relatively simple task of induction removing one mode basically destroys the performance? How exactly do you suppress the modes? What is the by-chance performance? And why do you think the model does not adjust to relearn the solution in some other direction?

AuthorsOpenReview ↗

Clarifying the main insight and causal intervention

Thank you for the follow-up. We clarify the main insight and causal test below.

Main insight

What coordinated velocities mean. A mode is a trajectory-derived direction in parameter space that specifies a shared signed pattern of parameter velocities. It determines which parameters move together or oppositely and their relative amounts. Its amplitude specifies how strongly that pattern is expressed at each step. A tensor-matched control randomly permutes and sign-flips the mode coordinates within each tensor. This preserves each mode's per-tensor norm but changes which parameters move together. Suppressing these controls does not produce the same impairment, showing that the specific coordination matters.

What this reveals about learning. Prior work showed that learning can exhibit low-dimensional parameter structure (Gur-Ari et al., 2019; Li et al., 2018; Brokman et al., 2024). Low dimensionality alone does not identify which coordinated movements change predictions or support capability acquisition. We find that large parameter movement does not necessarily imply large output change: smaller coordinated movements can affect predictions more, and these effects evolve during training. In controlled induction, suppressing the dominant output-changing modes delays held-out induction, whereas suppressing their tensor-matched controls does not. These results identify a compact, coordinate-specific route used by standard training to acquire induction.

Manipulation and capabilities. Modes provide concrete intervention targets. Projecting selected modes out of every gradient delays acquisition of held-out induction. This suggests using modes to intervene on learning dynamics, although we have tested only suppression using modes from a completed reference run.

Here, output change is the numerical movement of the logits, whereas a capability is successful behavior on unseen inputs. Held-out induction therefore provides a controlled capability measure. Reconstruction connects modes to output change. The causal experiment goes further: suppressing modes selected by their output contribution on training contexts impairs induction on unseen contexts. Testing whether this extends to broader capabilities, such as mathematical reasoning or code generation, is an important next step.

Causal intervention

How we suppress modes. For each seed, we train a reference model, extract fixed orthonormal velocity modes UKU_K, and restart from the same initialization on the same training contexts. At every SGD step, we replace the gradient gtg_t with

gt′=gt−UKUK⊤gt.g'_t = g_t - U_K U_K^\top g_t.

Thus, at every step, we remove the component of the update along the selected reference modes while leaving all other directions available. The modes are not recomputed, and neither parameters nor forward computations are removed.

Why a single mode can have a large effect. The causal result is consistent with the reconstruction evidence: retaining only the leading mode reproduces the trained model's induction decisions on the training contexts with 100% Top-1 agreement (revised Figure 4B), although additional modes improve the full logits (revised Figure 4A). We interpret suppression as strongly impairing acquisition, not completely destroying induction: mean held-out accuracy remains above chance, and some runs reroute with longer training. Larger sequence lengths or vocabularies are also associated with broader mode recruitment (revised Figure 3), so the single-mode effect should not be generalized to all capabilities.

Chance. The task-level chance baseline is 16.7% under uniform guessing among the six possible target tokens; uniform guessing among all ten output symbols would give 10%. At 10,000 steps, mean held-out accuracy under Top-1/3/5 suppression is 34.6%, 35.6%, and 48.3%, compared with 99.4% for the baseline. These conditions remain above chance on average but far below the baseline.

Adjustment and rerouting. We appreciate this question because it distinguishes delayed learning from permanent failure. Every baseline run exceeds 90% held-out accuracy by step 7,000. In contrast, none of the 15 Top-1/3/5 runs reaches 90% by step 10,000, whereas every tensor-matched control does.

We extended the same fixed-projection protocol to 100,000 steps. All five Top-5 runs reach 90% held-out accuracy by step 18,000. Two of five Top-1 runs reach it by step 13,000, and two of five Top-3 runs reach it by step 22,000. At step 100,000, the remaining six runs have held-out accuracy below 39%. Because the selected modes remain suppressed, recovery uses alternative directions. Thus, in this task, these modes provide an efficient learning route, while suppression forces slower and less reliable rerouting within the 100,000-step budget.

Reviewer_WNUJOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2(+1)
Presentation
2
Soundness
2
Read review
Paper summary

The paper introduces "Mode Dynamics", a framework that decomposes the training trajectory of the weights into "modes": the singular vectors of the concatenation of weight deltas between consecutive checkpoints. This framework approximates the mode effect on the output as a Jacobian-vector product evaluated along the trajectory. The paper evaluates if few modes are enough to reconstruct how the outputs evolve on different tasks and models. Importantly, they evaluate also on public Pythia-70M checkpoints.

Critical strengths
  1. The idea is simple, well defined and well motivated. I appreciated including the Pythia public checkpoints in the evaluation, allowing possible reproduction of the result without training
  2. The evaluations setups are clearly described, with proper held out data (although not for the toy transformer experiments, it is not clear why).
  3. From my point of view, the most interesting result is that different tasks (induction, random-label or copying) show different output change distribution across modes. In particular, copying induces outputs changes that involve a much smaller number of modes that stays more constant over sequence lengths.
Critical weaknesses
  1. The main weakness of the paper is that the novel insights gained from this analysis are limited to the one described in Strengths point 3. I think much more could have been explored, for example:

    1. if my understanding is correct, if an output change requires more than one mode, currently it is split additively across modes at each step. There is no concept or attempt to model outputs change that rely on joint-modes non additively
    2. There is not any analysis done to evaluate how the observed output change reflects on model capabilities (e.g., a downstream task, math, code). I think this is especially important, if we could show a link between modes and a specific capabilities it could be very interesting and potentially very useful (e.g., transfer of capabilities, similar to task vectors)
    3. There is no analysis of the order of active modes over time during training. Is it consistent across runs? Does it affect the final capabilities?
    4. There is not attempt to go beyond analysis: is it possible to intervene on the modes? For example, to limit an undesired capability that is emerging? Is it possible to transfer modes across models, or at least cross fine tunings from the same base checkpoint?
  2. It seems like the results depend on the checkpoints interval frequency. I feel like at least one experiment showing this is a sensible approximation is needed. Preliminary evidence reported in the paper goes against this being a good approximation (doubling the Pythia checkpoints used increase the modes from 13 to 21 for 90% of the energy)

I am willing to raise my score if the authors could show anything beyond purely analysis (e.g., mode intervention, informing actions during training) and a stronger link between output change and model capabilities.

Questions for authors
  • Can we use the mode dynamics to make predictions, before the training converges? Could we inform any training-related action effectively (e.g., change some training hparam depending on the current mode distribution)?
  • Is it not completely clear to me why individual directions of parameter change represent a behavioral change. Furthermore, do behaviors depend on multiple modes, or on their activation order?
  • In the toy settings, why not compute the modes from every optimization step? Does the effective number of modes converge as the checkpoint interval shrinks? I think that showing the checkpoint frequency is a reasonable approximation is very important.
3 comments
AuthorsOpenReview ↗

New mode intervention: suppressing dominant modes limits an emerging capability

Thank you for the constructive review. Our main insight is that the functional change produced by learning is organized in collective modes of parameter-space velocity, which capture how the velocities of many parameters are coordinated. You asked for a result beyond pure analysis, such as a mode intervention, and for a stronger link between output change and capabilities. We ran a new experiment that provides both: suppressing the dominant modes during training limits an emerging capability, held-out induction, whereas matched controls do not.

New causal intervention

In a one-layer transformer trained with full-batch SGD on an induction task, we found the dominant modes from a reference run, then retrained from the same initialization while projecting every gradient away from a fixed set of directions, so the parameters never move along them. Separate runs suppress the Top-1/3/5 dominant modes, the same number of modes with matched parameter motion but less output change, or tensor-matched controls that scramble which parameters move together within each tensor. Across five seeds and 216 contexts never used for training or ranking, step-7,000 accuracy falls from 95.7% to 29.8%, 21.5%, and 38.6% when the dominant modes are suppressed, versus 86–89% for energy-matched modes and about 96% for tensor-matched controls (revised Figure 4). No dominant-mode run reaches 90% by step 10,000, whereas all baseline and tensor-matched runs and 13 of 15 energy-matched runs do.

Why directions represent behavior

A mode is not a single parameter's direction but a pattern of coordinated velocities across many parameters, and a direction alone is not a behavior. Mode uku_k contributes gk,t(x)=ak,tJθf(x,θt)ukg_{k,t}(x)=a_{k,t}J_\theta f(x,\theta_t)u_k: how far the trajectory moves along it times its output effect at the current state. A mode matters only when this product is large. This is why the dominant modes, and not modes with equal parameter motion, control learning in the causal test above.

Predictions and training actions

The new causal test is a reference-informed training action: modes found in one run are suppressed in a second run, which changes how a capability is acquired. We have not yet predicted later behavior from an early prefix, steered a run online, adapted hyperparameters, linked modes to downstream math or code tasks, or transferred modes across models.

Multiple modes and their order

The first-order decomposition is additive because the Jacobian acts linearly on each update, but the modes are not independent. Their effects can align or cancel, and finite steps contain mixed higher-order terms that we do not attribute separately. Order enters through the state θt\theta_t at which each effect is evaluated. In a new experiment across five initializations, compactness is consistent: KeffK_{\rm eff}, the number of modes holding 90% of the contribution energy, stays between 2 and 5, and Top-5 R2R^2 is 0.974±0.0150.974\pm0.015. Each run has its own basis, however, so we have not tested whether the same modes activate in the same order. The revised manuscript states these limits.

Checkpoint density

Every-step analysis is possible but expensive. In a new analysis of five controlled trajectories, we instead refined uniform spacing tenfold, from 1,000 to 100 steps (100 to 1,000 modes). KeffK_{\rm eff} averages 2.0–2.2 from 1,000- to 200-step spacing and 3.0 at 100 steps, and Top-5 R2R^2 stays above 0.96 in every run. The counts nearly stabilize, although the leading subspaces still shift. Pythia's counts still grow from 51 to 101 checkpoints, and its public checkpoints are 1,000 steps apart after step 1,000, so we claim compactness over the tested schedules, not converged counts.

Held-out data in the toy transformer experiments

The submitted toy transformer analyses are in-sample by design. Each task has only 30 to 240 training sequences, and we asked how training reshapes outputs on them. The new causal test adds held-out evaluation: it enumerates all 1,080 valid induction contexts and holds out 216.

Manuscript revisions (marked in blue)

  • Added the causal test (revised Figure 4) and the initialization and checkpoint-density analyses.
  • Separated reference-informed action from prospective prediction and listed the untested uses above.
  • Stated that higher-order interactions and activation order are not attributed separately.

Thank you for the reply and the new experiments. I appreciated adding the mode suppression test on held out induction data, because the parameter motion control helps show that your output change weighting identifies important directions along the trajectory.

From my point of view, clear limitations still remain. The suppressed modes come from a reference run with the same initial weights and deterministic training, and some of your longer runs in the discussion still learn the task through other directions. It is still unclear if modes can guide a new run, and toy induction remains far from downstream capabilities. Also, the leading subspaces still change with denser checkpoints.

Even with these limits, you went beyond pure analysis as I asked, so I will raise my score to 3 (weak accept).

AuthorsOpenReview ↗

Thank you for your review and updated assessment

Thank you for the feedback and for increasing your score. We agree that testing whether modes can guide new training runs and extending the causal analysis to larger-scale capabilities are important next steps.

Reviewer_SWaoOpenReview ↗
Rating
2
Confidence
2
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

The paper introduces Mode Dynamics - a framework that connects coordinated parameter motion during neural network learning to changes in model outputs. The paper studies how changes in network parameters affect model outputs during training. It uses SVD to group recorded parameter updates into shared directions, called modes, and measures each mode’s contribution to output changes. Experiments on regression, next-token tasks, CIFAR-100 autoencoding, and Pythia-70M examine whether a small number of modes can reconstruct these changes.

Critical strengths
  1. The framework proposes a way to connect coordinated parameter changes with their effects on model outputs.

  2. The experiments are performed on both controlled tasks and larger models. Evaluating reconstruction on held-out images and language-model contexts strengthens the empirical evidence.

  3. The authors distinguish output reconstruction from prediction agreement. Additional checks also help separate numerical errors from errors caused by removing modes.

Critical weaknesses
  1. The main scientific insight needs to be clearer. The framework combines SVD with Jacobian-based attribution, and reconstruction using all modes follows from standard mathematical identities. The paper should explain more clearly what new understanding of learning comes from this analysis beyond describing the recorded training trajectory.

  2. Strong output reconstruction does not translate into matching predictions. On Pythia, 20 modes achieve R2≈0.967R^2 \approx 0.967, yet the trained model’s chosen token is outside the reconstruction's top 50 candidates on every test context. The authors acknowledge this gap, but it limits what the result establishes about capturing meaningful model behavior.

  3. The Pythia experiment does not compare ranking modes by their output effects with ranking them by their parameter-update energy. Without this comparison at the same number of retained modes, the benefit of the proposed behavioral ranking remains unclear.

  4. The concentration measures depend on checkpoint sampling. Increasing the Pythia checkpoints from 51 to 101 raises the number of modes accounting for 90% of the reported contribution energy from 13 to 21. Further analysis with denser checkpoints and different checkpoint spacing would help determine whether the concentration measures reflect a stable property of learning or depend strongly on how the training trajectory is recorded.

  5. The random controls provide limited evidence about what makes these modes useful. Random subspaces and signed permutations disrupt the learned updates. In CIFAR, the permutation can even move updates into a classification head that does not affect image reconstruction. Controls that preserve layer structure and relevant parameters would provide a stronger comparison.

  6. The presentation and overall conclusion need to be more clear. The paper introduces several concepts and measures, but their connection to the main research question is not easy to follow. The discussion summarizes individual findings without clearly explaining what they collectively establish about how neural networks learn. Simpler explanations and a focused conclusion linking the experiments to the claims would make the contribution easier to understand.

Questions for authors
  1. What is the main new insight about learning revealed by the modal analysis, beyond reconstructing observed output changes?

  2. What explains the high Pythia reconstruction score when the trained model’s chosen token is absent from the reconstructed top 50 candidates?

  3. How does behavioral ranking compare with parameter-energy ranking on held-out Pythia contexts at the same mode budget?

  4. Do the concentration measures stabilize with denser or evenly spaced checkpoints?

  5. Do the main findings hold with random controls that preserve layer structure and exclude parameters unrelated to the measured output?

3 comments
AuthorsOpenReview ↗

Main insight, a new causal test, and the analyses you requested

Thank you for the careful review.

Main insight

Our main insight is that the functional change produced by learning is organized in collective modes of parameter-space velocity. This organization lies not in individual parameters or their separate velocities, but in how the velocities of many parameters are coordinated. Treating each checkpoint update vt=θt+1−θtv_t=\theta_{t+1}-\theta_t as a discrete velocity, we define a mode as a direction along which parameters across many neurons and tensors move in fixed proportion. Mode Dynamics extracts these modes (decomposition), measures the output change each produces at the current state (reconstruction), and counts how many carry it (concentration). As you note, full-basis reconstruction is an identity check. The informative results are that, in the tested settings, a few modes carry most of the output change and that they need not be the modes with the largest velocity. This separates our claim from the known low dimensionality of training trajectories.

New causal intervention

To test whether learning depends on these modes, we ran a new five-seed experiment. In a one-layer transformer trained with full-batch SGD on an induction task, we found the modes from a reference run, then retrained from the same initialization while projecting every gradient away from a fixed set of directions, so the parameters never move along them. Separate runs suppress the Top-1/3/5 dominant modes, the same number of modes with matched parameter motion but less output change, or tensor-matched controls that keep each tensor's motion but scramble which parameters move together. On 216 held-out contexts at step 7,000, accuracy falls from 95.7% to 29.8%, 21.5%, and 38.6% when the dominant modes are suppressed, versus 86–89% for energy-matched modes and about 96% for tensor-matched controls (revised Figure 4). No dominant-mode run reaches 90% by step 10,000, unlike 13 of 15 energy-matched runs. In this setting, learning depends specifically on the dominant modes.

Pythia reconstruction versus prediction

Centered-logit R2R^2 pools squared error over all 50,304 logits, whereas the predicted token depends only on the largest logit. The leading 20 modes carry most of the output change (R2=0.9668R^2=0.9668), but the remaining 30 modes, though small, decide which logit is largest. Keeping all 50 modes raises Top-1 agreement from 0% to 77.3%, and to 97.7% with finer quadrature (Q=8Q=8), while 20 modes stay at 0%. We now state that R2R^2 measures output-change reconstruction, not decision recovery.

Behavioral versus parameter-energy ranking

As requested, we ran a new same-budget comparison on the locked basis and the same 128 held-out contexts. At K=1/3/5/10/20K=1/3/5/10/20, parameter-energy ordering gives R2=0.121/0.133/0.134/0.741/0.945R^2=0.121/0.133/0.134/0.741/0.945, versus the submitted 0.451/0.601/0.728/0.858/0.9670.451/0.601/0.728/0.858/0.967 for behavioral ordering. Behavioral ordering reconstructs output change far better at small budgets but does not uniformly improve KL or Top-1 agreement.

Checkpoint density

In a new analysis of five controlled induction trajectories, we refined uniform checkpoint spacing tenfold, from 1,000 to 100 steps (100 to 1,000 modes). KeffK_{\rm eff}, the number of modes holding 90% of the contribution energy, averages 2.0–2.2 from 1,000- to 200-step spacing and 3.0 at 100 steps. PR rises only from 1.7 to 2.0, and Top-5 R2R^2 stays above 0.96 in every run. The concentration measures nearly stabilize, although the leading subspaces still shift. Pythia's counts still grow with density, and its public checkpoints are 1,000 steps apart after step 1,000, so we claim compactness over the tested schedules, not converged counts.

Layer-preserving controls

We ran new CIFAR controls that preserve layer structure and exclude the classifier head: three per trajectory, each applying one random signed permutation within every RGB-relevant tensor, shared by all Top-50 modes. A control keeps each tensor's motion and the angles between modes, moves nothing across tensors or into the head, and only scrambles which parameters move together. Substituting the controls for the trajectory modes without retraining lowers Top-50 held-out RGB R2R^2 from 0.912±0.0120.912\pm0.012 to −0.909±0.559-0.909\pm0.559. The tensor-matched controls in the causal test use the same kind of shuffle, and suppressing them leaves learning intact. What matters is which parameters move together, not only how much each tensor moves. We did not run the corresponding Pythia control.

Manuscript revisions (marked in blue)

  • Rewrote the introduction, discussion, and conclusion around the main insight, with decomposition, reconstruction, and concentration as one chain.
  • Added the new experiments above. Revised Figure 4 shows the causal test, and its former qualitative panels moved to the appendix.
  • Limited the R2R^2 and checkpoint-density claims as stated above.

I thank the authors for the detailed response. The same budget comparison addresses my question about mode ranking, and the new intervention shows that suppressing the dominant modes impairs learning on the controlled induction task.

My main concern still is whether the modes identified by this analysis can explain model predictions in larger models such as Pythia. On Pythia, the leading 20 modes reconstruct much of the logit change and still yield 0% top-1 agreement. The estimated number of dominant modes also remains sensitive to checkpoint spacing. I appreciate the narrower claims in the revision but since my major concern is still remaining, I maintain my original rating of 2.

AuthorsOpenReview ↗

Clarification on Pythia resolution and prediction recovery

Thank you for the follow-up and for recognizing the same-budget comparison and causal intervention. We would like to clarify what the additional Pythia analyses show about prediction-level behavior.

The reported 0%0\% is Top-1 agreement with the trained model, not next-token accuracy. It applies to the locked 51-checkpoint reconstruction retaining 20 of 50 modes with two-point quadrature (Q=2Q=2), where QQ is the number of integration nodes per checkpoint chord. Although this truncation recovers 96.7%96.7\% of pooled logit change, it selects a different highest-logit token on the 128 evaluation contexts. Retaining all 50 modes raises agreement to 77.3%77.3\% at Q=2Q=2, while finer quadrature raises full-update agreement to 97.7%97.7\%. Thus, the 0%0\% result characterizes a specific truncation under the sensitive argmax metric, not the full reconstruction.

We varied checkpoint density and, independently, quadrature order (Appendix Figures 16–18). On the denser 101-checkpoint trajectory, Top-1 agreement rises from 0%0\% at 20 modes to 59.4%59.4\% at 80 modes under Q=2Q=2, 64.1%64.1\% at 80 modes under Q=8Q=8, and 96.9%96.9\% for the full-update reference. At matched 80%80\% retention, increasing from 51 to 101 checkpoints raises agreement from 46.1%46.1\% to 59.4%59.4\% under the same two-node rule. The denser analysis therefore already shows substantial prediction recovery, although near-full preservation with a strongly compressed subset is not yet established.

The absolute KeffK_{\mathrm{eff}} increases from 13/50 to 21/100 modes, while its fraction decreases from 26%26\% to 21%21\%. This supports relative compactness over the tested resolutions, without claiming convergence of the exact count.

Pythia constrains us to its released checkpoints, and the 101-checkpoint analysis already requires substantial computation. In contrast, for CIFAR we controlled checkpoint collection, saved 1,961 checkpoints at five-step spacing, and obtained full-update calibration R2R^2 values of 0.9830.983–0.9990.999 on the tested windows. We therefore view the Pythia results as a finite-resolution characterization of the available public trajectory.

Comments on the paper

1 comment
AuthorsOpenReview ↗

What Mode Dynamics reveals about learning: Main insight and supporting evidence

Thank you to all reviewers for the constructive discussion. The reviews raised a shared question: what does Mode Dynamics reveal about neural network learning? We summarize the central insight and supporting evidence below.

Main insight

During training, a neural network changes many parameters at once. We ask whether these changes are organized into coordinated patterns and whether those patterns matter for what the network learns.

We treat each checkpoint-to-checkpoint parameter update as a discrete velocity. A mode is a trajectory-derived direction representing a coordinated pattern of parameter velocities. It specifies which parameters change together or oppositely and in what relative amounts.

Each recorded update is a linear superposition of these fixed modes, with amplitudes that vary over training. This decomposition tells us how parameters move, but not how that movement changes the model's output function.

The key bridge in Mode Dynamics is Equation 4, which maps each parameter-space mode to its local, state-dependent effect on the output function:

ϕk(x,θt)=Jθf(x,θt)uk.\phi_k(x,\theta_t)=J_\theta f(x,\theta_t)u_k.

Here, JθfJ_\theta f is the parameter Jacobian, uku_k is the mode direction, and ϕk(x,θt)\phi_k(x,\theta_t) is its effect on the output for input xx at training state θt\theta_t. The mode amplitude tells us how strongly training moves along the direction, while ϕk\phi_k tells us the functional effect of that movement.

Because the Jacobian changes during training, the same mode can have different output effects at different model states.

With the complete basis, the parameter decomposition is exact and serves as an identity check. Our empirical questions begin after truncation: How many modes are needed to reconstruct the observed output change? Do the modes with the largest parameter motion also produce the largest output effects? Does preventing movement along the modes with the largest output contributions impair task acquisition?

We find that a small number of coordinated modes recover most observed output change. These are not necessarily the modes with the most parameter motion.

Supporting evidence

Functional concentration

On held-out Pythia contexts, ranking modes by output effect reconstructs output change substantially better than ranking them by parameter energy at the same mode budget. Parameter motion alone therefore does not determine functional importance.

Specific coordination

Tensor-matched controls preserve how much each tensor moves while changing which parameters move together. These controls do not reproduce the same functional effects or causal impairment as the original modes. The specific coordination pattern therefore matters, not only mode norm or per-tensor motion.

Causal role in learning

We tested whether the modes with the largest output contributions are important for acquiring the task. In controlled induction, we extracted fixed modes from a reference run and restarted training from the same initialization while removing selected directions from every gradient.

Across five seeds, step-7,000 held-out accuracy falls from 95.7% to 29.8%, 21.5%, and 38.6% under Top-1, Top-3, and Top-5 dominant-mode suppression. Energy-matched controls retain 86% to 89%, while tensor-matched controls remain near 96%.

The modes were selected using output effects on training contexts, while induction capability was evaluated on 216 unseen contexts. This connects mode-level functional attribution to acquisition of a held-out capability.

Longer runs show that some models eventually learn through alternative directions. We therefore interpret the dominant modes as an efficient route used by standard training, not as the only possible solution.

Revision highlights and current limits

The revised manuscript adds a causal intervention evaluated on held-out induction (revised Figure 4C-D), behavioral-versus-energy ranking, initialization and checkpoint-density studies, and tensor-matched controls. All changes are marked in blue.

The intervention uses fixed modes from a completed reference run, so it is reference-informed rather than prospective. We do not yet claim prospective prediction, online mode discovery, mode transfer across models, or generalization to broader capabilities.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.