acceptodds
Under review as a conference paper at ICLR 2027

Where Fourier Components Meet: How Features Interact in Grokking

Abstract

Learning useful components does not by itself explain how a network combines them into a computation. We develop a theoretical framework for feature interactions, using quadratic networks for modular addition to study how matching Fourier components combine across hidden units. An exact factorization separates interaction power into mode magnitude, block balance, and cross-block overlap, showing how cross-block pairing changes power while preserving each block's distribution of component strengths. We derive an exact state-dependent covariance law for overlap under gradient flow and a finite-step identity for realized optimizer updates. In Adam and Muon networks, within-block concentration accounts for most observed overlap growth, while relative alignment increases consistently. Fixed-marginal interventions that disrupt pairing degrade trained predictions, while phase-only controls preserve interaction power but alter predictions, showing the complementary role of readout compatibility. Within matched training configurations, grokking runs attain higher endpoint overlap. A population mean-squared-error lower bound connects interaction power and readout capacity to reconstruction. Together, these results link where learned components are placed to the interactions they form and the computations they support.

Primary area:
interpretability and explainable AI
Keywords:
Grokking, mechanistic interpretability, representation learning, feature interactions, learning dynamics, Fourier analysis, modular arithmetic

Reviews

Among:
Shape:
1234this paper, Top 32%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.50 over 2 reviews, above 68% of the others.
Reviewer_1D1VOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

The paper studies how two-layer quadratic neural networks coordinate Fourier features across separate left and right input weight blocks when learning modular addition.

Critical strengths

Clear conceptual distinction between tied and untied input streams Well-controlled surgical interventions Good empirical control by comparing grokking vs non-grokking using same hyper parameters

Critical weaknesses

While the central question (how features from distinct input streams co-locate on the same hidden units to form multiplicative interactions) is natural and well-posed, the theoretical contributions do not really explain grokking, and the empirical decomposition inadvertently reveals that the paper's headline concept ("cross-block relative alignment") plays only a minor ~10% role compared to standard within-block neuron sparsity (~70%).

Questions for authors

Table 4(c) shows that between grokking and non-grokking runs in matched configurations, endpoint overlap comes only in minor part from cross-block relative alignment. Doesn't this imply that cross-block alignment occurs robustly even in non-grokking (memorizing) runs, and that the true bottleneck for grokking is within-block neuron sparsity/concentration rather than cross-block alignment?

3 comments
AuthorsOpenReview ↗

Response to Weakness 1 and Question 1

Dear Reviewer 1D1V,

We sincerely thank you for the encouraging assessment, in particular for recognizing the conceptual distinction between tied and untied input streams, the well-controlled surgical interventions, and the empirical control within matched configurations. Your question led to one of the main additions of the revision and sharpened its central message. We respond to the weakness and the question together.

W1. The theoretical contributions do not really explain grokking, and the empirical decomposition shows that cross-block relative alignment plays only a minor (about 10%) role compared with within-block concentration (about 70%).

Q1. Does Table 4(c) imply that cross-block alignment occurs robustly even in non-grokking runs, and that the true bottleneck for grokking is within-block concentration rather than cross-block alignment?

Response to W1 and Q1.

Yes, your reading is correct, and the revision turns it into a quantitative result. The central quantity of the paper is cross-block overlap A=C⋅SA=C\cdot S, the product of concentration CC and alignment SS. We prove that alignment has bounded range: S≤1S\le1 and S≈1/2S\approx1/2 at initialization, so it can add at most about log⁡2\log 2 to log-overlap, whereas concentration can add far more (Appendix B.8, Proposition 14). Overlap growth must therefore be carried mainly by concentration: the bound gives a concentration share of at least 72%, against 77% measured. Alignment rises in every run, grokking or not, and on average uses up most of its available range, which is why it does not discriminate between outcomes. In the continued runs, alignment is already close to its ceiling before generalization, and over 90% of the subsequent growth in overlap comes from concentration (Appendix D.9, Table 10).

Pairing and concentration are two parts of one mechanism.

  • Pairing is the precondition for concentration to matter. With identical concentrations in both blocks, interaction power ranges from its maximum to zero depending on pairing (Proposition 13). In trained networks, once pairing is destroyed, no readout within the same norm budget recovers performance, whereas an equal-distance phase perturbation is fully repaired (Appendix F.4).
  • Pairing sets the rate of concentration. In the projected gradient flow, concentration changes at a rate equal to a positively weighted sum of pairwise concordances between the two allocations (Appendix B.9, Proposition 17).
  • Tested by intervention. In new small-scale runs we change only the initial pairing, at identical amplitude distributions and identical initial concentration. The alignment difference disappears quickly, as your reading of Table 4(c) suggests (Table 5(c) in the revision): alignment is close to one in all three arms within the first tenth of training. The initial lead is transmitted to concentration, which grows in all three arms, fastest under aligned and slowest under anti-aligned pairing in every matched comparison. The wait for generalization follows the same order in 57 of 60. For the network type and loss of the main experiments, it changes by up to an order of magnitude while the overlap at generalization is unchanged (Appendix G.1, Figure 14). Alignment thus matters as the starting condition of concentration and not as a separate bottleneck, which reconciles its small share in the decomposition with its role in the mechanism.

Theory and grokking. In a fixed-energy projected gradient flow, overlap grows monotonically and full overlap is the only attracting state, for every convex loss of the task coefficient (Propositions 16, 18, and 22). From a spread-out state this growth is slow, with a logarithmic rate of order 1/K1/K, which gives a structural reason for the wait before generalization (Proposition 21). The margin decomposition shows that generalization then requires the task margin to overtake the residual and gives a closed-form waiting time (Appendix B.11). Both terms are measured in the new runs: between memorization and generalization, the task margin grows and the residual advantage shrinks in every run with a measurable wait (Table 16). In the continued runs, the remaining wait decreases with overlap within each optimizer (Appendix D.9).

In summary, your observation is now a proven result of the paper, pairing and concentration appear as two parts of one mechanism supported by an intervention, and the theory gives a structural reason for the wait before generalization. We hope that these additions address your concerns, and we would be grateful if you would take them into account in your final assessment. We are glad to provide any further clarification.

Best regards,

The Authors

Reviewer_1D1VOpenReview ↗

Increasing score

Good answers. I am willing to increase my score

Dear Reviewer 1D1V,

Thank you very much for your reply and for your willingness to raise your score. We are glad that our responses addressed your concerns. Your question about the roles of alignment and concentration led to one of the main improvements of the revision, and we are sincerely grateful for it.

If it is not too much trouble, we would be very grateful if you could update the score in the system at your convenience. We fully understand if this takes some time, and we truly appreciate your support. We remain happy to answer any further questions.

Best regards,

The Authors

Reviewer_mnB8OpenReview ↗
Rating
2
Confidence
2
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies small networks with one hidden layer and a quadratic activation, trained on modular addition, where each of the two inputs has its own weights. It introduces a measure, called cross-block overlap, of whether the same Fourier frequency appears on the same hidden neurons in the weights of both inputs. It writes the useful signal of each frequency as a product of three factors, one of which is the overlap, and derives equations for how the overlap changes during training. Experiments with Adam and Muon follow the overlap during training, perturb trained networks, relate the overlap to grokking, and compare a lower bound on the loss with the measured loss.

Critical strengths

The proposed measure is clearly defined, and the experiments are carried out carefully, over many runs and with uncertainty estimates.

Critical weaknesses
  1. My main concern is novelty. For this architecture, the structure of the trained network is already known: each neuron uses a single frequency, the same for both inputs and for the output (Gromov, 2023; Morwani et al., 2024). In the paper's terms, this is a state of high overlap. The paper does not discuss how its results relate to this prior work. What could be new is a description of how training reaches this state, but the theoretical results do not provide one. The factorization and the equations for the change in overlap hold by definition. They do not say whether the overlap will grow, how fast it will grow, or when the network will generalize. The lower bound on the loss follows from a standard inequality and lies well below the measured loss. The theory also assumes plain gradient descent, whereas the experiments use Adam and Muon.

  2. My second concern is the connection to grokking. The main question about grokking is why generalization comes long after memorization, and the paper does not address it. The evidence offered is a correlation: networks that generalized have higher overlap at the end of training. Training stops at 10,000 epochs, and the successful runs generalize at roughly 8,000–10,000 epochs, so the other runs may simply be slower. The difference in overlap may then reflect how far training has progressed rather than a different outcome. That the relevant structure builds up gradually before generalization is already known (Nanda et al., 2023). The interventions do not clearly show that overlap matters: changing only the phases leaves the overlap unchanged, yet harms accuracy about as much as destroying the overlap does. Finally, the study is limited to a single architecture and loss function. The effect it studies does not arise when both inputs share weights, as in the transformers usually used to study grokking. The paper does not discuss its limitations.

Questions for authors
  1. What do the results add to the known structure of the trained network (Gromov, 2023; Morwani et al., 2024)? Can the theory predict whether and how fast the overlap grows, or when the network generalizes?

  2. Do the non-grokking runs generalize if trained longer? Does the difference in overlap remain when runs are compared at the same stage of training rather than at the same epoch?

  3. Changing only the phases leaves the overlap unchanged but harms accuracy about as much as destroying the overlap. What shows that the damage comes from the loss of overlap rather than from disturbing a trained network in general?

4 comments
AuthorsOpenReview ↗

Response to Weakness 1

Dear Reviewer mnB8,

We sincerely thank you for the careful reading and the constructive questions, and for noting that the proposed measure is clearly defined and the experiments are carefully carried out. Your comments led to additions that, we believe, substantially strengthen the paper: guarantees for the growth of overlap, an intervention on the wait for generalization, a control that separates overlap from a generic perturbation, and the continued training that you asked about. We respond to each point below.

W1. Novelty: (a) relation to the known solution structure; (b) whether the theory predicts that overlap grows, how fast, and when the network generalizes; (c) the loss bound; (d) gradient descent versus Adam and Muon.

Response to W1.

(a) Relation to the known solution structure. We thank the reviewer for this observation. Gromov (2023) and Morwani et al. (2024) characterize the endpoint of training: each neuron carries one frequency, shared by both inputs and the output, which is a high-overlap state in our terms. The revised Related Work states this relation and takes the endpoint as given. Our contribution is complementary and concerns the path to it. The revision makes this path explicit through the quantity that must grow (cross-block overlap), its exact law of motion under any optimizer, and guarantees for when and how fast it grows. It also identifies information that the endpoint description does not contain, namely pairing. With identical strength distributions and concentrations in both blocks, interaction power still ranges from its maximum to zero depending on pairing (Appendix B.8, Proposition 13).

(b) What the theory predicts. Beyond the identities, the revision adds five results (Appendix B.9).

  • Monotone growth. In a fixed-energy, single-mode projected gradient flow with an optimal ridge readout, overlap increases along every trajectory. The rate vanishes only when both allocations are uniform (Proposition 16).
  • Limit states. Full overlap is the only asymptotically stable state of this flow (Proposition 18).
  • Upper bound on the hitting time. Once the two blocks share a strictly dominant neuron, overlap converges to its maximum, and the time to reach any target overlap has an explicit upper bound determined by that state (Theorem 19). Numerical integration confirms the bounds for eight initial geometries (Table 2).
  • Lower bound on the hitting time. From a spread-out state the logarithmic growth rate of overlap is of order 1/K1/K, and no target overlap is reached before a time of order K/log⁡KK/\log K (Proposition 21).
  • Any optimizer. When the task-driven growth term exceeds a bound on the remaining gradient terms, overlap grows at a guaranteed rate. The discrete form of this statement uses only realized updates and therefore holds for Adam and Muon (Proposition 15).

Within this flow, the questions raised in the review have definite answers: overlap grows from every non-uniform state, full overlap is the only attracting state, and the time to reach a target overlap is bounded from below and, once the blocks share a dominant neuron, from above.

(c) The loss bound. We now compute the exact constrained optimum over the full feature spectrum under the same per-frequency readout budgets, and decompose the fixed-readout loss exactly into the power bound, a full-spectrum excess, and a readout gap (Appendix B.10). At trained checkpoints, the decomposition attributes the slack to two identified sources, the rest of the feature spectrum and the readout (Appendix F.4). The bound becomes the binding constraint precisely when overlap is removed: after anti-aligned pairing it accounts for 97% of the best achievable loss. The bound is thus the operative constraint in exactly the regime that the paper studies, and the gap elsewhere is explained and not left open.

(d) Optimizers. The finite-step identities are exact for any optimizer, because they use only the realized update. On realized Adam and Muon updates, the squared-radius identity reproduces the measured change to numerical precision (Appendix E.3). The conditional growth bound in (b) covers Adam and Muon in its discrete form, so the theory is not limited to plain gradient descent.

AuthorsOpenReview ↗

Response to Weakness 2 (Part 1)

W2. Connection to grokking: (a) the delay between memorization and generalization; (b) the 10k horizon; (c) Nanda et al. (2023); (d) phase controls; (e) a single architecture and loss, and shared weights; (f) limitations.

Response to W2, points (a) to (d).

(a) The delay between memorization and generalization. The revision addresses this question directly, with a two-part account and an intervention that tests it.

  • Overlap growth is necessarily slow at first. Reconstruction requires interaction power (Corollary 3), and overlap supplies it at fixed radius and balance. In the projected flow, the logarithmic growth rate of overlap from a spread-out state is of order 1/K1/K, and no target overlap is reached before a time of order K/log⁡KK/\log K (Proposition 21). A wait is therefore built into the starting point: from the spread-out allocations of an untrained network, overlap can only grow slowly, and faster growth becomes possible only as the allocations concentrate. The continued runs show this pattern: the growth rate of overlap increases with overlap across the twelve runs (rank correlation 0.98; Appendix D.9).
  • Generalization requires the task margin to overtake the residual. The held-out margin equals the task margin minus the residual advantage of the strongest competing class. Under exponential residual cleanup the waiting time has a closed form (Proposition 24). The residual carried by weight modes that the data gradient does not sustain decays exponentially, at a rate set by the margin of regularization over data-driven growth (Appendix B.11). This agrees with the matched regularization comparison of the original submission, in which every regularized run generalizes and no unregularized run does within the training horizon.
  • An intervention on the wait. In new small-scale runs we change only how the two blocks are paired at initialization: aligned, random, or anti-aligned. The data split, the amplitude distributions of both blocks, the readout, and the initial concentration are identical across the three. Networks generalize in the order aligned, random, anti-aligned in 57 of 60 matched comparisons. For the network type and loss of the main experiments, pairing alone changes the median wait by up to an order of magnitude, while the overlap at which generalization occurs is the same in all three. Since nothing but pairing differs, the change in the wait is caused by pairing, and the unchanged overlap at generalization indicates that pairing acts through the time needed to reach the required overlap (Appendix G.1, Figure 14 and Table 15).
  • Both margin terms are measured. Between memorization and generalization, the task margin grows and the residual advantage shrinks in every run with a measurable wait, and the two contribute about equally to closing the gap (Table 16 and Figure 15).

Across settings whose generalization times differ by two orders of magnitude, the overlap at generalization stays in a narrow band in 12 of 13 settings (Table 14). In these settings, the wait is therefore the time needed to reach a required level of overlap. Together with the intervention, this goes beyond the endpoint correlation of the original submission and gives a concrete answer to why generalization comes late: the required overlap is reached slowly, and the residual has to fall as well.

(b) The 10k horizon. We continued a stratified sample of these runs. Please see our response to Q2.

(c) Nanda et al. (2023). Our event ordering (memorization, overlap doubling, concentration, generalization) is consistent with the gradual formation of structure before generalization reported by Nanda et al., and the revised Section 5.2 states this. Our work refines this picture for networks with independent input blocks: it identifies a specific measurable quantity (cross-block overlap and its two factors), gives its exact dynamics, and links it quantitatively to interaction power and readout capacity.

(d) Phase controls. Readout refitting separates the two perturbations. Please see our response to Q3.

AuthorsOpenReview ↗

Response to Weakness 2 (Part 2) and Question 1

W2 (continued): (e) a single architecture and loss, and shared weights; (f) limitations.

Response to W2, points (e) and (f).

(e) Architecture, loss, and shared weights. The revision shows that the framework extends beyond the original setting, in theory (Appendices B.12 and G) and in experiments (Appendix G.1).

  • Geometry. The factorization, the role of pairing, and the bounded range of alignment (Propositions 1, 13, and 14) are statements about two allocation vectors. They hold for every separable bilinear interaction and involve neither the loss nor the optimizer.
  • Dynamics. The covariance law uses the loss only through its gradient, and the finite-step identity uses the optimizer only through its realized update. Both hold without change for cross-entropy and for Adam or Muon. The flow results hold for every convex loss of the task coefficient, including the logistic loss (Proposition 22).
  • Power requirement. The loss bound uses only the linearity of the readout. It holds for arbitrary hidden features, of any depth and activation, and it has a cross-entropy counterpart: without task-aligned power, the cross-entropy cannot fall below log⁡M\log M for any readout (Proposition 25).
  • Shared weights. With tied input blocks the framework specializes exactly: alignment is identically one, and overlap equals the concentration of the single shared allocation. This connects to the shared-weight analysis of He et al. (2026).
  • Experiments. New small-scale runs cover quadratic and bilinear features, MSE and cross-entropy, and independent and tied blocks. Every run that generalizes follows the same order: memorization, overlap doubling, generalization. Tied networks generalize earlier than their independent counterparts in every matched pair with a generalization event. Under both losses and both interaction types, refitting the readout repairs a phase perturbation and cannot repair anti-aligned pairing, after which no readout within the same norm budget does better than the uniform prediction (Tables 14 and 17).
  • The framework also accounts for a case without generalization. Quadratic networks with independent blocks trained with cross-entropy reach high overlap without reaching 95% held-out accuracy within the horizon. The margin terms identify the reason: the residual advantage stays above the task margin in all ten runs (Figure 15). Overlap supplies the task term of the margin, and generalization also requires the residual term to fall, which is the second part of our account of the delay. The paper reports this result and lists it among the limitations.

(f) Limitations. The revised Section 7 adds a limitations paragraph covering the experimental setting, the independent-block premise, the dependence of outcome labels on the horizon, the dominant role of concentration, the fact that high overlap does not suffice for generalization, and the conditions of each guarantee.

Q1. What do the results add to the known structure of the trained network? Can the theory predict whether and how fast the overlap grows, or when the network generalizes?

Response to Q1. Relative to the known structure, the revision adds four points.

  • Pairing is an independent degree of freedom that marginal statistics do not determine (Proposition 13).
  • Alignment has bounded range, which forces concentration to carry overlap growth and matches the measurements quantitatively (Proposition 14).
  • In the projected flow, overlap grows monotonically, full overlap is the only attractor, and the hitting time is bounded from above and from below (Propositions 16, 18, and 21, Theorem 19).
  • The constrained optimal loss decomposes exactly, which locates the slack of the bound (Appendix B.10).

On prediction: for the projected flow, the theory establishes that overlap grows and how fast, and that pairing sets the rate of concentration (Proposition 17). The new intervention confirms the predicted order in training: the early growth of concentration is fastest under aligned pairing and slowest under anti-aligned pairing in every matched comparison, and the wait for generalization follows the same order in 57 of 60 (Appendix G.1). For the time of generalization, Appendix B.11 gives a closed form under exponential residual cleanup, and the continued runs are consistent with its logarithmic dependence on overlap. In short, the revision adds to the known endpoint an account of how training reaches it, with guarantees in the flow and interventional evidence in training.

AuthorsOpenReview ↗

Response to Questions 2 and 3

Q2. Do the non-grokking runs generalize if trained longer? Does the difference in overlap remain when runs are compared at the same stage of training rather than at the same epoch?

Response to Q2.

Yes, the runs generalize under further training, as you anticipated. We continued a stratified sample of twelve runs that had not generalized by the original horizon, six per optimizer and covering the full range of accuracies, from their saved model and optimizer states (Appendix D.9, Figure 11 and Table 10). Seven of the twelve reach 95% held-out accuracy within the additional training, and two more come close. For the full set of such runs this corresponds to an estimated 58% (exact 95% interval 28% to 85%). The non-grokking label of the original submission therefore marks training progress within the window and not a failure to generalize, and the revision states this.

At the same stage, overlap is similar. At the moment of generalization, runs of the same optimizer have similar overlap, although they reach it at different times. Differences at a fixed epoch therefore reflect the stage of training, and overlap tracks progress toward generalization: within each optimizer, a run with larger overlap at the original horizon generalizes earlier in every case. Over 90% of this late growth in overlap comes from concentration. The revision accordingly presents the original groups as a division by training progress, which supports the central claim that overlap measures progress toward generalization.

Q3. Changing only the phases leaves the overlap unchanged but harms accuracy about as much as destroying the overlap. What shows that the damage comes from the loss of overlap rather than from disturbing a trained network in general?

Response to Q3. We separate the two perturbations directly by refitting the readout (Appendix F.4). The revised Section 6 notes that phase-only controls reduce fixed-readout accuracy as much as anti-alignment does, and introduces the refit for this reason. For the 22 high-accuracy checkpoints, we apply a phase perturbation that is exactly as large as anti-aligned pairing in every Fourier mode but preserves all amplitudes, hence overlap and interaction power. We then let the readout adapt within the same norm budgets.

  • Refit on training data only. After the phase perturbation, the refit restores held-out accuracy at every checkpoint (median 100%). After anti-aligned pairing, the same refit leaves a median of 0.007%.
  • Best possible readout. Even the optimal readout cannot recover from anti-aligned pairing: its loss is 0.990, against 0.215 after the phase perturbation, and it is higher at every checkpoint.

Two perturbations of equal size thus have different consequences. The phase perturbation creates a readout mismatch that a new readout removes. The pairing perturbation removes the representation itself: 97% of its best achievable loss is given by the power bound, that is, by the loss of overlap. The damage therefore comes from overlap and not from a generic disturbance. The same separation holds in the new small-scale runs for bilinear features and for cross-entropy (Appendix G.1, Table 17). We believe this resolves the ambiguity in the original interventions that you identified.

In summary, the revision adds a theory of when and how fast overlap grows, interventional evidence that pairing controls the wait for generalization, a control that separates overlap from a generic perturbation, and the continued training that you asked about. We hope that these additions address your concerns, and we would be grateful if you would take them into account in your final assessment. We are glad to provide any further clarification.

Best regards,

The Authors

Comments on the paper

1 comment
AuthorsOpenReview ↗

General Response to the Area Chair

Dear Area Chair, Senior Area Chair, and Program Chairs,

We sincerely thank you and both reviewers for your time and care. We are encouraged that both reviewers recognize the core elements of our contribution:

  • The central question. Reviewer 1D1V regards our question, how features from separate input streams come to share hidden units and form multiplicative interactions, as natural and well-posed.
  • The central quantity. Reviewer mnB8 finds the proposed measure, cross-block overlap, clearly defined.
  • The conceptual distinction. Reviewer 1D1V values the clear distinction between tied and untied input streams, on which our analysis of relative allocation rests.
  • The interventions and experiments. Reviewer 1D1V highlights the well-controlled interventions and the comparison of grokking and non-grokking runs under identical hyperparameters, and Reviewer mnB8 notes that the experiments are carried out carefully, across many runs and with uncertainty estimates. Reviewer 1D1V reports no substantive concerns about soundness and indicates having learned something interesting or useful from the paper.

Why the question matters and what is new. Knowing which features a network represents does not tell us how it computes with them: features must interact, and for this they must meet on the same units. Our paper studies this step directly. It shows that the pairing of components across two input blocks is a degree of freedom of its own, captures it in one measurable quantity with an exact law of motion under any optimizer, and introduces an intervention that changes pairing alone. Earlier work characterizes the solution that training reaches; our contribution concerns how training gets there.

The reviewers also raised constructive questions, and we believe the revision resolves each of them. It adds thirteen formal results with full proofs (Appendices B.8 to B.12) and three new experiments. All changes are marked in blue.

  1. From identities to guarantees (Reviewer mnB8, W1 and Q1). The concern was that the theory states identities without predicting growth. We now prove when overlap grows and how fast in a projected gradient flow, with upper and lower bounds on the time to reach a target overlap, a conditional counterpart for Adam and Muon, and extensions to every convex loss and to cross-entropy (Appendices B.9 and B.12).
  2. The roles of concentration and alignment (Reviewer 1D1V, W1 and Q1). Reviewer 1D1V's observation is now a proven result: alignment has bounded range, so concentration must carry the growth of overlap, with a predicted share of at least 72% against 77% measured. Pairing remains essential as the starting condition of concentration (Appendices B.8 and G.1).
  3. Pairing versus a generic perturbation (Reviewer mnB8, Q3). A phase perturbation of the same size as anti-aligned pairing is fully repaired by refitting the readout, whereas the loss of overlap is not. This separates the effect of overlap from a generic disturbance (Appendix F.4).
  4. Longer training (Reviewer mnB8, Q2). Seven of twelve runs that had not generalized by the original horizon do so when training continues, and runs of the same optimizer generalize at similar overlap. The earlier difference therefore reflects training progress, as the reviewer anticipated, and overlap tracks that progress (Appendix D.9).
  5. The wait for generalization (Reviewer mnB8, W2; Reviewer 1D1V, W1). The concern was that the link to grokking is correlational. In new small-scale runs we change only the initial pairing of the two blocks. For the network type and loss of the main experiments, the wait for generalization changes by up to an order of magnitude while the overlap at which generalization occurs is unchanged. This is interventional evidence that pairing controls the delay (Appendix G.1).
  6. Prior work, scope, and limitations (Reviewer mnB8, W1 and W2). We state the relation to Gromov (2023) and Morwani et al. (2024), explain the slack of the loss bound exactly, identify the results that hold for any loss and optimizer, add small-scale runs with bilinear features, cross-entropy, and tied blocks, and add a limitations paragraph.
  7. Presentation (both reviewers). For readability, the main text now names the two factors of overlap, concentration and alignment (Section 3.4, Table 1), Table 3 maps every experiment to its evidence, and Appendix G states the scope of each result.

We believe the revision resolves the concerns raised and makes clear what the paper contributes: an exactly defined quantity, its law of motion, guarantees for its growth, and interventional evidence for its role in the delay. Point-by-point responses follow, and we are glad to provide any further clarification.

Best regards,

The Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.