acceptodds
Under review as a conference paper at ICLR 2027

The Maternal Phase of Language Model Training

Abstract

What does initialization contribute to the training of a language model, and when and how does training take over from it? Early training is dominated by the initial weights, yet runs that differ only in their seed end with nearly the same loss, so control must pass from what a network is given to what it learns, at a point that has not been located. Embryos make the same handover at the maternal-to-zygotic transition, from materials deposited in the egg to the embryo's own genome, and embryologists locate it by following gene expression through the stages of development. We do the same with a developmental atlas of Pythia that follows the expression of every MLP unit across fifty runs at five sizes and four larger models, and we retrain small models to intervene where the atlas can only correlate. We find that (i) training begins with a maternal phase, in which expression is a copy of initialization set by each unit's projection onto the mean input of its layer; (ii) the phase ends not because the units' weights change, since they keep the initial pattern, but because the mean input rotates, the deepest layers first; (iii) its duration is set by the accumulated update measured against the initial weight scale; and (iv) little of it remains afterwards, and a small network damaged once training is under way restores its loss and its repertoire of unit behaviours but reassigns roles among its units. More broadly, the atlas suggests that the toolkit of developmental biology, from staging to transplantation and lesion, can be turned on training, provided that what a network inherits is first separated from what it learns.

Primary area:
interpretability and explainable AI
Keywords:
training dynamics, initialization, interpretability, language models

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_93X3OpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper defines "expression" as the average absolute GELU activation of the MLP unit over 46 fixed sentences, defines "initial memory" as the correlation between the current expression of each unit and its initial expression, and characterizes the decay time by the number of steps where the correlation is less than 0.10. The authors establish a Pythia map covering five scales, ten training sessions for each scale, and four larger models. They then use a re-trained small model to conduct experiments on transplanting initial working points, adjusting scale/learning rate, and resetting units. The main observation is that the projection of the early expression onto the average input across layers is closely related to the input weights; as the average input direction rotates, the initial correlation of the expression first decays from the deeper layers, while the projection of the weights in the original direction remains preserved; rescaling the cumulative learning rate can better align the controlled memory curve. After training with 14M damage, the training loss and the set of defined expression trajectories are close to recovery, but the unit identities change; the expression trajectory set of 70M does not recover in the same way.

Critical strengths
  1. Decompose a broad initialization phenomenon into directly verifiable unit-level relationships. At step 128 of 14M, the current weights still have an initial projection correlation of .99 when read against the initial mean, while the initial weights drop to .49 when read against the current mean, approaching an expression memory of .38; the hierarchical expression memories of four scales are approximately .98 in correlation with the cosine of the mean direction, and the inter-layer correlations within the same step are also very high. The decomposition, random projection identity, and actual expression readings are interconnected, providing a more detailed description than "small weight changes". The paper also clearly distinguishes the arithmetic verification of the standard random projection identity from the empirical findings for nonlinear expressions, and does not directly attribute the recovery of the constrained readings to the causal proof of the training dynamics.

  2. Self-checking of the instrument and comparison baseline substantially improves the credibility of the conclusion. The maps of five scales with ten seeds each are complementary to the initial value twin, orthogonal perturbation, repeated execution, different initial values, and intra-layer neighbor baselines. Especially in Appendix D: Removing step 0 still leaves almost the same early checkpoints as it was; after disrupting the later unit identities, the fate classification still has the explanatory power for the initial expression. This is a specific and operational failure diagnosis. The author also reports the stability of half sentences and all sentences, and acknowledges that this cannot replace the independent corpus verification; these controls make the applicable boundaries of the observational results clearer.

Critical weaknesses

The curve collapse supports an approximate scaling relationship, but is not sufficient to support the claim that "the learning rate and initial scale only act through this ratio". The controlled experiments only changed the multiplier of the same learning rate schedule, and 70M all stopped at the end of the same linear warmup. At this point, the cumulative learning rate was approximately proportional to "multiplier × number of steps squared", so these experiments can support the time rescaling of the tested schedule family, but cannot distinguish whether different schedule histories are equivalent when the cumulative value is the same. On the other hand, Appendix B.8 has reported that the minimum value is relatively flat, and the range of 14M contains k=1. Fixing k=1 at 14M/70M still reduces the divergence to 18.4%/17.4% of the original value, which is close to the optimal 17%/13%. Using 1.2/1.4 as an exact new index requires caution; currently, there is a lack of a clear interpolation grid for the collapsed objective function, the common intervals of each condition, and the weighting method. This doubt does not deny the controlled observation of the delayed decay of the initial scale increase, but limits "the sole clock" and the intensity of the index increment.

What remains unchanged/recovered is the measured loss and expression statistics, but it cannot be concluded that the predictive function remains unchanged or that the general functional role has been restored. §3 infers that "the content calculated by the network" has not changed based on the training loss change not exceeding 0.008 nats after transplantation; however, Appendix B.3 moves the input working point without corresponding replacement of output weights, which is not maintaining the function's neuron re-labeling. Similarly, §5.2's Hungarian matching uses a single average expression trajectory across training steps, discarding the responses to which sentences, tokens, and their symbols; a low matching distance can explain that this trajectory set is close to twin, but is not sufficient to prove that the same function is replaced by different units. The authors have limited 70M, the number of seeds, and the time window, and these limitations should be retained; the "recovery of calculation/role redistribution" should be clearly constrained to the defined loss and expression trajectory results.

Use the existing curves for limited validation. Please fully define the curve collapse loss, interpolation, common U_k interval, stage weights, and bootstrap objects, and compare the results of k=1 and the fitted k under a fixed common interval; can the fitted index be evaluated using the leave-one initialization method to improve the curve not involved in the fitting? If the improvement is still stable, especially at 70M, it can strengthen the index conclusion; if the main benefit comes from changing the comparison interval or fitting degrees of freedom, the result should be positioned as an approximate collapse compatible with k≈1. Do you agree that "only through the cumulative learning rate" is applicable to the current isomorphic schedule family?

The measurement objects of the conclusion. "Not changing the content calculated by the network" and "role redistribution" are operational names for the training loss and the expression trajectory represented by Equation 1, or do they contain a stronger assertion of the response to each input-output/function identity? If the former, the summary-style generalization, §3, and §5.2's wording can be tightened uniformly; if the cache can support the latter, provide corresponding output differences or sentence-by-sentence response results, and compare them with the existing twin/replicated run baseline. This answer will change the credibility of the evidence I give for the functional explanation, but does not require the addition of large-scale task benchmarks.

Questions for authors

see weeknesses

7 comments

We thank the reviewer for a precise review. We ran the proposed checks on the existing curves, and new experiments where the existing curves could not decide. The four paragraphs under Critical Weaknesses (W1 to W4, in the reviewer's order) form two pairs, each a criticism and the requests that follow from it. We answer W1 together with W3, then W2 together with W4, and number the quotes from each paragraph in the order we answer them (for example W1.2).

W1 and W3. The collapse and the exponent

W1.1 "The curve collapse supports an approximate scaling relationship, but is not sufficient to support the claim that 'the learning rate and initial scale only act through this ratio'. …"

The reviewer is right about what the collapse supports. The measurements in the submission stand: in the schedules we tested, a larger initial scale and a lower learning rate delay the decay, and the curves collapse approximately. What was not aligned was the wording, since "only act through this ratio" reads as a statement about every schedule, while the experiments covered one family. To resolve this we ran the checks the reviewer proposes and new experiments under other schedule shapes. They show where the relationship holds, and the revision states it there: an approximate collapse within the family of warmup schedules, with no exponent reported as a measured constant.

W1.2 "The controlled experiments only changed the multiplier of the same learning rate schedule … cannot distinguish whether different schedule histories are equivalent when the cumulative value is the same."

W3.1 "Do you agree that 'only through the cumulative learning rate' is applicable to the current isomorphic schedule family?"

We narrow the claim to the family of warmup schedules. The submitted runs tested "only through the cumulative learning rate" within one family. The runs changed the multiplier of a single schedule, and inside its warmup the accumulated learning rate is proportional to multiplier × step², so they cannot tell the accumulated learning rate from other functionals of that schedule. We therefore trained networks under four further schedule shapes, at 14M and at 70M, from three initializations each: a constant learning rate from the first step, a 1,000-step warmup followed by a constant rate, a two-level schedule (a quarter of the peak until step 2,000, then the peak), and a high-then-low schedule (the peak until step 500, then a tenth of it).

Memory at equal accumulated learning rate (14M, mean of three initializations)

ScheduleLearning rate at the start28321285001,0002,000
Warmup over 4,000 steps (submitted)rises from zero0.470.390.340.270.200.130.08
Warmup over 1,000 steps, then constantrises from zero0.460.370.320.250.180.120.07
Quarter rate for 2,000 steps, then peaka quarter of the peak0.430.360.310.200.150.110.06
Constantthe peak0.430.350.320.120.090.070.05
Peak for 500 steps, then a tenththe peak0.430.350.320.120.08——

The accumulated learning rate is counted in steps at the peak rate.

Where the memory falls to 0.10 (initializations 0 / 1 / 2)

ScheduleLearning rate at the start14M: step14M: accumulated learning rate70M: step70M: accumulated learning rate
Warmup over 4,000 steps (submitted)rises from zero2,772 / 2,925 / 4,922961 / 1,069 / 2,8881,406 / 1,584 / 1,458247 / 314 / 266
Warmup over 1,000 steps, then constantrises from zero1,376 / 1,542 / 2,977876 / 1,042 / 2,477696 / 837 / 623242 / 351 / 194
Quarter rate for 2,000 steps, then peaka quarter of the peak2,043 / 2,306 / 3,899543 / 806 / 2,399752 / 1,387 / 922188 / 347 / 231
Constantthe peak122 / 150 / 714122 / 150 / 71415 / 14 / 1515 / 14 / 15
Peak for 500 steps, then a tenththe peak122 / 150 / 2,151122 / 150 / 66515 / 14 / 1515 / 14 / 15

Early in the decay the schedules agree: at accumulated learning rates of 2, 8 and 32 the five 14M curves are within 0.05 of each other, at steps up to 63 times apart. Later they separate according to the rate at which they start. The two warmup schedules stay within 0.03 of each other at every value of the first table, and a schedule that starts at the peak rate reaches a memory of 0.10 at an accumulated learning rate 4 to 8 times smaller than the submitted warmup. The accumulated learning rate thus removes a large part of the difference between schedules (the largest gap between two schedules is a factor of 7 to 23 in steps and a factor of 4 to 8 in accumulated learning rate), and not all of it.

At 70M the 0.10 crossing gives the same picture, and the separation is larger. The two warmup schedules reach a memory of 0.10 at accumulated learning rates within a factor of 1.4 of each other, at steps a factor of 1.9 to 2.3 apart. A schedule that starts at the peak rate is already below 0.10 at step 16; it reaches 0.10 at an accumulated learning rate 17 to 22 times smaller than the submitted warmup (94 to 113 times in steps).

So the answer to the reviewer's question is yes. The revision narrows the claim to the family of warmup schedules, where the collapse is approximate. This does not affect the conclusions of the paper. The paper describes the early phase that warmup produces (Section 7), and every Pythia run and every run of the paper uses a warmup. Within warmup schedules the new runs support the collapse at both sizes, for two warmup lengths. The other three findings do not depend on the collapse.

W1.3 "… there is a lack of a clear interpolation grid for the collapsed objective function, the common intervals of each condition, and the weighting method."

W3.2 "Please fully define the curve collapse loss, interpolation, common U_k interval, stage weights, and bootstrap objects …"

For one initialization there are five conditions cc (unmodified; block weights scaled by αc∈{0.5,2}\alpha_c\in\{0.5, 2\}; learning rate scaled by λc∈{0.5,2}\lambda_c\in\{0.5, 2\}). Every logged step s≥8s \ge 8 of condition cc is placed at

uc(s)=log⁡λc+log⁡∑t<smt−klog⁡αc,u_c(s) = \log \lambda_c + \log \sum_{t<s} m_t - k \log \alpha_c ,

where mtm_t is the schedule's multiplier of the peak learning rate at step tt; this is log⁡Uk\log U_k. Each memory curve is linearly interpolated in uu onto 50 points spaced uniformly over the interval on which all five curves are defined, [max⁡cmin⁡suc, min⁡cmax⁡suc][\max_c \min_s u_c,\ \min_c \max_s u_c]. The spread D(k)D(k) is the variance across the five conditions at each grid point, averaged over the grid and then over initializations. Stages therefore carry no weights of their own: the grid weighs log⁡Uk\log U_k uniformly. The "raw" spread is the same quantity with u=log⁡su = \log s. The reported exponent minimizes DD over k∈{0,0.05,…,2}k \in \{0, 0.05, \dots, 2\}. The bootstrap resamples initializations with replacement (1,000 draws) and refits; with three initializations it is essentially their range, as the appendix says. In the submission this interval depends on kk. The comparison below fixes it, using the intersection of these intervals over every kk of the grid.

W1.4 "Fixing k=1 at 14M/70M still reduces the divergence to 18.4%/17.4% of the original value, which is close to the optimal 17%/13%."

W3.3 "… compare the results of k=1 and the fitted k under a fixed common interval …"

k=1k = 1 against the best kk

ObjectiveQuantity14M70M
SubmittedBest kk1.201.40
Spread at the best kk, relative to the step axis0.170.13
Spread at k=1k = 1, relative to the step axis0.180.17
Interval fixedBest kk1.201.40
Spread at k=1k = 1, relative to the best kk1.101.43

14M has three initializations and 70M has five. With one interval fixed for every kk, the best kk does not change, and k=1k = 1 gives 1.10 times the smallest spread at 14M and 1.43 times at 70M. At 70M the gain over k=1k = 1 therefore does not come from the moving interval.

W3.4 "… can the fitted index be evaluated using the leave-one initialization method to improve the curve not involved in the fitting?"

Yes. We fitted kk on all initializations but one (submitted objective) and scored the one left out. The fitted kk does better than k=1k = 1 on the held-out curve in two of three cases at 14M and in all five at 70M. Relative to k=1k = 1, the held-out spread is 0.85 to 1.44 at 14M and 0.72 to 0.81 at 70M.

W1.5 "Using 1.2/1.4 as an exact new index requires caution …"

W3.5 "If the improvement is still stable, especially at 70M, it can strengthen the index conclusion; if the main benefit comes from changing the comparison interval or fitting degrees of freedom, the result should be positioned as an approximate collapse compatible with k≈1."

The dependence on the initial scale is stronger than linear at 70M and compatible with linear at 14M. The reviewer's test separates the two sizes: at 70M the improvement is stable under both checks, and at 14M it is not. The numerical value of the exponent depends on how the collapse is scored. A statistic that needs no interval, the step at which each curve passes 0.8, 0.6, 0.4, 0.3 and 0.2 (with kk from 0 to 3), gives a best kk of 1.8 at 14M and 2.0 at 70M, against 1.2 and 1.4 by the spread. The revision therefore reports the direction at 70M and does not present 1.2 and 1.4 as measured constants. The exponent is a detail of the third finding: the abstract does not state it, and the other three findings do not depend on it.

W1.6 "This doubt does not deny the controlled observation of the delayed decay of the initial scale increase, but limits 'the sole clock' and the intensity of the index increment."

We share this reading. The controlled observation stands, and the revision narrows two statements. The abstract and Section 4.2 will state the result as an approximate collapse on accumulated learning rate over initial scale within the family of warmup schedules, which replaces "the learning rate enters only through its sum" in Section 4.2 and the matching row of Table 1. Section 4.2 will describe the dependence on the initial scale as compatible with k≈1k \approx 1 at 14M and stronger at 70M, which replaces the measured exponent where the paper lists it as a result (Section 6, Section 7, Appendix A and Table 1). Appendix B.8 will carry the full definition above, the fixed-interval and held-out comparisons, the interval-free statistic, and the schedule experiment. What a unit inherits and how the inherited state is lost rest on neither statement.

W2 and W4. What is restored, the loss or the function

W2.1 "What remains unchanged/recovered is the measured loss and expression statistics, but it cannot be concluded that the predictive function remains unchanged or that the general functional role has been restored. …"

The reviewer is right about what the two experiments measure. The measurements in the submission stand: after the transplant the loss is unchanged, and after the damage the loss recovers and the repertoire of mean-expression trajectories returns. What was not aligned was the wording, since the two phrases that the reviewer quotes read as statements about the function, while the measurements concern the loss and Equation 1. To resolve this we reran both experiments with the weights saved and read the predictions and the responses of single units. The revision states each result at the level that was measured, and the contribution of the paper is not affected.

W4.1 "'Not changing the content calculated by the network' and 'role redistribution' are operational names for the training loss and the expression trajectory represented by Equation 1, or do they contain a stronger assertion of the response to each input-output/function identity?"

In the submission they were operational names for the training loss and for the mean-expression trajectories of Equation 1.

Reviewer_JuuYOpenReview ↗
Rating
3
Confidence
2
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

The paper looks at an interesting aspect of learning within neural network-based models. The paper looks at how initialization of neural networks transitions into their converged weights. Specifically, the paper looks at how initialization influences early layers and when learning really takes over. The paper finds that the initialized weights slowly fade during training. The key finding is that the initial influence on a neuron's activity fades primarily because its inputs change, even while its own weights initially retain much of their original structure. The initialization determines the early neuron behavior, as the neuron's initial expression correlates strongly with the projection of its weight vector onto the layer's mean input. The similarity between initial and current mean inputs shows that the initial patterns are still retrained throughout training. Large initial weights delay the transition during training, while larger learning rates accelerate it. The authors, through reinitalizing half of the MLP units during training, find that it temporarily increases the loss, but the small models recover while redistributing neuron roles. This shows that networks can recover after substantial damage.

Critical strengths
  • The paper provides an interesting observation on how training dynamics can be observed with respect to the initialization.
  • The empirical evidence is clearly demostrated on how the initazation memory closely tracks the similarity between initial and current mean inputs.
  • The experimentation setup with 50 runs across 5 Pythia mdels sizes and larger models gives an extensive empirical evidence across the training trajectories.
Critical weaknesses
  • The analysis relies on the mean absolute activation of neurons. It would be better to elaborate on why the central choice of metric is the mean absolute activation of neurons. It compresses lots of information, atleast, it would also be better to look at variance as well to give a more broader picture. Looking at the variance or token-conditioned activation statistics could reveal whether neurons retain meaningful functional behavior despite declining initialization-memory correlations.
  • The transition threshold being 0.10 is unclear to me. Is it even necessary to have this threshold? I think it would be interesting to look at whether the quantitative clearance times and comparisons across model sizes remain robust under alternative thresholds.
  • And most importantly, the authors argue that input rotation causes neurons to lose their initialization-dependent expression patterns. But their intervention experiment that holds the mean input fixed also changes the direction used to measure the neuron expression. This makes the results interesting in terms of correlation but lacks the causation aspect. Although the authors do mention it as a limitation, I feel it is important because the rotation mechanism is one of the paper's central claims.

Sidenote:

  • Maybe the controlled retraining experiments may not reproduce the original Pythia training dynamics. Since it is not exactly the same setup as Pythia. To measure the difference, were you able to run the original Pythia experiment from scratch just once, so that it can give a rough idea of whether there would be variation?
Questions for authors
  1. Given that the mean-input intervention also changes the expression measurement, what evidence supports input rotation as the causal mechanism behind initialization-memory clearance?

  2. How closely do the controlled retraining experiments reproduce the original Pythia training dynamics, and could differences in training configurations affect the main conclusions?

6 comments

We thank the reviewer for the positive assessment and for four concrete requests. Each is answered below with a new analysis or a new experiment.

W1. Why mean absolute activation; variance and token-conditioned statistics

W1.1 "It would be better to elaborate on why the central choice of metric is the mean absolute activation of neurons."

We chose the mean absolute activation because, at initialization, it is governed by one identifiable quantity, the unit's mean pre-activation w⋅μw\cdot\mu. That is what lets the paper say what a unit inherits. The fading of the memory is not specific to this choice, as shown below.

W1.2 "It compresses lots of information, atleast, it would also be better to look at variance as well to give a more broader picture."

We tested additional readout functions (the statistics the reviewer names) and a larger corpus (Pile validation text, 7,352 tokens, against the 515 tokens of the 46 sentences used in the paper), on the released checkpoints of five model sizes with three seeds each. The table below gives the results.

Memory of initialization under four readout functions: steps 128 → 2,000 → 143,000 (end of training)

Readout function14M31M70M160M410M
Mean absolute activation (used in the paper)0.35 → 0.11 → 0.000.27 → 0.05 → 0.020.17 → 0.03 → 0.010.15 → 0.02 → 0.000.12 → 0.02 → 0.00
Standard deviation of the activation across tokens0.48 → 0.12 → −0.070.40 → −0.02 → −0.080.25 → −0.08 → −0.090.21 → −0.08 → −0.070.19 → −0.06 → −0.04
Fraction of tokens on which the unit is active0.42 → 0.19 → 0.060.33 → 0.12 → 0.060.23 → 0.09 → 0.070.19 → 0.05 → 0.040.14 → 0.04 → 0.03
Token profile of each unit0.66 → 0.18 → 0.020.61 → 0.15 → 0.020.56 → 0.12 → 0.020.48 → 0.09 → 0.010.43 → 0.07 → 0.01

Values are means of three seeds on Pile validation text.

Under every readout function, most of the memory is gone by step 2,000 at every model size. The standard deviation of the activation across tokens loses its memory over the same steps as the mean. The corpus does not change this: on Pile validation text the memory of the paper's readout stays within 0.12 of its value on the 46 sentences, at every step of all 15 runs.

W1.3 "Looking at the variance or token-conditioned activation statistics could reveal whether neurons retain meaningful functional behavior despite declining initialization-memory correlations."

The token profile answers this question directly. The first three rows of the table describe each unit by one number, an average over all tokens. The last row keeps the tokens apart. For each unit we correlate its activations on the 7,352 tokens at step 0 with its activations on the same tokens at step s, and we average this correlation over units. It is high when a unit still responds to the same tokens as at initialization. Units keep part of their token-level behavior through the transition and none at the end: the correlation is 0.14 to 0.29 at each run's clearance step, 0.09 to 0.18 at twice that step, and 0.01 to 0.03 at the end of training.

Early in training the token profile keeps more memory than the other three readouts: at step 128 of 14M it is 0.66, where the mean absolute activation is 0.35. This is what a rotation of the mean input predicts. The mean input is shared by all tokens, so its rotation moves a unit's response to all tokens together. This changes how active the unit is overall before it changes which tokens the unit responds to.

The revision reports these readouts in a new appendix figure and gives the reason for the choice of readout in Section 2.

W2. The 0.10 threshold

W2.1 "The transition threshold being 0.10 is unclear to me. Is it even necessary to have this threshold?"

The value 0.10 is a convention: it marks the point at which little of the initial correlation is left. Any threshold from 0.05 to 0.40 gives the same order of the model sizes (W2.2). The threshold summarizes a curve by one number. The central claim about timing (Section 4.2) is tested without a threshold, by comparing whole curves. The revision says in Section 2 that the threshold is a summary.

W2.2 "I think it would be interesting to look at whether the quantitative clearance times and comparisons across model sizes remain robust under alternative thresholds."

The order of the model sizes does not depend on the threshold. We recomputed the clearance step of all 50 runs at six thresholds.

Mean clearance step at six thresholds (ten runs per model)

Model0.050.100.150.200.300.40
14M3,7762,0871,4821,062443147
31M1,8991,4091,07274720288
70M1,3888996764789137
160M9246363861742924
410M757451178432419
Pairs in which the larger model clears first, of 1,000953980971936983996

The last row counts pairs of runs from two different model sizes.

At every threshold the mean clearance step falls monotonically from 14M to 410M, and every run crosses every threshold. The clearance step itself moves with the threshold, as it must for a decaying curve, so the paper compares runs at one threshold. The revision adds the table to Appendix E.

W3. The causal status of the rotation

W3.1 "But their intervention experiment that holds the mean input fixed also changes the direction used to measure the neuron expression. This makes the results interesting in terms of correlation but lacks the causation aspect."

The reviewer is right about the submitted test. Replacing the mean input at measurement reads the units against the old direction, so it is a decomposition of the readout and the revision presents it as one.

W3.2 "Given that the mean-input intervention also changes the expression measurement, what evidence supports input rotation as the causal mechanism behind initialization-memory clearance?"

The evidence is an intervention in training that removes every change of a layer's input and leaves the measurement untouched.

We ran two experiments on a target layer. In the first, the input of the target layer's MLP is frozen ("Input frozen" in the table). Pythia's blocks are parallel-residual: the MLP of a block reads the stream that enters the block, not the output of that block's attention. The input of the target layer's MLP therefore depends only on the embedding, on the blocks below the target layer, and on the LayerNorm in front of that MLP. We froze these three, so the input is the same function of the text at every step. The target layer and everything above it trained as usual. In the second experiment we froze only the input weights and biases of the target layer's MLP ("Weights frozen") and let everything else train, so the input changes and the weights do not. In both, the units are read in the ordinary way. The target layer is the last layer or the third layer of 14M and of 70M, which have six layers each, and each case is run from three initializations.

ModelTarget layerInput frozenWeights frozenStep 128Step 1,000Step 2,000Last step
14Mlast--0.20–0.330.04–0.250.05–0.17−0.03–0.15
last✓-0.97–0.980.59–0.660.46–0.510.32–0.37
last-✓0.17–0.340.06–0.250.09–0.200.05–0.24
third--0.63–0.760.33–0.530.21–0.390.00–0.11
third✓-0.990.90–0.920.78–0.820.46–0.54
third-✓0.63–0.770.37–0.560.29–0.480.14–0.33
70Mlast--0.16–0.220.05–0.110.03–0.060.03–0.07
last✓-0.85–0.860.33–0.440.26–0.350.23–0.31
last-✓0.14–0.200.13–0.190.10–0.180.12–0.15
third--0.55–0.580.26–0.37−0.01–0.16−0.06–0.11
third✓-0.970.71–0.740.33–0.360.26–0.32
third-✓0.56–0.600.36–0.450.22–0.300.12–0.20

Ranges over three seeds. ✓ marks what is frozen. The last step is 4,000 at 14M and 3,000 at 70M. "Weights frozen" refers to the input weights and biases of the layer's own MLP.

Three results hold in all cases. (1) With the input frozen, the target layer holds 0.85 to 0.99 at step 128, where the unmodified layer holds 0.16 to 0.76, and it stays above the unmodified layer by at least 0.15 at every later step. (2) With the weights frozen, the target layer loses its memory early as the unmodified layer does: the two differ by at most 0.04 through step 128 and by at most 0.06 through step 512. (3) With the weights frozen, the memory of the target layer stays within 0.14 of the cosine between its initial and its current mean input at every logged step (0.04 on average). This is the value that Section 4.1 predicts, with nothing fitted.

The change of the target layer's input is therefore the cause of the early loss of memory, and the change of its weights is not. With the input frozen, the memory still declines, but more slowly, because the layer's own weights change. This is a second route by which memory is lost. The paper reports it for late training (Figure 2d), and the revision states both routes in finding (ii).

W4. Do the retrained runs reproduce Pythia?

W4.1 "To measure the difference, were you able to run the original Pythia experiment from scratch just once, so that it can give a rough idea of whether there would be variation?"

Yes. We retrained 14M once from its released step-0 weights in Pythia's own configuration: batches of 1,024 sequences taken in order from Pythia's preshuffled stream, the 1,430-step linear warmup, and the cosine schedule of the released run. We ran the first 4,000 steps, which contain the transition, and compared the rerun with the released checkpoints, with the same instrument.

Our rerun in Pythia's configuration against the released run (14M)

MeasureRunStep 16Step 64Step 256Step 1,000Step 2,000Step 3,000Step 4,000
Memory of initializationPythia's run0.9850.5160.2330.1960.0800.0400.020
Our rerun0.9840.5160.2300.1890.1250.1030.087
Range of Pythia's ten runs0.983–0.9900.397–0.5520.233–0.3680.128–0.2800.029–0.175−0.029–0.107−0.045–0.066
Correlation of unit changesOur rerun against Pythia's run1.0001.0000.9910.9590.9280.8990.872
LossPythia's run10.9110.277.695.244.334.103.99
Our rerun10.9110.277.645.174.284.053.96

Pythia's run is the public 14M run, and our rerun is our retraining in the same configuration. The range is taken over the ten public 14M runs, which differ in their seed. The correlation of unit changes compares, across units, the change in expression since step 0 in our rerun and in Pythia's run. It is 1 when every unit changes in the same way. The loss is the held-out loss, in nats.

The rerun reproduces the released run through step 1,000, by which four fifths of the memory is gone. To that step the two memory curves differ by at most 0.02, and the change of each unit's expression since step 0 agrees between the two runs (correlation 0.959 or more). After step 1,000 the two executions separate slowly. The rerun keeps 0.05 to 0.07 more memory than the released run; it is inside the range of the ten released 14M runs at steps 2,000 and 3,000 and 0.02 above it at step 4,000. The unit-level correlation is still 0.87 at step 4,000, and the held-out losses stay within 0.09 nats of each other.

The released run crosses the 0.10 threshold at step 1,775 and the rerun at step 3,218. Read on the checkpoint grid of Appendix B.1, the ten released runs cross between steps 1,226 and 3,221 and the rerun at step 3,147, at the late end of their range. On this flat tail of the curve a difference of 0.05 in memory moves the crossing by about a thousand steps, and the ten released runs themselves differ by a factor of 2.6 in it.

Reviewer_b11cOpenReview ↗
Rating
3
Confidence
2
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

The paper asks how long a language model’s random initialization keeps shaping what its units do, and when training takes over. Using Pythia checkpoints across many seeds and sizes, the authors find that the earliest checkpoints are near-copies of the initialization; the “maternal phase” fades layer by layer, with the deepest layers first. They also show that the length depends on how much the weights have been updated relative to their initial scale, and a small model damaged mid-training recovers its loss while its units take on different roles.

The framing borrows heavily from embryology, in particular the maternal-to-zygotic transition.

Critical strengths

The experiments are careful and multi-seed, with sensible baselines such as repeat runs and different-seed twins.

The authors have clearly stated the negative results: the failed clamp intervention, the threshold sensitivity, and the fate-map artefact.

The authors show a novel connection between weight initialization and the early stages, taking inspiration and probing the checkpoints accordingly.

Critical weaknesses
  1. The sharp split between initialization and training is not convincing. The paper presents training as a handover, with initialization “in control” first and training afterwards. I don’t think the evidence supports a clean division, and some of it points the other way: The paper itself shows the weights still hold the initial pattern almost perfectly (0.995 at 410M) after “memory” has supposedly cleared. What changes is the input the units are read against. The “handover” may therefore reflect the chosen readout (mean absolute activation on 46 sentences) rather than a real change in what drives the network. The only causal test of the proposed mechanism, the mean-input clamp, failed. The authors acknowledge this, which leaves the central account descriptive.

  2. The paper is missing related ML work on “prenatal” or pre-data training. It leans on developmental biology but does not engage with ML work that has already brought developmental ideas into training. For example, vision models have been pretrained on retinal-wave-like or structured noise patterns before seeing real data, and this was shown to instill useful properties. Language models have similarly been pre-trained on synthetic or structured non-linguistic data. That literature shows the early phase can be designed and exploited, not just observed, and it is the more natural ML comparison. Discussing it would clarify what is new here and what the practical takeaway is.

  3. The novelty is limited.

  4. The presentation is hard to follow. The extended biological analogy often makes the ML content harder to read rather than easier. Key claims are stated in embryology terms rather than in plain optimization terms.

Questions for authors

How does this work relate to methods that deliberately shape the early phase with synthetic or wave-like pretraining data such as prenatal retinal waves?

5 comments

We thank the reviewer for the careful reading. The review raises four points: (1) whether the "handover" is more than a property of our readout and whether its mechanism is causal, (2) how the work relates to designed early phases, (3) what is new, and (4) whether the content can be stated in optimization terms. We answer all of these questions.

W1. The split between initialization and training, the readout, and the causal test

W1.1 "The sharp split between initialization and training is not convincing. … The paper itself shows the weights still hold the initial pattern almost perfectly (0.995 at 410M) after 'memory' has supposedly cleared. What changes is the input the units are read against."

We agree: early in training the weights keep the initial pattern, and what changes is the input. This is finding (ii) of the paper. The words "handover" and "control must pass" suggested a clean division that the paper does not measure. The revision replaces them with what the paper measures. Early in training, the activity of each MLP unit is a function of one number fixed by the initial weights, its mean pre-activation w⋅μw\cdot\mu. This dependence fades gradually, layer by layer: in 410M the last six layers cross the threshold by step 32, the second layer crosses it at step 4,831, and the first layer never does.

W1.2 "The 'handover' may therefore reflect the chosen readout (mean absolute activation on 46 sentences) rather than a real change in what drives the network."

The fading is not specific to the readout function or to the corpus. We tested additional readout functions (other statistics of a unit's activation) and a larger corpus (Pile validation text, 7,352 tokens, against the 515 tokens of the 46 sentences used in the paper), on the released checkpoints of five model sizes with three seeds each. The table below gives the results.

Memory of initialization under four readout functions: steps 128 → 2,000 → 143,000 (end of training)

Readout function14M31M70M160M410M
Mean absolute activation (used in the paper)0.35 → 0.11 → 0.000.27 → 0.05 → 0.020.17 → 0.03 → 0.010.15 → 0.02 → 0.000.12 → 0.02 → 0.00
Standard deviation of the activation across tokens0.48 → 0.12 → −0.070.40 → −0.02 → −0.080.25 → −0.08 → −0.090.21 → −0.08 → −0.070.19 → −0.06 → −0.04
Fraction of tokens on which the unit is active0.42 → 0.19 → 0.060.33 → 0.12 → 0.060.23 → 0.09 → 0.070.19 → 0.05 → 0.040.14 → 0.04 → 0.03
Token profile of each unit0.66 → 0.18 → 0.020.61 → 0.15 → 0.020.56 → 0.12 → 0.020.48 → 0.09 → 0.010.43 → 0.07 → 0.01

Values are means of three seeds on Pile validation text.

The first three rows describe each unit by one number, an average over all tokens. The last row keeps the tokens apart. For each unit we correlate its activations on the 7,352 tokens at step 0 with its activations on the same tokens at step s, and we average this correlation over units. It is high when a unit still responds to the same tokens as at initialization. To summarize, under every readout function, most of the memory is gone by step 2,000 at every model size. The corpus does not change this: on Pile validation text the memory of the paper's readout stays within 0.12 of its value on the 46 sentences, at every step of all 15 runs.

W1.3 "The only causal test of the proposed mechanism, the mean-input clamp, failed. The authors acknowledge this, which leaves the central account descriptive."

The change of the input is the cause of the early loss. The clamp did not hold the input fixed in the way a test of the mechanism needs. It reset the mean of each MLP input during the forward pass, but the blocks below kept training and kept changing the residual stream. When the units were then read in the ordinary way, without the clamp, they received an input whose mean had rotated. We therefore ran a different intervention. It removes every change of a layer's input, and the units are read in the ordinary way.

We ran two experiments on a target layer. In the first, the input of the target layer's MLP is frozen ("Input frozen" in the table). Pythia's blocks are parallel-residual: the MLP of a block reads the stream that enters the block, not the output of that block's attention. The input of the target layer's MLP therefore depends only on the embedding, on the blocks below the target layer, and on the LayerNorm in front of that MLP. We froze these three, so the input is the same function of the text at every step. The target layer and everything above it trained as usual. In the second experiment we froze only the input weights and biases of the target layer's MLP ("Weights frozen") and let everything else train, so the input changes and the weights do not. In both, the units are read in the ordinary way. The target layer is the last layer or the third layer of 14M and of 70M, which have six layers each, and each case is run from three initializations.

ModelTarget layerInput frozenWeights frozenStep 128Step 1,000Step 2,000Last step
14Mlast--0.20–0.330.04–0.250.05–0.17−0.03–0.15
last✓-0.97–0.980.59–0.660.46–0.510.32–0.37
last-✓0.17–0.340.06–0.250.09–0.200.05–0.24
third--0.63–0.760.33–0.530.21–0.390.00–0.11
third✓-0.990.90–0.920.78–0.820.46–0.54
third-✓0.63–0.770.37–0.560.29–0.480.14–0.33
70Mlast--0.16–0.220.05–0.110.03–0.060.03–0.07
last✓-0.85–0.860.33–0.440.26–0.350.23–0.31
last-✓0.14–0.200.13–0.190.10–0.180.12–0.15
third--0.55–0.580.26–0.37−0.01–0.16−0.06–0.11
third✓-0.970.71–0.740.33–0.360.26–0.32
third-✓0.56–0.600.36–0.450.22–0.300.12–0.20

Ranges over three seeds. ✓ marks what is frozen. The last step is 4,000 at 14M and 3,000 at 70M. "Weights frozen" refers to the input weights and biases of the layer's own MLP.

Three results hold in all cases. (1) With the input frozen, the target layer holds 0.85 to 0.99 at step 128, where the unmodified layer holds 0.16 to 0.76, and it stays above the unmodified layer by at least 0.15 at every later step. (2) With the weights frozen, the target layer loses its memory early as the unmodified layer does: the two differ by at most 0.04 through step 128 and by at most 0.06 through step 512. (3) With the weights frozen, the memory of the target layer stays within 0.14 of the cosine between its initial and its current mean input at every logged step (0.04 on average). This is the value that Section 4.1 predicts, with nothing fitted.

The change of the target layer's input is therefore the cause of the early loss of memory, and the change of its weights is not. With the input frozen, the memory still declines, but more slowly, because the layer's own weights change. This is a second route by which memory is lost. The paper reports it for late training (Figure 2d), and the revision states both routes in finding (ii).

W2. Related work on designed early phases

W2.1 "The paper is missing related ML work on 'prenatal' or pre-data training. It leans on developmental biology but does not engage with ML work that has already brought developmental ideas into training."

We agree that this is the natural ML comparison, and the revision adds it to Section 6 and Appendix A.

The literature. Four lines of work first train a model on data other than that of its task. Vision networks pre-trained on simulated or recorded retinal waves acquire Gabor-like first-layer filters and more separable translation manifolds [1, 2]. Networks pre-trained on formula-generated fractals or on structured noise learn representations that are useful for natural images [3, 4]. Training on random noise before data pre-aligns the forward weights for feedback alignment and makes later learning faster [5]. Language models pre-pretrained on music, code, or formal languages transfer to natural language [6, 7, 8, 9].

W2.2 "That literature shows the early phase can be designed and exploited, not just observed, and it is the more natural ML comparison. … How does this work relate to methods that deliberately shape the early phase with synthetic or wave-like pretraining data such as prenatal retinal waves?"

The relation. That work asks what such pre-training gives a model. We ask what a random initialization gives to single units, how long it lasts, and what removes it. The two meet in two places.

  1. Statistics against arrangement. Wu et al. [10] find that initializing with only the per-tensor mean and standard deviation of a synthetically pre-trained model recovers 39% of the benefit of natural pre-training. We find a parallel for random initialization: exchanging the inherited values among units leaves the training loss unchanged (Section 3), while the scale of the initial weights changes how long the initial state governs the units (Section 4.2).
  2. Where the effect of pre-training can persist. By our decomposition, the initial pattern of unit activity is erased within hundreds to thousands of steps as inputs rotate, while the weights still carry it (0.995 at 410M at step 512, when the memory in activity has cleared). Hu et al. [8] report a related observation for attention heads: heads acquired in pre-pretraining remain crucial for syntactic evaluations after training on natural language.

W2.3 "Discussing it would clarify what is new here and what the practical takeaway is."

The practical takeaway. Our results give this literature two predictions that it can test with its own checkpoints. After a model pre-trained in this way starts training on the data of its task, (1) the pattern that pre-training left in unit activity should fade layer by layer, the deepest layers first, and (2) at the step where unit activity has lost this pattern, the weights should still hold it. Both are what we find for random initialization (Section 4.1), and we add them to the revised paper. For any analysis of early training, the paper also gives a check: first confirm that the early checkpoints are no longer copies of initialization (Appendix D).

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.