- Rating
- 1
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
The paper investigates the emergence, growth, and concentration of massive residual-stream activations. It combines training runs of 51M–380M transformers with analyses of released Pythia checkpoints spanning 70M–6.9B parameters. Its main findings concern early, seed-dependent channel identity; collective RMS as a relevant scale for gradient attenuation; weight decay’s effect on global activation turnover; and increasing concentration onto a small channel cohort. Validation includes checkpoint analyses, channel ablations, perturb-and-restore experiments, learning-rate controls, and a four-point weight-decay sweep. A balance model connects activation peaks to optimizer-mediated weight growth and decay, while survivor selection remains unresolved.
Critical strengths
-
Preserving optimizer state and batch order while changing decay, together with a constant-learning-rate control, helps establish that the global median’s decline cannot be attributed solely to the learning-rate schedule.
-
Separating raw magnitude, max-to-median ratio, and token RMS prevents misleading interpretations of activation trajectories. The matched-RMS perturbations also improve on random-channel comparisons by addressing a concrete magnitude confound. These controls make the observations in the 51M continuum regime more informative.
Critical weaknesses
-
Equation (7) is a stationarity condition for the writer-weight norm. Applying it to an activation peak requires a sufficiently stable relationship between activation magnitude and that norm. Yet Appendix A reports that the activation declines while its writer norm continues increasing: the norm rises from 10.8 to 11.5 while activation per unit norm falls from 4.2 to 3.2. The changing relationship between weights and activations is therefore central to the phenomenon, and its cause remains unexplained. Furthermore, the drive exponent is inferred from the measured peak-scaling relationship; this is model calibration, not an independent prediction of that relationship. Section 4.2 also explicitly limits the clean turnover intervention to the global median, with the individual massive activation plateauing under constant learning rate. These results support an effect of decay on activation scale, but leave a substantial gap in the proposed explanation of the massive activation’s peak.
-
Appendix E estimates a preconditioned gradient from fresh gradient statistics on WikiText-2 at fixed Pythia checkpoints. This is not the historical AdamW update, whose first and second moments depend on the preceding training trajectory. The paper does not establish that this proxy preserves the sign or magnitude of the actual maintaining force, yet compares it directly with the decay coefficient and concludes that AdamW preconditioning sustains the cohort. That inference exceeds the evidence. The overlapping forces for winners and collapsers further leave the proposed “competitive consolidation” without an identified selection mechanism. The larger-model observations cannot resolve these gaps because the reported perturb-and-restore experiments are confined to the 51M continuum regime. Validating these claims requires substantive evidence about actual optimizer dynamics.
Questions for authors
-
What evidence validates the WikiText-2-based maintaining-force proxy against actual AdamW updates during training, particularly its sign and its position relative to the decay threshold? Are such comparisons available for the models whose optimizer states were saved?
-
At the measured activation peak, does the actual preconditioned maintaining force satisfy Equation (7), and does the writer-weight norm simultaneously become stationary? How is the claimed activation-peak condition reconciled with the continued growth of that norm reported in Appendix A?
-
Appendix E describes the sweep as repeating a near-peak continuation protocol, whereas Figure 1b reports peak locations ranging from approximately 4k to 17k steps. Were these runs trained from initialization at each decay coefficient, or continued from checkpoints? Please specify the fork steps, observation windows, and whether the plotted steps are absolute or relative. This directly affects the interpretation of the scaling evidence.
Review responses
thank you for your detailed analysis; you helped us identify several issues in the original analysis that upon correction made the paper much stronger. We have also corrected some relevant measurements and repeated some experiments where needed:
-eq. 7 and continued writer-norm growth: Your observation exposed a measurement error in our original analysis. The reported values were taken at layer 2, but the aggregate writer norm included blocks that could not contribute to that layer. We have corrected this by measuring at the MA's own layer (5), using the saved AdamW moments (Table 2). With this correction, the windowed maintaining force predicts the observed writer-norm growth to within 0.025 and crosses λ near step 21k, where the writer norm peaks. The MA itself peaks at 20k. In our second seed, both peak at 19k. We also find that m/∥w∥ remains stable within 2% after step 14k. These results show that Eq. 7 does locate the writer-norm peak. We have withdrawn the earlier claim that the decline is not due to weight shrinkage and corrected Appendix A accordingly.
-calibrated exponent: We agree that the exponent q should not have been presented as an independent prediction. It is now explicitly identified as a calibrated parameter. The independent evidence for the proposed mechanism comes from the direct optimizer-state measurements described above.
-the proxy: We tested the proposed proxy against the actual optimizer dynamics, as you suggested. It failed to predict both the sign and magnitude of the observed changes, so we have withdrawn it (General Response, Point 1). Direct measurements also show that winning, collapsing, and randomly selected channels grow at nearly the same rate. This changes our interpretation: preconditioning appears to maintain the scale of the channel population rather than selectively supporting a particular cohort.
-constant learning rate: As discussed in our response to reviewer yjgi, the revised experiments distinguish the behavior of the MA from the global activation scale. Weight decay is necessary for the observed MA turnover, although under a constant learning rate the MA approximately plateaus rather than declining.
-sweep protocol: We identified an issue in our description of the original sweep. The runs were forked at step 4k, not near the peak as originally stated, which censored the result for λ=0.4. We have now repeated the full sweep for all four values of λ, starting from step 2k (Fig. 1b, Fig. 5b, and Appendix G). The corrected results show a strong negative correlation between λ and peak timing (−0.99 for the MA and −1.00 for the writer norm). The peak writer norm scales approximately as λ−0.60, reasonably close to the predicted λ−1/2. The MA magnitude follows a steeper scaling of approximately λ−0.76, consistent with Eq. 7 constraining the writer norm ∥w∥, rather than the activation magnitude m directly.
perturb-and-restore at 51M: We agree that this remains a limitation and have made it explicit in §4.5. Magnitude-matched controls require other channels of comparable magnitude, which are available only in the continuum regime. We therefore do not claim that this experiment establishes the same behavior across larger models.
Again, thank you for taking the time to give such detailed feedback, it was very helpful in improving the results