acceptodds
Under review as a conference paper at ICLR 2027

Dynamical Anatomy of Emergent Abilities in Large Language Models

Abstract

In large language models (LLMs), the phenomenon of emergent abilities, characterized by abrupt performance improvements at specific scales or training stages, has received considerable attention. Notably, the learning process is observed to transition through three distinct phases: an initial non-linguistic output phase, a formally and syntactically consistent but semantically hollow phase, and finally, a meaningfully valid output phase. However, the underlying properties of internal information processing within these three phases remain unknown. In particular, it remains to be clarified what qualitative shifts in the model's internal processes underpin the acquisition of linguistic proficiency and higher-order intelligence. In this study, we focus on the high-dimensional internal dynamics of the LLM and analyze its dynamical properties across the three identified phases. We employ power spectral density (PSD) to investigate the long-range temporal correlations within the model's internal dynamics during text processing. Our analysis reveals that the transition from the first to the second phase is characterized by the emergence of a power-law spectrum with an exponent of , which reflects the characteristic strength of long-range correlations typically observed in natural language corpora. In the third phase, measuring across individual hidden dimensions shows a heavy-tailed broadening of the exponent distribution while maintaining a constant layer-mean, marked by the emergence of distinct outlier units. Further inspection of these outlier units, specifically those exhibiting weakened temporal correlations, reveals that they selectively respond to functional vocabulary like punctuations. Together, these observations suggest that analyzing processing dynamics along the temporal axis offers a valuable new lens for understanding how language capabilities emerge in LLMs.

Primary area:
interpretability and explainable AI
Keywords:
Large Language Models, Emergent Abilities, Power Spectral Density, Internal Dynamics, Mechanistic Interpretability

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_dVUXOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper investigates the underlying mechanisms of "emergent abilities" in LLMs from a dynamical systems and multivariate time-series perspective. The study characterizes the long-range temporal correlations within the network activations via the spectral exponent β\beta (P(f)∝f−βP(f) \propto f^{-\beta}). The primary empirical contributions are: Self-organization to natural language scaling and Dynamic diversification and heavy-tailed divergence.

Critical strengths

Anchoring the macroscopic progression of pre-training to cumulative compute (FLOPs, estimated via C≈6NDC \approx 6ND) and tracking internal temporal dynamics offers a fresh, quantitative angle on the emergence phenomenon. This provides an intrinsic lens that does not rely exclusively on discrete downstream benchmark metrics.

The observed dynamical transition points correspond remarkably well with the phase shifts and performance plateaus documented in prior literature.

Critical weaknesses

The paper defines the boundaries between Phase A, B, and C using specific FLOPs cutoffs (2.5×10192.5 \times 10^{19} and 1.0×10211.0 \times 10^{21} FLOPs) calibrated against MMLU 5-shot accuracy. However, in practice, the emergence trajectory is profoundly modulated by external data variables, including task diversity, sample complexity/difficulty, token distribution, and vocabulary granularity. A model trained on syntactically repetitive or code-heavy corpora may cross the grammatical threshold (Phase A →\rightarrow B) or reasoning threshold (Phase B →\rightarrow C) at significantly different compute budgets. Because FLOPs solely accounts for non-embedding parameter count and total tokens, treating these numerical FLOPs boundaries as universal thresholds oversimplifies the role of data composition.

While the paper establishes a compelling correlation between the kurtosis spike (appearance of outlier units) and benchmark leaps, the mechanistic link remains largely observational.

Questions for authors

Recent studies have explored emergent capabilities by quantifying internal representational structures, information propagation, and circuit formation (for example [1]). How do the authors reconcile their macroscopic, compute-oriented spectral dynamics with these fine-grained internal mechanistic formulations of semantic emergence?

I strongly encourage the authors to expand their literature review to include this body of work on internal/semantic emergence and discuss how spectral dynamics can be integrated with these mechanistic approaches.

[1]. Quantifying Semantic Emergence in Language Models

Reviewer_kSR8OpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies the the internal dynamics of language models over the course of pretraining, focusing in particular on the nature of temporal dynamics in token sequences. They define three different stages in pretraining, during which the model transitions from producing invalid outputs, to achieving chance accuracy on MMLU tasks, to finally achieving greater than change benchmark performance. The authors study the dynamics of sequences of hidden state activations using the power spectral density to analyze temporal correlations, and perform their analysis on a variety of models and architectures. After the first phase shift, the authors show that power law decay is seen in later layers (fifth layer and beyond) with exponent converging to β≈0.5\beta \approx 0.5, but not in earlier layers. They claim this relates to the kind of long-range correlations that one sees in natural language. They then show that the second phase shift causes the distributions of β\beta exponents for certain dimensions to become less concentrated around ≈0.5\approx 0.5 and they further show that the kurtosis increases during this shift. Finally, they show that some units with particularly low exponents selectively respond to certain tokens with specific linguistic features, such as punctuation, pronouns, and structural tokens.

Critical strengths

This paper has a few different strengths. First, the authors have broad model coverage in their analysis. They consider many models covering Transformer-based, Mamba-based, and hybrid architectures. They additionally consider several pretraining checkpoints for most models.

Second, the authors' general focus on the temporal structure of activations across sequences, as well as their usage of PSD in particular, seem novel and underexplored compared to the more standard focus on representations, probing, circuits, and individual features.

Finally, the authors provide interesting supplementary experiments in the Appendix which strengthen the case for their paper. Specifically, I found the domain robustness experiments to be useful in showing that some findings are not simply limited to MMLU. Additionally, I was intrigued by the attention matrix experiments in the appendix, particularly the experiment about generating temporal dynamics, and thought they should be explored further.

Critical weaknesses

I found a few aspects of the paper to be weak.

First, the analysis is fairly limited, since only OLMo has checkpoints across all three phases. Additionally, it is unclear if Llama, Qwen, and Falcon are adding anything to Figure 3c since there is only one checkpoint available for each.

Second, there are some general methodological weaknesses:

  • The authors only use MMLU accuracy when describing the relationship between performance and phases. They do provide some domain robustness tasks, but those have more to do with the β\beta distributions for different corpora. I think we should also see performance on other benchmarks through all the phases to determine the true relationship between the performance-based and FLOPs-based notion of phase.
  • The experiments in Figure 4 seem interesting but preliminary. In particular, the specific tokens seem a bit ad-hoc for Unit 2250.
  • I also have some issues with the PSD experiments. By using a window size of 256 and additionally filtering out ultra-low frequencies, the analysis does not really deal with long-range correlations at all. The authors should also provide additional analysis of temporal correlations, for example they should also compute the autocorrelations in token activations for different token lags.

Third, although the authors mention this limitation directly, I think causal analysis would improve the paper significantly. For example, the authors can perform controlled interventions where they suppress high β\beta dimensions to see if that suppression hurts performance specifically on tasks that require information from farther behind in context.

Finally, there are some clarity issues with some of the figures in the paper. Figure 2 is a bit confusing and difficult to understand. Additionally, it is unclear if the benchmark score curve is adding anything to Figure 3b and it may be better to just have markers for phase boundaries, because it appears busy as-is.

Questions for authors

In the paper, you seem to define the phases in a few different ways, with FLOPs and based on performance. Additionally, some of the OLMo checkpoints do not seem to be at the correct FLOPs point based on the 6ND approximation. Can you clarify how you are deciding the phases and how the checkpoints were plotted?

In the paper, you mention that generating 4,096 tokens for analysis. Is that only for OLMo or for all models? If for all models, how do you deal with the Pythia context length of 2,048?

Reviewer_ECHvOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
3(+1)
Presentation
3
Soundness
2
Read review
Paper summary

The paper asks whether the three performance phases of LLM pre-training (invalid → well-formed but chance-level → above-chance MMLU answers; Nakagi et al., 2025) have signatures in hidden-state dynamics. Each unit's activation over 4,096 self-generated tokens is treated as a time series, a spectral exponent β is fitted to its Welch PSD, and the per-layer mean and kurtosis of β are tracked across checkpoints of OLMo-2-7B (13), Pythia-6.9B, SmolLM3-3B, Tri-7B and Zamba-7B, plus final Llama-3.1-8B, Qwen2.5-7B and Falcon-Mamba-7B. Layer-mean β rises from roughly zero to about 0.5 at the first transition; after the second, the mean stays near 0.5 while intra-layer kurtosis exceeds 10 (Figs. 2–3). A Layer-20 case study reports token-selective low-β units, e.g. Unit 2250 (β = 0.03) with response ratios above 0.30 to ' able', ' didn' and ' doesn', versus below 0.10 for every other unit (Fig. 4).

Critical strengths

S1. Treating token-indexed hidden-state trajectories as multivariate time series and tracking per-unit spectral exponents across public checkpoints (Sec. 3, Fig. 1b–c) complements static probing and SAE analyses. Coverage spans eight models, including attention-free Falcon-Mamba and hybrid Zamba (Table A.1), with 13 OLMo checkpoints (Table A.2), and layer-resolved heatmaps aligned with MMLU (Figs. 2c, 3b) aid inspection.

S2. The appendix adds useful checks: fit residuals (RMSE mostly below 0.1, Fig. E.10c), an encoding-regime replication on four natural-text domains plus a random-token control (β ≈ 0; App. E.4), a sub-layer decomposition (App. E.5) and a random-attention null (App. H.2). The early rise of mean β replicates in OLMo and Pythia (Fig. 2d).

Critical weaknesses

W1. Long-range power-law structure is not established: 256-token Welch segments with f > 0.025 (App. E.1) limit fits to periods of 2–40 tokens (≈1.3 decades), where a low RMSE cannot separate a power law from short-memory spectra, and no shuffled or phase-randomised surrogates are shown, so the comparison with corpus-level correlations (Sec. 5) is not like-for-like. App. E.4's β ≈ 0 for random tokens also sits uneasily with App. H's claim that attention itself produces β ≈ 0.5, and a random-Q/K null cannot rule out low-pass filtering by recency-biased attention (p. 35). Suggestion: fit β over ≥ 2 decades (longer segments or DFA), compare with surrogates and AR(1) fits, and add structured attention nulls.

W2. The kurtosis–emergence link is confounded with training progress: Pythia-6.9B never leaves chance on MMLU (Fig. 2a) yet its kurtosis exceeds 10 (Fig. 3c), contradicting "consistent behavior across all tested architectures" (p. 7) and the "benchmark-free diagnostic" (p. 8); in OLMo, kurtosis rises within Phase B and jumps most after MMLU leaves chance (Fig. 3b). Several models contribute single points, no variability is reported, and phase labels are inconsistent (Pythia-299B exceeds the 10^21 bound under Eq. 1; Zamba-185B is "Phase C" despite apparently chance-level MMLU). Suggestion: add bootstrap CIs, relate β statistics to continuous metrics and other benchmarks, and test predictive value beyond training tokens.

W3. The closest prior work is not discussed — outlier dimensions (Kovaleva et al., 2021; Puccetti et al., 2022), emergent large-magnitude features (Dettmers et al., 2022), massive activations (Sun et al., 2024) and token-detector neurons (Gurnee et al., 2024) — although the "outlier units", Unit 2250's spikes to ≈20–25 (Fig. 4a) and Unit 3130's punctuation responses (Fig. F.22) may be instances of them. Suggestion: measure the overlap between β-outliers and magnitude outliers, recompute kurtosis without them, and reposition the contribution.

W4. The specialisation claims rest on three hand-picked units in one layer of one checkpoint, with the abstract's "punctuations" resting on one unit (3130), and the 64-unit ablation promised in App. C never appears. Details conflict: "three standard deviations (mean + 4σ)" (p. 7); 4,096 generated tokens "within the context limits" (p. 5) versus Pythia's 2,048 (Table A.1); duplicated Pythia/SmolLM checkpoint names (Tables A.4–A.5). Suggestion: correlate a selectivity index with β across all units against controls, report the ablation, and fix these errors.

Questions for authors

Q1. Under Eq. (1) and Table A.1's N, OLMo's Phase-A checkpoints (1B and 3B tokens) give about 4.1×10^19 and 1.2×10^20 FLOPs, above the 2.5×10^19 bound (Sec. 4.1), and the Phase-B checkpoints at 34B–126B tokens about 1.4–5.2×10^21, above 10^21; the labels would instead be consistent with C ≈ N·D. Which formula produced the thresholds and plotted x-axes, and which criterion decides the phase when FLOPs and accuracy disagree (Pythia-299B, Zamba-185B)?

Q2. Do the rise in mean β and the later kurtosis rise survive comparison with time-shuffled and phase-randomised surrogates, and estimation over periods up to about 1,000 tokens?

Q3. Pooling all checkpoints, does β-kurtosis explain MMLU accuracy after controlling for training tokens or loss, and does it track benchmarks with different emergence points?

Q4. What fraction of the units driving the kurtosis rise are activation-magnitude outliers, and is a unit's β predicted by the occurrence spectrum of its preferred tokens?

Q5. What did the 64-unit ablations of App. C do to MMLU accuracy, perplexity and downstream β, and how were Pythia's generations handled beyond 2,048 tokens?

Comments on the paper

1 comment

Thank you for the thorough response. The DFA and extended PSD analyses with AR(1) and shuffled controls address my main methodological concern, the FLOPs labels are corrected, and the narrowed framing of the kurtosis and outlier claims (a necessary rather than sufficient marker, and a temporal perspective on known outlier dimensions) now matches the evidence. I am therefore raising my rating from 2 to 3, and I encourage the authors to bring the promised main-text updates, confidence intervals and outlier-literature discussion into the final version. No further response is needed.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.