acceptodds
Under review as a conference paper at ICLR 2027

A Small-World Perspective on Capability Emergence in Language Models

Abstract

As model scale increases, large language models may exhibit abrupt performance gains on certain tasks—gains that are difficult to predict by extrapolating performance trends observed in smaller models. This phenomenon is commonly referred to as “emergence”. To explain emergence, existing work has investigated it from perspectives such as tasks, evaluation metrics, and model scale; other work has further traced the internal changes within Transformers, for example by tracking the paths of information propagation under a specific input. However, one question remains unanswered: does the Transformer undergo a holistic qualitative change, rather than a single or local change, that gives rise to the dramatic leap in LLM performance? Inspired by the view that “Language Modeling Is Compression”, we propose that model capability should be reflected in whether highly compressed information under a given input can be rapidly reached (finding content seen during pretraining) and rapidly composed (combining multiple related pieces of content). To demonstrate this hypothesis, we borrow the famous six degrees of reachability theorem from small-world theory—namely, that any two nodes in a graph are reachable within six steps, to characterize the internal structural changes of Transformers. Specifically, we first model the Transformer as a directed graph based on the information propagation mechanism of attention; we then extend small-world theory to the directed-graph setting, define metrics that characterize the small-world properties of the Transformer, and describe the quantitative relationship between reachability/composability and these metrics; finally, across different tasks and parameter scales, we demonstrate that changes in the small-world properties of the Transformer's internal organization are highly consistent with the phenomenon of emergence. This is a creative characterization of emergence and may offer inspiration for unlocking greater potential in LLMs. For reproducibility, the code used for the experiments and analyses is available in an anonymous repository: https://anonymous.4open.science/r/iclr2027-A12C/.

Primary area:
interpretability and explainable AI
Keywords:
large language models, capability emergence, mechanistic interpretability, small-world

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_aYocOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper studies whether internal organization of models (particularly, graph metrics that attempt to measure small-world organization) relates to capability emergence across scales and tasks. They evaluate these metrics and performance across several model families, scales, and task domains, and perform additional analyses such as interventions on identified networks to attempt to validate these results.

Critical strengths

The idea of relating the performance of the system to its internal functional organization using graph metrics is interesting, and could be useful if thoroughly validated.

Critical weaknesses

Although I find the idea somewhat interesting, I think the evidence in the paper is far for sufficient for establishing a meaningful relationship between small-world organization and task performance.

  • The main results in the paper are essentially correlational analyses — just observing that the metrics change in similar ways across scales with the tasks — on 4 data points per task, without even statistically quantifying the strength of those relationships.

  • These analyses may be heavily influenced by particular choices in performing the analyses. For example, because of the clipping in the SWP metric, the scores floor at 1 - 1/sqrt(2) and many of the plotted curves are at this floor before emergence. We therefore might miss important organizational changes below this floor (in a similar way to how Schaeffer et al. argue that discrete metrics create the appearance of emergence).

  • There are many other graph organization metrics (betweenness-centrality, degree distributions, etc.) that have been related to functional organization in networks (e.g., in neuroscience). The paper does not make any effort to show that SWP is uniquely related to accuracy relative to any of the other graph metrics that might have been considered.

  • Although intervention experiments could help to resolve these issues, the experiments used at present only show that the attribution method finds contributors to the performance; it doesn’t show that the small-world organization per se is the relevant factor (as the paper itself notes “The interventions do not isolate the causal contribution of small-world topology”).

  • etc.

Questions for authors
  • Can you quantify the statistical significance of the relationship between the SWP metrics and performance, and show that floor effects, etc. aren't driving that relationship?
  • Can you provide evidence that the small-world metrics are uniquely predictive of performances changes relative to other commonly used metrics of graph structure?
  • Can you perform interventions that, again, selectively validate that small-world organization is the unique structural feature that determines performance?
Reviewer_6HUbOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper links the emergent abilities of language models with small-world theory's concept of reachability. Specifically, given a specific task, for each model authors construct a circuit depicting the interaction between attention heads and calculate its corresponding Small-World Propensity (SWP). Authors then track the SWP value and task performance across models of different sizes. Experiments show that SWP values (as indicators of model internal structures) and task performance correlates strongly, and discovered circuits causally relate to model performance. These results suggest that SWP can serve as a novel viewpoint for the cause of emergent abilities.

Critical strengths
  1. The link to small-world theory is novel and interesting. Such mechanistic analysis bring new insights into the causes behind emergent abilities.
  2. Claims are clear and experimental results support the claim.
Critical weaknesses
  1. This paper stops at an early point. The SWP--emergent abilities correlation is interesting but this explanation and viewpoint can only become solid if it can relate to existing explanations---how can it help explain, support, or disprove them. This is particularly important given that we already have many explanations for emergent abilities.
  2. The main finding is correlational observation, and causal analysis is insufficient. Specifically, if the emergent abilities are due to smaller averaged distances, will breaking the several critical paths to drop the averaged distance deteriorates a large model with emergent abilities back to its smaller counterpart with near-random performance? To what extent/portion can this theory explain emergent abilities?
  3. The case that a "post-pruning circuit" becomes denser means that the component attributions become larger. However, the attribution is based on the model's capability. This may suggests that SWP is a consequence of emergent abilities, not a cause. That is, when models start to give correct answers high logits/probabilities (i.e., capabilities emerge), attribution values subsequently become larger than those of smaller models which gave random logits/probabilities to both correct and incorrect answers.
Questions for authors
  1. How can your explanation relate to other existing explanations and findings?
  2. Is SWP actually a consequence instead of a cause? What will happen if you increase the averaged path distance (do minimal intervention to prevent from totally destroying models), will model performance degrades?
Reviewer_7Ms8OpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper asks whether capability emergence in LLMs coincides with a holistic change in the organization of internal computation. For each model and task, it builds a functional graph of attention heads:

  • Nodes are heads repeatedly ranked in the top fraction by attribution to the answer margin on a discovery split.
  • Candidate edges run from lower to higher layers and are selected using a receiver-specific edge-attribution score (Pair-EAP).
  • Edge weights are mean absolute changes in the margin under connection interventions on a held-out split.

The directed graph is projected to an undirected weighted graph. Its Small-World Propensity (SWP; Muldoon et al., 2016) is computed against lattice and random references.

The study covers 21 checkpoints from Pythia, Qwen2.5, Qwen3 and Gemma-3 and 12 task settings. The authors report that, in several highlighted adjacent-scale intervals, accuracy and SWP increase together. They also report that answer margins rise in the same intervals and that 112 of 160 graphs have SWP above a layer-order-preserving random-edge null model. In intervention experiments, attribution-selected heads affect the margin more than random heads in 54 of 72 cases. The authors state that these interventions do not isolate the causal contribution of small-world topology.

Critical strengths
  1. The paper explores a potentially useful connection between circuit analysis and network science. Summarizing task-conditioned head graphs through their global structure could complement analyses of individual circuits. The study spans four model families and multiple tasks, and several design choices are thoughtful: discovery and evaluation use separate splits, the null model preserves layer order, and the analysis applies global BH correction. Appendix G reports all 12 null-model settings, and the main text acknowledges that the findings are correlational. If the structural measure can be validated, this approach could offer a useful way to study how internal computation changes with scale.
Critical weaknesses
  1. The abstract gives a substantially stronger account of the contribution than the paper supports.

    The abstract invokes a six-degree reachability theorem, claims an extension of small-world theory to directed graphs, and promises a quantitative relationship between reachability or composability and the proposed metrics. I could not locate corresponding results in the main text. Six degrees of separation is an empirical observation associated with social networks, not a general six-step bound for arbitrary graphs; the small-world model of Watts and Strogatz (1998; DOI 10.1038/30918) does not supply such a bound either.

    The actual analysis applies the existing Small-World Propensity measure of Muldoon et al. (2016; DOI 10.1038/srep22057) to an undirected projection. Fig. 2 explicitly notes that this does not characterize directed information propagation. Reachability and composability are not measured, and the introduction more cautiously presents the statistics as candidate structural correlates. The graph is also constructed through attribution and activation interventions, rather than directly from attention patterns.

    These differences affect how readers understand the scientific contribution. An observational study of functional graph structure can be worthwhile on its own, but it needs to be presented and evaluated on that basis. Revising the abstract to match the methods and evidence would resolve the framing issue; validating the empirical interpretation remains the more substantive concern below.

  2. It is not yet clear whether SWP tracks capability or properties of the measurement procedure and architecture.

    The clipping in Eq. 8 is particularly important. When (ΔC, ΔL) = (1, 0), SWP is approximately 0.293; when both are zero, it is 1. Many plotted values lie near these boundaries, and the recurring increase of about 0.71 is their difference. This raises the possibility that some apparent structural transitions reflect saturation of the metric. It does not establish that the underlying graph statistics are uninformative, but the unclipped components are needed to understand what changes. Once SWP reaches 1, it cannot register further improvement in capability.

    The claimed alignment also rests largely on visual comparisons across only 4–8 scales per family, without a stated rule for selecting the highlighted intervals or a statistical test of the association. The figures include informative exceptions: in Fig. 3(i), accuracy rises from about 3% to 16% while SWP falls slightly; in Fig. 6(a), accuracy rises by about 23 points with no SWP change. In Fig. 3(g), the unhighlighted 0.6B→1.7B interval has larger increases in both measures than the highlighted interval. These examples make it difficult to assess how representative the proposed alignment is.

    Architecture and graph size are plausible alternative explanations. The Qwen2.5 logical-deduction settings have nearly identical SWP trajectories despite differences in difficulty and prompting, while graph size grows with the number of heads. Comparisons at fixed architecture, such as intermediate Pythia training checkpoints, would be especially informative. Randomized-model controls and tasks without an apparent emergence transition could provide further context. The continuous answer-margin analysis is useful, but does not resolve the separate issue introduced by clipping SWP; a smoothly changing margin alongside abrupt accuracy is also compatible with the metric-based explanation discussed by Schaeffer et al. (2023; arXiv:2304.15004).

    Reporting and implementation details add uncertainty. BH correction covers 272 null-model tests and 928 intervention contrasts, while 160 and 576 are reported, respectively. The missing settings need to be identified. The supplementary pseudocode also differs from the paper in edge selection, the order of averaging and taking absolute edge effects, normalization, and the definition of the incorrect-answer margin. A reproducible specification is important because these choices can affect the graph and its statistics.

    Finally, showing that attribution-selected heads affect the answer margin does not isolate the contribution of small-world topology, as the authors acknowledge. Direct structural interventions, as in related work by You et al. (2020; arXiv:2007.06559), or a validated predictive use of SWP would strengthen the scientific case. At present, I find the perspective interesting, but cannot tell what reliable conclusion about emergence follows from the measure.

Questions for authors
  1. Aligning the abstract with the contribution. Could you indicate which results support the abstract's claims about a six-degree theorem, an extension to directed small-world theory and a quantitative reachability relationship? The presented analysis appears to use existing SWP on an undirected projection. Clarifying whether these statements describe motivations or established results would help define the paper's intended contribution.

  2. Understanding the metric's boundaries. Do the frequent SWP values near 0.29 and 1 correspond to clipping of ΔC and ΔL? Could you report the unclipped components and the observed and reference graph statistics? These would help determine whether the recurring increase of about 0.71 reflects a graded structural change or a transition between saturation boundaries.

  3. Assessing the claimed alignment. How were the highlighted scale intervals selected, and does the accuracy–SWP association hold when all intervals are included? For example, Fig. 3(g) appears to show larger changes in the preceding, unhighlighted interval. A quantitative analysis that accounts for the small number of scales per family and the reported counterexamples would make the strength of the association clearer.

  4. Completeness and reproducibility. Could you identify the settings accounting for the 272-versus-160 null-model tests and the 928-versus-576 intervention contrasts, and summarize their outcomes? It would also help to specify which implementation generated the results, particularly the edge budget, order of averaging and taking absolute effects, and incorrect-answer margin. Runnable code would resolve these differences from the supplementary pseudocode.

  5. Separating capability from architecture. What evidence suggests that SWP tracks capability rather than model architecture or graph size? Intermediate checkpoints at fixed architecture would offer a focused control. Robustness to graph-construction choices and alternative network measures would further help establish whether the observed association is specific to the proposed structural account or sensitive to how the graph is measured.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.