acceptodds
Under review as a conference paper at ICLR 2027

Looking Inside LLMs: Small-World Connectivity as a Signature of Reasoning Performance

Abstract

Understanding large language model (LLM) reasoning requires looking beyond behavioral performance to examine how reasoning ability is reflected in internal organization. Inspired by neuroscience findings linking higher intelligence to stronger small-world organization in functional brain networks, we investigate small-world connectivity as a structural signature of LLM reasoning. We construct functional graphs from attention-head activation similarities and find that a higher small-world index (SWI), capturing local clustering and short global paths, consistently correlates with better fluid reasoning performance across models and training checkpoints. Since local clustering is central to small-world organization, we further examine how heads important for model performance connect within and across communities. We find that these heads tend to have a larger share of connection weight within their own communities (high core scores) and a more concentrated weight distribution across communities (low bridge scores). These observations motivate the hypothesis that high core and low bridge scores serve as structural indicators of head importance for reasoning capability. We validate this hypothesis through pruning, introducing Small-World Allocation (SWA), a hierarchical sparsity allocation method guided by these scores. Across six LLMs, SWA better preserves small-world organization and model performance than competing allocation strategies, reducing WikiText perplexity by up to 20%. Together, these findings identify small-world functional connectivity as a measurable signature of LLM reasoning performance, offering a structural perspective that complements behavioral evaluation.

Primary area:
interpretability and explainable AI
Keywords:
LLM Interpretability, small-world organization

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_KR5kOpenReview ↗
Rating
3(+1)
Confidence
4
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

The paper constructs undirected functional graphs over attention heads using similarities between question-level, mean-pooled query activations. It uses a Fisher-z sum of SVCCA correlations, retains the strongest edges at a fixed density, and measures small-world organization relative to degree-preserving randomized graphs. SWI increases alongside decreasing DRE-Bench reference-answer NLL across seven Pythia checkpoints and Qwen/Llama models. Community-based core and bridge scores are then used in Small-World Allocation (SWA), which assigns layer and head sparsity budgets for Wanda and SparseGPT. Experiments report improved WikiText perplexity, average zero-shot accuracy, and ARAOC output-shape preservation, with hierarchical allocation and graph-source ablations.

Critical strengths
  1. The paper offers a clear way to summarize relationships among attention heads using activation-based functional graphs. The association between the small-world index (SWI) and DRE-Bench performance is examined both across model families and across checkpoints of the same model.

  2. The authors also turn their observation into a pruning heuristic and report improvements over several sparsity-allocation baselines.

Critical weaknesses
  1. Limited mechanistic insight. The paper applies a small-world metric to a graph built from attention-head activation similarities, but the resulting SWI–performance correlation does not explain how this structure supports reasoning. It could also reflect broader improvements in model quality during training.

  2. Limited evidence for practical impact on reasoning. SWA improves pruning results, but the main evaluations emphasize WikiText perplexity and general zero-shot performance. The abstract-reasoning evaluation measures only output-grid shape correctness. These results provide limited evidence that the proposed principle preserves reasoning ability, and do not show that it can improve the reasoning ability of an unpruned model.

Questions for authors
  1. Does SWI predict reasoning performance beyond model size, training progress, and general language-modeling quality? Is the association specific to reasoning tasks?

  2. How does SWA compare with allocating sparsity directly according to measured attention-head importance? What additional benefit do the graph-based scores provide?

  3. Can the authors report held-out reasoning accuracy after pruning? Perplexity and output-grid shape correctness provide limited evidence that reasoning ability is preserved.

6 comments

Additional Evidence for Preserving Reasoning Performance (w2, q3)

To directly assess whether SWA preserves reasoning capabilities beyond general language modeling performance, we use ReasonEval [1] to evaluate the validity and redundancy of intermediate reasoning steps, alongside final-answer accuracy on GSM8K. As shown in Table 1, SWA consistently achieves the highest reasoning accuracy and validity, as well as the lowest redundancy, across both models and pruning methods.

Specifically, we evaluate Qwen3-8B and Meta-Llama-3-8B on GSM8K, comparing Wanda, SparseGPT, ATP, and our SWA method at 50% sparsity. Validity measures the correctness of reasoning steps, while redundancy measures unnecessary but logically valid steps. Both metrics are evaluated on shared correct responses. All comparisons use the same base model and sparsity level, controlling for model size and training progress. These results provide additional evidence that SWA helps preserve both reasoning accuracy and the quality of intermediate reasoning steps.

Table 1: Comparison of reasoning performance on GSM8K using ReasonEval at 50% sparsity. We report accuracy (ACC ↑), validity (↑), and redundancy (↓). Best results for each model and pruning method are highlighted in bold.

ModelPruning MethodACC ↑Validity ↑Redundancy ↓
Qwen3-8BWanda (50% sparsity)74.0 ± 0.700.877 ± 0.0210.214 ± 0.007
ATP75.60 ± 0.190.8787 ± 0.0110.2030 ± 0.003
Ours (SWA)77.10 ± 0.300.881 ± 0.0100.195 ± 0.009
SparseGPT (50% sparsity)73.60 ± 0.700.871 ± 0.0080.232 ± 0.008
ATP75.47 ± 1.020.8793 ± 0.0050.221 ± 0.002
Ours (SWA)79.00 ± 1.400.896 ± 0.0180.199 ± 0.014
Meta-Llama-3-8BWanda (50% sparsity)65.16 ± 0.070.8117 ± 0.0350.2607 ± 0.03
ATP66.19 ± 0.050.8234 ± 0.0300.2585 ± 0.02
Ours (SWA)70.46 ± 0.050.8550 ± 0.0420.2123 ± 0.02
SparseGPT (50% sparsity)66.02 ± 0.070.8220 ± 0.030.2764 ± 0.03
ATP67.42 ± 0.050.8300 ± 0.010.2699 ± 0.04
Ours (SWA)70.00 ± 0.060.8639 ± 0.040.2139 ± 0.04

Mechanistic insight from community interventions (w1, q1)

To provide mechanistic insight into the communities identified by our functional graph, we examine how community interventions affect the model’s representations and predictions on counting tasks. We first find that different communities induce distinct directional changes in the final hidden representation, as shown by cosine similarity analysis (Figure 4 in Appendix A.6 of the revised manuscript) and PCA visualization (Figure 5 in Appendix A.6 of the revised manuscript). We then use a counting case study to show that a specific community-associated direction is related to object count and can influence the model’s numerical predictions (Figure 6 in Appendix A.6 of the revised manuscript), providing a task-specific semantic interpretation.

To examine how each community affects the model’s final hidden representation, we first examine the six largest communities identified by applying Louvain community detection to our functional graph. We perturb each community separately by multiplying the outputs of all its attention heads by 0.5, following [8, 9]. For each question, we subtract the original representation from the intervened representation at the output of the final Transformer block and normalize the difference to obtain an intervention direction:

u_C(q)=\frac{\Delta r_C(q)}{\|\Delta r_C(q)\|_2}.$$ We compare these directions across communities for the same held-out questions. The off-diagonal cosine similarities range from −0.20 to 0.28, indicating limited alignment across communities (Figure 4 in Appendix A.6 of the revised manuscript). The corresponding PCA visualization shows largely separate groups, with particularly compact groups for C206, C207, and C208 (Figure 5 in Appendix A.6 of the revised manuscript). These results indicate that the identified communities have distinguishable effects on the model’s final hidden representation. Additionally, to assess the validity of the communities, we first examine whether each cluster captures distinct semantics in the output space. Specifically, perturbing a valid cluster should induce consistent directional changes in the outputs across different inputs, while perturbations to different clusters should produce sufficiently distinct directions. To examine these properties, we first estimate a representative semantic direction for each community using training questions. Specifically, we compute the changes in the representations caused by perturbing each community, then average the normalized changes across training questions to obtain a fixed, community-specific semantic direction: $$d_C=\operatorname{normalize}\left( \frac{1}{|\mathcal T|}\sum_{q\in\mathcal T}u_C(q) \right).$$ As a case study, we focus on community C125 and examine whether its semantic direction induces consistent changes across different counting questions. Specifically, we add a scaled version of this direction to the original representation at the same location, while keeping the input unchanged: $$r'(q)=r_0(q)+\alpha d_C.$$ To quantify the resulting changes in numerical predictions, we compute a probability-weighted count score based on the predicted probabilities of digits 1–9: $$E_\alpha(q)=\sum_{n=1}^{9}n\,P_\alpha(n\mid q).$$ By comparing changes in this score across held-out questions, we assess whether shifting representations along the same community-specific semantic direction produces consistent effects on numerical predictions. As shown in Figure 6 in Appendix A.6 of the revised manuscript, increasing the scaling coefficient along the fixed C125 semantic direction consistently increases the expected count across both input conditions.These results suggest that C125 is associated with majority-related semantic information. Overall, these findings support our view that the functional graph captures task-dependent aspects of a trained model's internal organization. The distinguishable effects of different communities on hidden representations, together with the C125 case study linking a specific semantic direction to the notion of majority, provide further evidence that the identified communities exhibit meaningful functional differences and can capture interpretable, task-specific semantics. ### Additionally reasoning accuracy (q3) To evaluate whether our SWA better preserves reasoning ability after pruning, we conduct additional experiments on held-out GSM8K and Prealgebra. As shown in Table 2, SWA consistently achieves higher reasoning accuracy than the SparseGPT baseline across all three models and both sparsity levels. For example, on Meta-Llama-3-8B at 50% sparsity, SWA improves GSM8K accuracy from 16.20% to 21.00% and Prealgebra accuracy from 11.80% to 13.00%. These results provide direct evidence that SWA better preserves reasoning performance after pruning. Table 2: Reasoning Accuracy (↑) on GSM8K and Prealgebra After Pruning at 50% and 70% Sparsity. | Model | Sparsity | GSM8K | | Prealgebra | | | :--- | :---: | :---: | :---: | :---: | :---: | | | | SparseGPT | + SWA (Ours) | SparseGPT | + SWA (Ours) | | Llama-3.2-1B | 0.5 | 2.10 ± 0.12 | **2.67 ± 0.12** | 3.80 ± 0.20 | **3.93 ± 0.23** | | Llama-3.2-1B | 0.7 | 1.33 ± 0.12 | **1.73 ± 0.23** | 1.80 ± 0.20 | **2.07 ± 0.12** | | Llama-3.2-3B | 0.5 | 6.87 ± 0.12 | **7.50 ± 0.12** | 6.40 ± 0.20 | **7.27 ± 0.12** | | Llama-3.2-3B | 0.7 | 2.60 ± 0.20 | **2.87 ± 0.20** | 3.00 ± 0.20 | **3.20 ± 0.23** | | Meta-Llama-3-8B | 0.5 | 16.20 ± 0.20 | **21.00 ± 0.20** | 11.80 ± 0.20 | **13.00 ± 0.16** | | Meta-Llama-3-8B | 0.7 | 1.27 ± 0.12 | **1.73 ± 0.12** | 3.47 ± 0.12 | **3.73 ± 0.31** |

Comparison with Attention-Head Importance-Based Sparsity Allocation (q2)

To assess whether SWA outperforms sparsity allocation based directly on measured attention-head importance, we compare SWA with a head-importance-based allocation baseline using Wanda at 50% sparsity. As shown in Table 3, SWA consistently achieves lower WikiText perplexity across all three models. These results suggest that SWA's structural information provides a more effective signal for sparsity allocation than attention-head importance alone.

Table 3: Comparison of WikiText perplexity (↓) between attention-head importance-based sparsity allocation and SWA using Wanda at 50% sparsity.

ModelSparsityHead Importance + WandaSWA + Wanda (Ours)
Llama-3.2-1B0.527.24 ± 0.5021.92 ± 0.05
Llama-3.2-3B0.517.41 ± 0.1713.32 ± 0.12
Meta-Llama-3-8B0.511.12 ± 0.2010.14 ± 0.03

Justification for the Functional Graph (w1)

We would like to explain the physical meaning of our graph. For a given input, attention heads in a neural network exhibit correlations, and our graph captures these dependencies. Thus, the graph constructed in this way captures differences in the physical angles between the query subspaces of different attention heads for the same input.

By representing statistical dependencies among attention heads, our graph serves a role analogous to that of functional brain graphs, which capture dependencies between signals from different brain regions. In neuroscience, co-activation may arise from physical interactions, whereas in our setting, these dependencies emerge through neural network training. Although their underlying mechanisms may differ, both forms of dependency can be represented and studied using graphs.

To characterize these dependencies theoretically, we show that the similarity used to construct our graph measures the alignment between the query representation subspaces retained by PCA. This gives the graph’s edges a geometric interpretation. We outline the proof below and provide the full derivation in Appendix A.7 of the revised manuscript.

Proof sketch. Consider any two attention heads evaluated on the same set of questions. For each head ii, let Yi∈RN×piY_i\in\mathbb R^{N\times p_i} be its centered query matrix after PCA, where NN is the number of questions and pip_i is the number of retained dimensions. Assuming full column rank, define

Ui=Yi(Yi⊤Yi)−1/2.U_i=Y_i(Y_i^\top Y_i)^{-1/2}.

The columns of UiU_i form an orthonormal basis for the column space Ui\mathcal U_i of YiY_i. Whitening the sample cross-covariance of heads ii and jj gives

=U_i^\top U_j.$$ The singular values of this matrix are both the canonical correlations between the retained query representations and the cosines of the principal angles between their column spaces. Therefore, $$\rho_k(i,j)=\sigma_k(U_i^\top U_j) =\cos\theta_k(\mathcal U_i,\mathcal U_j), \qquad 1\leq k\leq\min(p_i,p_j).$$ Here, $\rho_k(i,j)$ and $\sigma_k$ denote the $k$-th largest canonical correlation and singular value, respectively, and $\theta_k$ is the $k$-th smallest principal angle. By aggregating the these canonical correlations into edge weights, our graph captures dependencies among attention heads through the alignment of their query representation subspaces. Regarding the functional roles of the clusters identified in our graph, we have provided explanations in the “Clarification on function graph” section of our rebuttal. Please let us know whether any additional clarification or explanation would help address your concerns about the graph’s validity. ## References [1] Evaluating Mathematical Reasoning Beyond Accuracy. Proceedings of the AAAI Conference on Artificial Intelligence, 39(26), 27723–27730.
Reviewer_Wgu3OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This study investigated the small-worldness of large language models (LLMs) and its relationship to LLM reasoning performance. The authors reported that higher small-worldness is consistently associated with better fluid reasoning performance across models and training checkpoints; that important heads tend to exhibit high core scores (stronger within-community connectivity) and low bridge scores (more concentrated connectivity across communities). Based on these observations, the authors proposed SWA ( Small-World Allocation) that prunes LLMs with the guidance of core and bridge scores. The authors reported that SWA better preserves both small-world organization and model performance.

Critical strengths

1.Explored the relationship between network topology (i.e., small-world index) and model performance, drawn inspiration from brain network neuroscience. 2.Used local topological attributes (i.e., core and bridge score) for better model pruning.

Critical weaknesses

1.The way to build head-level graph might be questionable. The model itself is a computational stream rather than a relational graph (as defined in You ICML 2020) that is more suitable for topological assessment. The author did not evaluate the validity of the constructed functional graph, e.g., speaking to ‘functional’, what communities are there and what do they do? Which heads are included in a representative community and where are they?

Questions for authors

1.The authors treat each attention heads as nodes and the functional similarity between paired nodes as edges to construct functional graph of an LLM, mimicking the construction of brain networks in neuroscience. However, there seems a fundamental difference: nodes of brain regions in brain networks communicate with each other, either directly of indirectly, people then use Pearson correlation or other statistical metrics to simplify brain network construction. However, it is not the case in a transformer neural network-- Do heads communicate with each other? As defined in You ICML 2020, a relational graph, built from the original computational model, would facilitate topological assessment of neural networks. Please also see Chen 2022 (see below) which tried to build relational graph for Transformer. In this case, the author need to justify why head-level functional graph built in the current study are eligible for topological assessment of a transformer neural network. Chen et al. A Unified and Biologically Plausible Relational Graph Representation of Vision Transformers. IEEE Transactions on Neural Networks and Learning Systems 36 (2022): 3231-3243. 2.In addition to the justification aforementioned, it is also necessary to evaluate the validity of the head-level functional graph. It would be helpful to illustrate intermediate results like statistical distribution of edge weights, communities and heads included in representative communities, and even the behaviors of those communities/heads. Are those communities and the formation of communities reliable across different dataset/hyperparameters (e.g., edge density)? My point is, the relationship between model small-worldness and model performance, as well as local-topology based pruning are attractive, they should build upon solid functional graph construction. 3.In Figure 3, more models covering grater range of SMI space could be helpful. Why not using Pythia training checkpoints like those in Figure 2? In addition, more checkpoints (e.g., denser sampling in early stage) would help in Figure 2. 4.In Activation Collection, is the operation of token-level average reliable? The activation across tokens could be heterogeneous and average operation could erase true activation patterns. 5.Are the main conclusions/results sensitive to the hyperparameter edge density? 6.Details of integrate local topology with SparseGPT and Wanda are missing. 7.Fluid intelligence is not equivalent to reasoning capability, although they correlate with each other. People use specific metrics beyond perplexity to evaluate reasoning capability. 8.Section 5.4, “Moreover, SWA(Head) degrades more than SWA(Layer) in every setting. These results suggest that layer-level scores provide the main allocation signal, while head-level scores refine pruning within layers.” I am a little bit confused. Shouldn’t be “head-level scores provide the main allocation signal”?

10 comments

Clarification on function graph (w1, q1)

We would like to clarify that our functional graph characterizes a trained model’s functional organization with respect to a particular capability. To study that capability, we select a dataset containing questions that reflect it. For example, to study reasoning, we use reasoning questions and construct the graph from the model’s activations while it answers those questions.

Our goal is to understand how this functional organization relates to model performance. Models with identical architectures can develop different capabilities because of differences in training data or training settings. By using the trained model’s activations, our analysis can reflect these learned differences. The resulting graph also depends on the questions the model answers, since different datasets may involve different capabilities. Changes in the graph across training conditions or datasets are therefore expected.

To investigate the functional meaning of the communities identified by this graph, we examine how community interventions affect the model’s representations and predictions on counting tasks. We first find that communities have distinct directional effects on the final hidden representation, shown by Cosine similarity analysis (Figure 4 in Appendix A.6 of the revised manuscript.) and PCA visualization (Figure 5 in Appendix A.6 of the revised manuscript.). We then use a counting case study to show that a specific community-associated direction is related to object count and can influence the model’s numerical predictions (Figure 6 in Appendix A.6 of the revised manuscript.), providing a task-specific semantic interpretation.

Specifically, we select the six largest communities (by number of attention heads) identified by applying Louvain community detection to our functional graph. To examine how each community affects the model’s final hidden representation, we multiply the outputs of all attention heads in one selected community by 0.5, following [8, 9]. For each question, we compare the intervened and original representations at the output of the final Transformer block. We compute the change relative to the original representation and normalize it to obtain the intervention direction:

u_C(q)=\frac{\Delta r_C(q)}{\|\Delta r_C(q)\|_2}.$$ Figure 4 in Appendix A.6 of the revised manuscript compares these directions across communities for the same held-out questions. The reported off-diagonal cosine similarities range from −0.20 to 0.28, indicating limited alignment between the directions induced by different communities. Figure 5 in Appendix A.6 of the revised manuscript shows the corresponding PCA visualization, where each point represents one community–question pair. Points associated with different communities occupy largely separate regions, with particularly compact groups for C206, C207, and C208. These results show that the communities identified by our functional graph have distinguishable effects on the model’s final hidden representation. We next examine the functional meaning of a particular community-associated direction. Additionally, to assess the validity of the communities, we first examine whether each cluster captures distinct semantics in the output space. Specifically, perturbing a valid cluster should induce consistent directional changes in the outputs across different inputs, while perturbations to different clusters should produce sufficiently distinct directions. To examine these properties, we first estimate a representative semantic direction for each community using training questions. Specifically, we compute the changes in the representations caused by perturbing each community, then average the normalized changes across training questions to obtain a fixed, community-specific semantic direction: $$d_C=\operatorname{normalize}\left( \frac{1}{|\mathcal T|}\sum_{q\in\mathcal T}u_C(q) \right).$$

As a case study, we focus on community C125 and examine whether its semantic direction induces consistent changes across different counting questions. Specifically, we add a scaled version of this direction to the original representation at the same location, while keeping the input unchanged:

r′(q)=r0(q)+αdC.r'(q)=r_0(q)+\alpha d_C.

To quantify the resulting changes in numerical predictions, we compute a probability-weighted count score based on the predicted probabilities of digits 1–9:

Eα(q)=∑n=19n Pα(n∣q).E_\alpha(q)=\sum_{n=1}^{9}n\,P_\alpha(n\mid q).

By comparing changes in this score across held-out questions, we assess whether shifting representations along the same community-specific semantic direction produces consistent effects on numerical predictions. As shown in Figure 6 in Appendix A.6 of the revised manuscript, increasing the scaling coefficient along the fixed C125 semantic direction consistently increases the expected count across both input conditions. These results suggest that C125 is associated with majority-related semantic information.

Overall, these findings support our view that the functional graph captures task-dependent aspects of a trained model's internal organization. The distinguishable effects of different communities on hidden representations, together with the C125 case study linking a specific semantic direction to the notion of majority, provide further evidence that the identified communities exhibit meaningful functional differences and can capture interpretable, task-specific semantics.

Hyperparameter study (q2, q5)

To further evaluate the robustness of the identified communities, we conduct two additional analyses to examine (1) their sensitivity to graph density thresholds and (2) their agreement across different tasks.

First, we examine the sensitivity of community assignments to the graph density threshold. As shown in Figure 7 in Appendix A.8 of the revised manuscript., the mean best-match Jaccard similarity remains relatively high across density thresholds ranging from 0.125 to 0.375, with values between 0.732 and 0.982, consistently exceeding the random baseline of approximately 0.13. This suggests that the identified communities are relatively robust to the choice of graph density threshold.

Second, we examine whether community agreement reflects task similarity. As shown in Table 1, the closely related Left movement and Right movement tasks exhibit high community agreement, achieving a best-match Jaccard similarity of 0.864 and an Adjusted Rand Index (ARI) [10] of 0.977. In contrast, comparing Left movement with Count yields lower agreement, with a Jaccard similarity of 0.538 and an ARI of 0.435. Both Jaccard scores exceed the random baseline of 0.165. These results indicate that related tasks exhibit more similar community structures, while different tasks lead to more distinct community assignments.

Table 1: Community Agreement Across Different Tasks.

Task ComparisonBest-Match Jaccard ↑Random Jaccard BaselineAdjusted Rand Index (ARI) ↑
Left Movement vs. Right Movement0.8640.1650.977
Left Movement vs. Count0.5380.1650.435

For the experimental setup, we vary the graph density from 0.125 to 0.375 while keeping the remaining settings fixed. For the cross-task study, we construct separate graphs using equal numbers of questions from three DRE-Bench tasks: Left movement, Right movement, and Count, while keeping the graph density fixed at 0.25. We select the six largest communities from the Left movement graph as references and match each to the community with the highest Jaccard similarity in the comparison graph. We additionally report ARI to evaluate agreement between the overall community partitions.

Overall, these findings suggest that our functional graph identifies communities that are relatively stable across graph density thresholds while capturing meaningful differences in functional organization across tasks.

Robustness to Instruction Paraphrasing (q2, q5)

We test whether perturbing the same community changes the model’s final hidden representation in a consistent direction when the instruction is rephrased. The result is shown in Table 2

Table 2: Robustness results to instruction paraphrasing

CommunityRobustness checkConditions comparedMean directional consistency95% CI
C208Instruction paraphrasingOriginal vs. paraphrased instruction; both at gate = 0.50.9819[0.9807, 0.9830]
C206Instruction paraphrasingOriginal vs. paraphrased instruction; both at gate = 0.50.9402[0.9371, 0.9428]

Both communities show high directional consistency: 0.9819 for C208 and 0.9402 for C206, with narrow 95% confidence intervals. These results indicate that their intervention directions remain stable under the tested instruction paraphrases.

For each question, we compare the intervention directions obtained using the original and paraphrased instructions. In both conditions, we multiply the outputs of all attention heads in the selected community by 0.5. We compute each direction relative to the unperturbed representation under the corresponding instruction, then normalize it. Directional consistency is measured by the inner product of the two normalized directions and averaged across questions.

Attention-Head Interactions and Functional Connectivity (q1)

We thank the reviewer for raising this important point. We clarify that attention heads in a well-trained neural network are not statistically isolated computational units. Prior theoretical and empirical studies have shown that well-trained neural networks exhibit strong correlations among learned weights. Specifically, Martin and Mahoney [1, 2] demonstrated that well-trained neural networks often exhibit heavy-tailed spectral distributions in their weight matrices, reflecting strong correlations across multiple scales. Hodgkinson et al. [3] further investigated the theoretical relationship between heavy-tailed spectral behavior and complex correlation structures.

These papers show that well-trained neural networks have strongly correlated weights. These correlations naturally propagate to the correlations between attention heads. Such strong dependencies reflect important properties of the network's learned functionality, even when attention heads are not directly connected.

Therefore, we construct a functional graph to capture these learned dependencies between attention heads rather than explicit architectural connections. Specifically, we treat attention heads as nodes and quantify their functional relationships through activation-based similarities across task inputs. This provides a principled basis for studying the learned functional organization of LLMs through graph topology.

Reliability of Token-level Averaging in Activation Collection (q4)

Our goal is to measure statistical relationships between attention heads across questions drawn from the intelligence-benchmark distribution. For each question, we average each head's activations over the generated answer tokens to obtain a representative activation vector. We then measure the relationships between pairs of heads across the calibration questions using CCA with Fisher-z aggregation:

Sij=∑k=1rarctanh⁡(ρ~ij,k),S_{ij}=\sum_{k=1}^{r}\operatorname{arctanh}(\tilde{\rho}_{ij,k}),

where ρ~ij,k\tilde{\rho}_{ij,k} denotes the canonical correlation. This approach is consistent with the statistical perspective of functional connectivity in neuroscience [4, 5], where connectivity is estimated from correlations across observations. While token-level averaging may discard some token-specific information, it retains how each head's average activation varies across questions, allowing us to characterize statistical relationships between heads under the benchmark distribution.

Integration of SWA with Wanda and SparseGPT (q6)

We clarify that SWA modifies only the sparsity allocation strategy while retaining the original pruning procedures of Wanda [6] and SparseGPT [7]. Specifically, we compute core and bridge scores from the attention-head functional graph and use them to assign sparsity ratios at both the Transformer-block and attention-head levels, while maintaining the target global sparsity.

For Wanda, we compute weight importance as the product of weight magnitude and the L2 norm of the corresponding input activations. We then prune the lowest-scoring weights according to the sparsity ratios assigned by SWA, without updating the remaining weights. For SparseGPT, we follow its second-order pruning procedure, which uses calibration activations to approximate layerwise reconstruction error and updates the remaining weights to compensate for pruning-induced errors. In both cases, SWA determines the sparsity allocation.

Reviewer_dMjXOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper investigates the relationship between small-world organization in attention-head functional graphs and LLM reasoning performance. It reports associations between the small-world index and performance across models and training checkpoints, and proposes Small-World Allocation (SWA), which uses graph-derived core and bridge scores to guide sparsity allocation during pruning

Critical strengths

The paper addresses an interesting question about the internal organization underlying LLM reasoning. The authors conduct extensive experiments across multiple models, training checkpoints, and pruning methods to investigate this relationship and demonstrate its potential practical value for model compression.

Critical weaknesses

Direct evaluation of reasoning performance remains limited. The main evidence relies on answer-token NLL and language-modeling perplexity, which do not directly establish whether models produce correct reasoning outcomes. The authors should additionally report final-answer accuracy under chain-of-thought prompting on GSM8K or comparable reasoning benchmarks, both before and after pruning. This would provide more direct evidence that SWA preserves reasoning capability.

The contribution of topology is not sufficiently separated from layer-dependent effects. Previous work reports substantial variation in reasoning-graph properties across Transformer layers, although it uses a different graph representation from this paper [1]. While the authors provide layer-only and head-only ablations, these do not fully establish whether SWA benefits from the identified topology or primarily captures differences in layer importance and pruning sensitivity. A controlled experiment that fixes layer-wise sparsity budgets and shuffles head scores within each layer would help determine whether the topological scores provide information beyond layer identity.

[1] Topology of Reasoning: Understanding Large Reasoning Models through Reasoning Graph Properties

Questions for authors

None

4 comments

Additionally reasoning accuracy

To evaluate whether our SWA better preserves reasoning ability before and after pruning, we conduct additional experiments on held-out GSM8K and Prealgebra. As shown in Table 1 and 2, SWA consistently achieves higher reasoning accuracy than the SparseGPT baseline across all three models and both sparsity levels. For example, on Meta-Llama-3-8B at 50% sparsity, SWA improves GSM8K accuracy from 16.20% to 21.00% and Prealgebra accuracy from 11.80% to 13.00%. These results provide direct evidence that SWA better preserves reasoning performance after pruning.

Table 1: Reasoning Accuracy on GSM8K and Prealgebra Before Pruning.

ModelGSM8K sparseGPTPrealgebra sparseGPT
Llama-3.2-1B6.6 ± 0.206.7 ± 0.23
Llama-3.2-3B29.4 ± 0.4014.2 ± 0.35
Meta-Llama-3-8B52.5 ± 0.1221.6 ± 0.20

Table 2: Reasoning Accuracy on GSM8K and Prealgebra After Pruning at 50% and 70% Sparsity.

ModelSparsityGSM8KPrealgebra
SparseGPT+ SWA (Ours)SparseGPT+ SWA (Ours)
Llama-3.2-1B0.52.10 ± 0.122.67 ± 0.123.80 ± 0.203.93 ± 0.23
Llama-3.2-1B0.71.33 ± 0.121.73 ± 0.231.80 ± 0.202.07 ± 0.12
Llama-3.2-3B0.56.87 ± 0.127.50 ± 0.126.40 ± 0.207.27 ± 0.12
Llama-3.2-3B0.72.60 ± 0.202.87 ± 0.203.00 ± 0.203.20 ± 0.23
Meta-Llama-3-8B0.516.20 ± 0.2021.00 ± 0.2011.80 ± 0.2013.00 ± 0.16
Meta-Llama-3-8B0.71.27 ± 0.121.73 ± 0.123.47 ± 0.123.73 ± 0.31

Control with Fixed Layer-Wise Sparsity Budgets

To test whether our topological scores provide information beyond layer identity, we conduct a controlled experiment that holds layer-wise sparsity budgets fixed while shuffles head scores.

As shown in Table 2, our method achieves lower WikiText perplexity than the shuffled-score control across all three models at both sparsity levels. These results suggest that the topological scores provide useful information about individual heads beyond the allocation of sparsity across layers.

Specifically, we fix each layer’s sparsity budget to that used by our method and randomly shuffle the topological scores among heads within that layer. This preserves the layer-wise budgets and score distributions while changing which head receives each score. We then apply the same SparseGPT pruning procedure and evaluate WikiText perplexity at overall sparsity levels of 0.5 and 0.7.

Table 2: WikiText Perplexity (↓) with Shuffled Head Scores.

ModelSparsityShuffled head scores + SparseGPTOurs + SparseGPT
Llama-3.2-1B0.522.96 ± 0.7820.47 ± 0.15
Llama-3.2-1B0.7112.05 ± 1.67105 ± 3.6
Llama-3.2-3B0.513.86 ± 0.5111.27 ± 0.98
Llama-3.2-3B0.755.14 ± 1.1749.3 ± 0.93
Meta-Llama-3-8B0.512.00 ± 0.2010.15 ± 0.08
Meta-Llama-3-8B0.740.75 ± 0.6638.93 ± 1.29

Thank you again for your positive assessment and constructive feedback. We have addressed your comments in our rebuttal and would greatly appreciate any further thoughts before the discussion period ends. Please let us know if there are any remaining points we could clarify.

Thanks for your response. I would like to keep my positive score.

Thank you for your response and continued support of our paper. We appreciate your time and thoughtful feedback throughout the review process.

Best,

Authors of Submission 2839

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.