- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes a framework for analyzing how LLMs organize reusable predictive mechanisms, which the authors call “skills.” Rather than starting from predefined architectural units such as attention heads, circuits, or neurons, the approach is data-driven: token-level loss gradients with respect to the last Transformer block are clustered, and each cluster is treated as a skill. The authors then build a Skill Co-activation Graph (SCG), connecting two skills whenever they co-occur in the same example. On the theoretical side, the paper derives a generalization bound for a graph-smooth skill-alignment function class, in which the complexity term depends on the task-conditioned resistance Tr(L⁺Σ_T) of the SCG. A stability-aware extension accounts for the graph being estimated from data. Empirically, across six Pythia model scales on ARC-Challenge and SciQ, lower task-conditioned resistance is strongly correlated with higher held-out accuracy. Ablating task-relevant skill subspaces degrades gold-answer likelihood more than ablating random subspaces of the same rank. Finally, the authors use per-example SCG resistance as a data-selection criterion for budget-limited fine-tuning, and report that selecting high-resistance examples yields accuracy gains over random, high-loss, and skill-coverage baselines on Pythia-2.8B and Qwen2.5-3B.
Critical strengths
-
The idea is interesting and offers a new perspective on how LLMs work internally. Inducing “skills” directly from the model’s own loss-gradient geometry, rather than from predefined architectural units or external task labels, is a fresh, data-driven angle.
-
The theoretical development is clear and intuitive. The derivations are complete, and the resulting task-conditioned bound, with the spectral interpretation of , gives a thoughtful way to reason about how a task’s skill demands interact with the structure of the skill graph.
-
The empirical results are consistent with the theory. The authors further turn their observations into a practical data-selection metric (SCG resistance) for budget-limited fine-tuning, which shows a potential use of the framework beyond analysis.
Critical weaknesses
-
Some modeling choices are oversimplified and may undermine the main empirical conclusion. For example, the SCG connects two skills whenever they co-occur in at least one example. With many examples, this binary rule likely makes the graph nearly fully connected. In a fully connected graph, Rnorm reduces to exactly , so its value is determined mainly by the number of retained skills rather than by the graph’s structure.
-
The fine-tuning gains are small and may not support the conclusion. For Qwen2.5-3B on ARC-Challenge, the improvement from SCG High-R is below 0.1 percentage points, roughly one question on the test set. Across settings, the error bars of several strategies overlap, with only three seeds. Statements such as High-R “consistently exceeds” the baselines therefore seem too strong. More seeds, significance tests, or larger evaluation sets would make this result more convincing.
Questions for authors
-
Can you report , Rnorm × , edge density, and the number of connected components for every model–task pair, and explain any cases where Rnorm < ?
-
How do the results change with a weighted graph (using N_ij) instead of the binary existence rule?
-
How are skill subspaces constructed for ablation, and why are the 70M/160M effects an order of magnitude larger than for the other models?
Appreciate the detailed response provided by the authors. Most of my concerns have been adequately addressed, and I will maintain my positive evaluation score.