- Rating
- 2(−1)
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 3
Read review
Paper summary
This paper identifies sparse MLP neurons, termed Sink-Control Neurons (SCNs), that causally regulate attention sinks in large language models. Experiments on four Qwen and Llama checkpoints show that suppressing a small number of SCNs reduces sink strength by 12%–51%, while amplifying their activations increases it by 1%–6%. The authors further investigate the underlying query–key pathways, evaluate the effects of SCN suppression on downstream task performance, and examine whether restoring the original attention sink can recover the resulting behavioral changes. Finally, they demonstrate that preserving SCN coordinates in high precision recovers 65.3% of the quantization-induced cross-entropy loss in a W8A8 experiment. The results suggest that a small population of MLP neurons contributes to attention sink formation while also influencing broader model computations.
Critical strengths
-
The paper provides a systematic causal identification pipeline for locating sink-controlling neurons. The combination of gradient-based candidate selection, direct neuron-level interventions, and layer/count-matched random controls provides evidence that the identified neurons genuinely influence attention sink strength.
-
The authors go beyond identifying sink-controlling neurons and investigate their underlying mechanisms. The query–key pathway decomposition and restoration experiments provide useful insights into how SCNs influence attention allocation, while also revealing differences across model checkpoints.
-
The downstream behavioral evaluation is informative. Experiments across multiple benchmarks show that SCN suppression can substantially alter model predictions, even when the net accuracy change is small. The analysis of prediction transitions further highlights behavioral changes that aggregate accuracy alone may obscure.
Critical weaknesses
-
Missing relevant work on sparse causal neuron identification. The paper does not discuss the closely related work https://arxiv.org/abs/2510.10238. This work proposes a perturbation-based method for identifying ultra-sparse critical neuron populations and investigates their effects on language model capabilities, including their layer-wise distribution and the consequences of disrupting these neurons. Given the overlap in neuron-level causal identification and intervention-based analysis, the authors should discuss this work and clarify how the present study differs from and advances beyond it.
-
The generality of the findings remains unclear. The experiments cover only four instruction-tuned checkpoints with at most 8B parameters, and several analyses, particularly the pathway decomposition and quantization experiments, are conducted on only a subset of these models. Moreover, neuron localization relies on short neutral-text sequences. It is therefore unclear whether the identified SCNs and their functional roles generalize to base models, larger models, and long-context settings.
-
The evidence connecting attention sink restoration to downstream behavior is not fully conclusive. Although the authors restore the original sink attention distribution while keeping SCNs suppressed, the behavioral recovery is limited. However, the paper explicitly reports that the sham-control adequacy criterion fails for all evaluated tasks. This limits the strength of the causal interpretation and makes it difficult to isolate the specific contribution of position-0 attention to the observed behavioral effects.
-
The practical advantage of SCN-based quantization protection is not fully established. Although preserving SCN coordinates recovers 65.3% of the W8A8 quantization-induced cross-entropy loss, both layer-matched and global magnitude-based protection perform better in the reported experiment. Thus, the results establish that SCNs are relevant to quantization sensitivity, but do not demonstrate that SCN-based protection is more effective than simpler magnitude-based alternatives.
Questions for authors
-
The authors should evaluate the generalization of SCNs across model scales, model families, and context lengths. In particular, testing on base checkpoints and longer sequences would help establish whether the identified neurons reflect a robust mechanism or a property specific to the selected instruction-tuned models and localization setup. Additional cross-model analyses of SCN overlap and functional consistency would also be valuable.
-
The authors should strengthen the attention restoration experiments with better-controlled sham interventions that satisfy the predefined adequacy criterion. Additional controls and behavioral mediation analyses would help determine whether the limited recovery is genuinely attributable to the restored attention sink or whether other effects of the intervention account for the observed behavior.