acceptodds
Under review as a conference paper at ICLR 2027

Understanding Context Utilization via Neuron Sinks

Abstract

Transformer language models apply the same computational blocks at every position, although causal masking limits the context available near sequence beginnings. We identify neuron sink, a phenomenon across diverse language models in which a sparse subset of MLP neurons exhibits disproportionately frequent strong activations near sequence beginnings. This pattern reappears at the start of a passage following an irrelevant prefix, suggesting that neuron sink depends on the availability of useful context rather than absolute position or context length. Ablating sink neurons selectively impairs prediction near sequence beginnings, demonstrating their functional importance for low-context processing. Beyond this role, we find evidence that neuron sink contributes to behavioral position bias. In a controlled model, transplanting the sequence-initial sink-neuron state to middle positions reduces first-over-middle retrieval bias, causally linking this activation bias to the primacy component of the lost-in-the-middle phenomenon. These findings uncover a mechanism for low-context processing and identify a potential target for mitigating position bias in language models.

Primary area:
interpretability and explainable AI
Keywords:
Transformer, Large Language Models, Mechanistic Interpretability, Neuron Sink, Context Availability, Position Bias

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_nt8rOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper investigates "sink neurons", defined as neurons that activate "disproportionately often near sequence beginnings". The authors first show that these sink neurons exist and harm early-token prediction, that these neurons are more robust than absolute position alone and ignore unrelated context, and that copying sink neuron activations to middle tokens (in a 2-layer model) reduces the lost-in-the-middle effect and improves retrieval for those tokens.

Critical strengths
  • The writing is mostly clear and the experiments cover most potential confounders for the presence and effect of these sink neurons.
  • The result in Section 6 is a very interesting way to go beyond just showing certain neurons that activate early in the sequence exist, and showing that sink neurons impact model behavior in an interesting way.
Critical weaknesses
  • The high level motivation for the work is lacking. From prior work we know that various positional neurons exist, and so it does not feel particularly surprising that early-position neurons also exist [1,2]. Could the authors clarify how their sink neurons differ from the positional neurons found in prior work and whether sink neurons behave in any unforeseen ways? The paper currently seems like finding neurons that are activate early in the sequence and then showing results that are nearly by-definition (ablating sink neurons harms early sequence loss)
  • The result in Section 6 is very cool but I do not buy the current result. Why is a model trained from scratch for this result? If the authors can find sink neurons in open-weight models, why can these not also be tested? Additionally, why can error bars not be evaluated for the current result? I see that the random patch is computed as an average over multiple neuron groupings, but can the intervention evaluation not be extended further to get a sense of whether this 2% improvement is a robust result or due to noise?

[1] Gurnee, Wes, et al. "Universal neurons in gpt2 language models." arXiv preprint arXiv:2401.12181 (2024). [2] Voita, Elena, Javier Ferrando, and Christoforos Nalmpantis. "Neurons in large language models: Dead, n-gram, positional." Findings of the Association for Computational Linguistics: ACL 2024. 2024.

Questions for authors
  • Could an explanation be provided for how exactly the sink neuron copying result should improve lost-in-the-middle? This seems like a very interesting result that is currently underexplored.
4 comments

We thank the reviewer for the careful review and for finding the retrieval result interesting. We first clarify the motivation and how sink neurons differ from positional neurons (1). We then address the retrieval result: new evidence in a pretrained LM (2), its robustness (3), and how the transplant changes retrieval (4).

(1) Motivation, and relation to positional neurons (W1). Our question is whether a language model computes differently when useful context is scarce, not whether some neurons prefer early positions. Prior work characterizes early-position neurons by absolute position; we find sink neurons respond to useful context even at fixed positions, are shaped by the training data, and affect how evidence is later retrieved. In detail:

  • Motivation. Useful context is scarce at the start of a sequence, after an unrelated passage, or when required evidence is missing. Sequence beginnings are the most common such case, which is why sink neurons appear to prefer early positions. We open the introduction with this motivation.
  • Context-aware, not positional. Both works study models with learned position embeddings: Gurnee et al. describe "neurons which activate as a function of absolute position rather than token or context" (their §4.3), and Voita et al. leave the function open (their §5.5). We will discuss both works in §2.1. In our study, sink neurons instead track useful context. A positional account predicts no change in any comparison below, since each holds positions fixed:
Comparison (positions held fixed)Result
Same 1,024 target tokens after an unrelated vs. related prefixTarget-start activation rate ×1.52 (Qwen3-0.6B), ×1.69 (Llama-3.1-8B)
Required evidence masked at a late query, all tokens unchangedStandardized activity +0.067, vs. +0.013 for matched non-sink neurons
Copy task with the first useful token delayed from k = 2 to 64; only the training data differSink span 2.25 → 38.75 positions; IID control: brief initial elevation only
Sink neurons ablated at the same 32 target positions, unrelated vs. related prefixLoss increase 0.044 vs. 0.011 nats/token
  • Shaped by the training data. The early-position pattern needs no BOS token, document-opening text, or attention sink (§5.1), and in the copy task (third row), the sink span follows how long predictions lack useful context.
  • Not by definition. We agree that ablating neurons selected for early activity is expected to hurt early predictions. Selection does not predict the last row, however, where position is held fixed and only context varies. There, the same target passage follows either its own preceding text (related prefix) or another document (unrelated prefix), so the 32 ablated positions (1,024–1,055) hold the same tokens in both cases. Ablating sink neurons raises the loss by 0.011 nats/token after the related prefix but by 0.044 after the unrelated one, whereas non-sink neurons matched on activity and projected contribution barely change (0.022 vs. 0.024). The rise from related to unrelated prefix is thus 0.031 nats/token larger for sink neurons.
  • Affects later retrieval. Transplanting the beginning state of sink neurons into middle-position evidence improves its retrieval in a pretrained LM ((2)); in the controlled model, later queries attend more to evidence that carries this state ((4)).

(2) Behavioral evidence in a pretrained LM (W2). Following the reviewer's suggestion, we tested the open-weight model in which we characterized neuron sink: transplanting the beginning state of sink neurons into middle-position evidence improves its retrieval. We had used a controlled model because the paired first/middle protocol of Wu et al. (2025) fixes content exactly and its small size allows exhaustive controls.

  • Setup. We use Qwen3-0.6B-Base, which shows a clear middle deficit, and keep the paper's sink population fixed. We copy the sink-neuron activations of evidence placed first, in all layers, into the same evidence placed in the middle. As the transplant targets known evidence, it serves as a causal probe, not a mitigation method.
  • Controls. Random neuron sets matched in layer and count.
  • Tasks. KV retrieval uses 2,048-token prompts with 408 key–value pairs, with the queried pair placed 1st or 205th. Natural Questions is the multi-document QA task of Liu et al. (2024). Each question comes with 10 Wikipedia passages, one of which contains the answer; we place this passage 1st or 5th, evaluate questions the model cannot answer without the passages, and score exact match.
TaskClean first / middle (%)Middle after transplant (%; Δ [95% CI])Gap, clean → after (pp)Random Δ middle (pp; mean ± SD)
KV retrieval (2,000 prompts)99.7 / 78.792.1 (+13.4 [11.7, 15.1])21.0 → 7.6+0.1 ± 2.8 (100 sets)
NQ (2,107 questions)52.7 / 32.836.2 (+3.3 [2.3, 4.4])19.8 → 16.5+0.1 ± 0.4 (10 sets)

(3) Error bars and robustness of the controlled result (W2). The controlled-model effect is stable across samples and training runs.

  • Paired CIs. Clean and patched runs share inputs, so we bootstrap over paired samples. On the paper's 2,000 samples, the gap changes by −2.05 [−3.10, −1.05] points, and middle-target accuracy by +1.55 [0.80, 2.30] (44 errors fixed, 13 introduced; McNemar p = 4.7×10⁻⁵). With 10,000 samples, the gap change is −1.71 [−2.22, −1.23].
  • Training seeds. We trained five more checkpoints from scratch, each with a new initialization, data stream and task draw, and reselected the sink neurons in each with the unchanged rule. The effect replicates in all six checkpoints (gap change −2.87 ± 0.74 pp, mean ± SD across checkpoints; all CIs exclude zero). Random sets instead increase the gap (+3.11 pp) and never reach the sink effect.
  • Why the effect is small. In the two-layer model, only first-layer multilayer perceptron (MLP) outputs at context positions can reach the final query: second-layer MLP outputs at those positions are never read, and copying them leaves accuracy unchanged. The original intervention is thus confined to eight neurons of one layer and moves the state in one direction only; moving each pair's own state in both directions reduces the gap by 10.7 pp (response (4)).

(4) How the copy reduces first-over-middle preference (Q). In short, neuron sink is an internal processing bias that likely gives rise to the behavioral one, and the copy weakens it. Evidence at the start is processed with scarce context and carries the state that sink neurons mark; later queries attend more to evidence carrying this state. The copy gives middle evidence this state, narrowing the internal difference between first and middle evidence and, with it, the first-over-middle preference. The controlled task lets us trace each step:

  • Internal bias. A query must find the pair whose content matches it and read out that pair's label. In the paired evaluation, the first and fifth pairs carry the same content vector, but under causal masking the first pair's label token sees only its own content token and thus carries the low-context state, whereas the fifth pair's label token follows eight tokens from other pairs.
  • From internal to behavioral bias. Despite the identical content, the query's second-layer attention to the first pair is 2.8 times that to the fifth (0.27 vs. 0.10), consistent with the higher accuracy when the first pair holds the answer (§6).
  • Reducing the internal bias reduces the behavioral bias. We transplant each pair's own state, so that no content or label passes between pairs: the middle pair's beginning state comes from running the pair alone, and the first pair's middle state from placing it fifth in another sample. Giving the middle pair the beginning state raises the attention it receives by about 28%, and giving the first pair the middle state lowers its attention by about 21%; the same transplants on random neurons change attention by less than 5%. Each transplant also shrinks the gap, by 10.7 points when both are applied (means over the six checkpoints; gap changes in points, mean ± SD; negative values shrink the gap; every CI excludes zero, and no random set reaches the sink effect):
InterventionQuery attention, first / middle pairGap change, sink neurons (pp)Gap change, random sets (pp)
None (clean)0.27 / 0.10––
Original copy (§6): first pair's activations → middle pair0.26 / 0.12−2.87 ± 0.74+3.11 ± 0.57
Middle pair ← its own beginning state0.26 / 0.12−3.67 ± 0.71−0.46 ± 0.20
First pair ← its own middle state0.21 / 0.10−5.75 ± 0.73−1.25 ± 0.46
Both in one forward pass0.20 / 0.14−10.66 ± 1.37−1.79 ± 0.61
  • Gurnee, Wes, Theo Horsley, Zifan Carl Guo, Tara Rezaei Kheirkhah, Qinyi Sun, Will Hathaway, Neel Nanda, and Dimitris Bertsimas. (2024). [Universal Neurons in GPT2 Language Models]
  • Liu, Nelson F., Kevin Lin, John Hewitt, Ashwin Paranjape, Michele Bevilacqua, Fabio Petroni, and Percy Liang. (2024). [Lost in the Middle: How Language Models Use Long Contexts]
  • Voita, Elena, Javier Ferrando, and Christoforos Nalmpantis. (2024). [Neurons in Large Language Models: Dead, N-gram, Positional]
  • Wu, Xinyi, Yifei Wang, Stefanie Jegelka, and Ali Jadbabaie. (2025). [On the Emergence of Position Bias in Transformers]

If the authors have updated the draft, could they use a consistent color throughout to show changes? It is currently difficult to track changes.

Additionally, the results on Qwen3 0.6B seem to be much more drastic than the previous results. Could the authors provide additional information on this experimental setup?

  • Is the patching procedure consistent among all positions? Does the patching degrade the early items?
  • What is meant by "copy the sink-neuron activations of evidence placed first, in all layers"? This patches the "31 neurons in each of the 28 layers based on their relative activation rates in the first 32 tokens."?
  • Why do the authors think the gain does not hold on the natural questions dataset?

We thank the reviewer for these follow-up questions. We have now highlighted all revisions in the same color throughout the updated manuscript.

(1) Experimental setup. The full experimental setup is now detailed and highlighted in Section 6.3 and Appendix I of the revised manuscript. We summarize the key details below.

We use Qwen3-0.6B-Base with 2,048-token KV prompts containing 408 mappings, placing the queried mapping first or 205th. NQ uses ten passages, placing the answer-containing passage first or fifth, and evaluates 2,107 questions answered incorrectly without passages. The Qwen intervention covers 31 coordinates per layer across 28 layers. Appendix I specifies the prompts, token alignment, scoring, and paired statistics.

(2) Consistency across positions and effects on early items. The patching rule is fixed across target positions, and patching middle evidence improves Qwen retrieval with little average change in early-item accuracy.

In the two-layer experiment, the same first-to-middle copy is applied in both first-target and middle-target trials. Table 1 already reports both accuracies: across six checkpoints, First decreases from 86.14% to 85.25%, while Middle increases from 69.11% to 71.08%.

For Qwen, we additionally measured both target positions under each fixed rule. The forward transplant always patches the current middle item with its own clean beginning state; the reverse transplant always patches the first item with its own clean middle state. Neither rule changes with the queried item or gold-passage identity. The two directions are evaluated separately on the original samples.

TaskInterventionFirst accuracy (%)Middle accuracy (%)
KVClean99.7078.70
KVOwn beginning → middle99.7092.05
KVOwn middle → first99.5077.95
NQClean52.6832.84
NQOwn beginning → middle52.4036.17
NQOwn middle → first50.1233.79

Patching middle evidence causes no First correctness flips on the 2,000 KV tables. For NQ, First changes by −0.28 percentage points (95% paired CI [−1.04, 0.52]), while Middle improves by +3.32 points [2.28, 4.41]. The estimated First cost is therefore small, although modest harm remains possible. The reverse transplant reduces NQ First accuracy by 2.56 points; its KV First change is only −0.20 points.

(3) Meaning of “copy … in all layers.” Yes: we patch the fixed 31 neurons in each of the 28 layers, using aligned activations from the same evidence.

These neurons are selected by first-32-token relative activation rates using Q98. That window defines selection, not the patch span. A clean donor run places the evidence first; the recipient places it in the middle. At each evidence token and layer, we replace the selected pre-down-projection MLP coordinates with their corresponding donor values. The span covers the complete five-token KV mapping or the complete NQ passage, including its title and terminating newline. See Appendix I for patching and Section 3.2/Appendix A for selection.

(4) Why is the NQ improvement smaller? Many gold-answer scores improve without changing the generated answer, which may partly explain the smaller accuracy gain.

In a post-hoc analysis, 510 NQ questions were correct with first-position evidence but incorrect with middle-position evidence. The transplant corrected 79. Among the remaining 431, gold-answer log-probability increased in 383 cases (88.9%). KV selects one token from 408 candidate values, whereas NQ freely generates an answer: improving the teacher-forced likelihood of the best gold alias need not change the greedy-generated response.

Reviewer_Wui7OpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper identifies MLP coordinates whose large-magnitude activations occur disproportionately near the beginning of a sequence, naming this pattern “neuron sink.” The pattern appears across several pretrained model families and sizes. In Qwen3-0.6B, ablating the selected coordinates increases early-token prediction loss much more than late-token loss. Experiments with unrelated prefixes, delayed predictability in synthetic sequences, and missing retrieval evidence associate sink activity with limited useful context. A separate intervention in a two-layer retrieval model copies beginning-position sink activations to middle positions and reduces the first-versus-middle accuracy gap by 2 percentage points. The work proposes that these neurons participate in processing with limited contextual support and can contribute to retrieval position bias.

Critical strengths
  1. The matched-prefix controls provide the strongest support for the paper’s central claim. By fixing both the target tokens and the selected neurons while replacing a relevant prefix with an unrelated prefix of equal length, the authors show that activation rates of the same population increase substantially.

  2. The ablation experiments further establish functional relevance. Zeroing the identified coordinates produces a marked increase in early-token loss that substantially exceeds the effect of layer- and count-matched random sets and decays rapidly with position.

Critical weaknesses
  1. The ablation controls remain insufficiently matched. Sink neurons are selected precisely for elevated early activation frequency, yet the random baselines match only layer and count. Consequently, the large early-token loss increase demonstrates that these coordinates matter under zero ablation, but does not isolate a distinctive computational role beyond their greater early activity and projected contribution.

  2. The retrieval intervention yields only a modest 2-point gap reduction in a single two-layer toy model, with no paired uncertainty estimates and no independent training seeds.

Questions for authors
  1. In the retrieval intervention, does the gap reduction survive when content-position and label-position activations are patched separately, or when position-matched donors carrying different content are used?
3 comments

We thank the reviewer for the careful review and for the precise questions about the retrieval intervention, which led to cleaner tests. We first address the retrieval intervention (1)–(4), beginning with new evidence from a pretrained model, and then the ablation controls (5).

(1) Effect size beyond the two-layer model (W2). In a pretrained LM, the same transplant improves middle-position retrieval by 13.4 points in KV retrieval and by 3.3 points in natural multi-document QA.

  • Setup. We use Qwen3-0.6B-Base, which shows a clear middle deficit, and keep the paper's sink population fixed. We copy the sink-neuron activations of evidence placed first, in all layers, into the same evidence placed in the middle.
  • Controls. Random neuron sets matched in layer and count.
  • Tasks. KV retrieval uses 2,048-token prompts with 408 key–value pairs, with the queried pair placed 1st or 205th. Natural Questions (NQ) is the multi-document QA task of Liu et al. (2024). Each question comes with 10 Wikipedia passages, one of which contains the answer; we place this passage 1st or 5th, evaluate questions the model cannot answer without the passages, and score exact match.
TaskClean first / middle (%)Middle after transplant (%; Δ [95% CI])Gap, clean → after (pp)Random Δ middle (pp; mean ± SD)
KV retrieval (2,000 prompts)99.7 / 78.792.1 (+13.4 [11.7, 15.1])21.0 → 7.6+0.1 ± 2.8 (100 sets)
NQ (2,107 questions)52.7 / 32.836.2 (+3.3 [2.3, 4.4])19.8 → 16.5+0.1 ± 0.4 (10 sets)
  • Why the controlled effect is smaller. In the two-layer model, only first-layer MLP outputs at context positions can reach the final query: second-layer MLP outputs at those positions are never read, and copying them leaves accuracy unchanged. The original intervention is therefore confined to eight neurons of one layer, whereas the pretrained transplant covers the sink population of every layer.

(2) Paired uncertainty and independent training seeds (W2). The controlled-model effect is stable across samples and training runs.

  • Paired CIs. Clean and patched runs share inputs, so we bootstrap over paired samples. On the paper's 2,000 samples, the gap changes by −2.05 [−3.10, −1.05] points, and middle-target accuracy by +1.55 [0.80, 2.30] (44 errors fixed, 13 introduced; McNemar p = 4.7×10⁻⁵). With 10,000 samples, the gap change is −1.71 [−2.22, −1.23].
  • Training seeds. We trained five more checkpoints from scratch, each with a new initialization, data stream and task draw, and reselected the sink neurons in each with the unchanged rule. The effect replicates in all six checkpoints (gap change −2.87 ± 0.74 pp, mean ± SD across checkpoints; all CIs exclude zero). Random sets instead increase the gap (+3.11 pp) and never reach the sink effect.

(3) Transplants in both directions (W2). In the controlled model, the sink state affects the gap more than the original copy suggests: transplanting it in both directions changes the gap by −10.66 points, versus −2.87 for the one-way copy in (2).

Construction. Each transplant gives a pair the sink-neuron activations it would have at the other position. These activations therefore carry the pair's own content and label, whereas the original copy gave the middle pair the first pair's activations.

  • Middle pair ← its own beginning state: we run the middle pair alone as a two-token sequence, which under causal masking gives exactly the first-layer activations it would have as the first pair.
  • First pair ← its own middle state: we place the first pair in the fifth slot of another sample and record its activations there.

Results. Every CI excludes zero, and no random set reaches the sink effect in any checkpoint. First- and middle-target accuracy (%, mean) and changes in the first–middle gap (points, mean ± SD; negative values shrink the gap) across the six checkpoints:

TransplantFirst target (%)Middle target (%)Gap change, sink (pp)Gap change, random sets (pp)
None (clean)86.169.1––
Middle pair ← its own beginning state84.9 (−1.3)71.5 (+2.4)−3.67 ± 0.71−0.46 ± 0.20
First pair ← its own middle state83.5 (−2.7)72.2 (+3.1)−5.75 ± 0.73−1.25 ± 0.46
Both in one forward pass81.8 (−4.4)75.4 (+6.3)−10.66 ± 1.37−1.79 ± 0.61

(4) Separate patching and position-matched donors (Q). The reduction is carried by the label position, and it survives position-matched donors; it is largest when the donor carries the evidence's own content.

  • Separate patching. Copying only the label position reproduces the effect (−2.91 ± 0.69 across the six checkpoints; all CIs exclude zero), whereas copying only the content position has none (−0.02 ± 0.17). By construction, the first and fifth content vectors are identical, so the content copy changes only the position of the same input.
  • Label identity. If the label copy carried over the first label's identity, first-target accuracy would rise and middle-target accuracy would fall. The original copy does the opposite (first −0.89 ± 0.37, middle +1.97 ± 0.43), whereas copies of random neurons widen the gap (+3.11).
  • Position-matched donor, same content. The first transplant in (3) uses this donor: a beginning state that carries the middle pair's own content and label. It changes the gap by −3.67 ± 0.71 points, versus −0.46 ± 0.20 for random sets.
  • Position-matched donor, different content. With another sample's first pair as donor, the gap still falls on average (−1.08, ranging from +0.50 to −2.80) and lies below the random-set mean in all six checkpoints, though its CI excludes zero in only two. Such donors also corrupt the pair's content–label binding. To hold the unrelated content fixed, we compare donors taken from the first and fifth pairs of the same other sample, whose content vectors are identical by construction: the beginning-state donor changes the gap by −0.96 ± 0.66 points relative to the middle-state donor, negative in all six checkpoints (−0.88 [−1.35, −0.41] on 10,000 samples of the original checkpoint).

(5) Ablation controls (W1). With contribution-matched controls, sink neurons are no more important than other neurons with the same early contribution; what distinguishes them is that their contribution is confined to early positions.

  • Purpose of the ablation. Sink neurons are selected for early activity, but selective activity does not guarantee importance: class selectivity, for example, poorly predicts the effect of ablating a unit (Morcos et al., 2018). The ablation therefore tests directly, on held-out documents, whether early prediction depends on these neurons.
  • Contribution-matched controls. We add 20 non-sink sets with the same layerwise counts. They are selected on the localization data alone so that their early projected contribution approximately matches that of the sink set. The relative activation rate is over the first 32 tokens (median over layers).
Neuron setEarly projected contribution (% of sink)Relative activation rateΔNLL, first 32 predictionsΔNLL, positions 129–2047
Sink neurons1007.70.5810.027
Contribution-matched (20 sets)98.3 ± 4.31.40.596 ± 0.1180.206 ± 0.054
Random, layer and count (20 sets)37.7 ± 4.91.30.078 ± 0.0770.038 ± 0.020

Contribution-matched neurons thus have comparable early contributions without the beginning preference that defines sink neurons, and removing them causes a comparable increase in early loss. The two sets differ in where they matter. Removing sink neurons barely affects later predictions, whereas removing the matched neurons does.

  • Scope of the claim. Our objective is a causal link between this positionally asymmetric activity and behavioral position bias, not a complete or unique computation implemented by sink neurons. The ablation shows that early prediction uses this activity; the transplants in (1)–(4) test its behavioral consequence. We will revise §4.3 accordingly.

  • Liu, Nelson F., Kevin Lin, John Hewitt, Ashwin Paranjape, Michele Bevilacqua, Fabio Petroni, and Percy Liang. (2024). [Lost in the Middle: How Language Models Use Long Contexts]

  • Morcos, Ari S., David G. T. Barrett, Neil C. Rabinowitz, and Matthew Botvinick. (2018). [On the importance of single directions for generalization]

I thank the authors for their thorough and thoughtful rebuttal. My concerns have been addressed, and I have therefore decided to increase my overall rating.

Reviewer_1m2yOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper identifies MLP coordinates whose strong activations occur disproportionately often near sequence beginnings. It studies this pattern across pretrained model families, tests its response to useful context through prefix, copy, and evidence-removal experiments, and uses ablation to assess its role in early prediction. In a separate controlled retrieval model, copying initial-position activations to middle positions reduces first-over-middle preference.

Critical strengths
  1. Context interventions provide evidence that coordinates selected by their initial-position activity respond to available information, not just absolute position. This is the paper’s main empirical advance beyond describing positional selectivity.

  2. Analysis across model families, selection checks, and ablation on held-out data give the initial-position pattern a solid empirical basis. The ablation also connects this pattern to prediction quality.

Critical weaknesses
  1. The novelty and explanatory advance are limited by prior work. Voita et al. (2024), §5, p. 1292 already describe neurons active only at the first few positions. The discussion in §2.1 emphasizes broad positional ranges and does not clearly acknowledge this closer observation. The new criterion, broader model analysis, and ablation extend the evidence, but initial-position specialization itself is not new.

    Stolfo et al. (2024), §6 and Appendix I already show that confidence-regulation neurons respond to useful repeated context, including a prefix intervention. The new result here is this sensitivity in coordinates selected by their initial-position activity. Its relation to known confidence regulation remains unexplored. Removing evidence changes activity, while a separate ablation harms early prediction; these results do not establish that the observed modulation enables adaptation to missing information. This limits the explanatory advance and makes the abstract’s claim to “uncover a mechanism for low-context processing” stronger than the evidence supports.

  2. The behavioral result has limited scope, and its robustness is not assessed. The causal evidence comes from a separate small model on a synthetic retrieval task (§6; Appendix H). The intervention affects 12.5% of the first MLP’s coordinates, whereas the main pretrained-model analysis selects 1% per layer. This substantially larger fraction further limits transfer of the behavioral conclusion to the population studied in the main analysis. Changing position bias through activation patching does not establish that neuron sink is a source of retrieval degradation whose correction would improve performance. The connection to middle-position retrieval failures in long-context LMs, and the benefit of correction on natural tasks, remain untested. The practical significance of this direction for mitigating position bias is therefore only weakly supported.

    Results use one training seed and 2,000 examples per condition, without paired confidence intervals for changes in accuracy or the gap. The five random coordinate sets assess variation in coordinate selection on the same model. Given the small absolute change, the lack of paired uncertainty estimates and variation across training runs limits confidence in the robustness of the behavioral result.

Questions for authors
  1. (W1) What evidence distinguishes the proposed explanation of low-context processing from known confidence-regulation processes? In particular, what supports the stronger mechanistic interpretation in the abstract beyond the demonstrated contextual response?

  2. (W2) What motivated selecting 12.5% of the first MLP in the retrieval experiment, compared with 1% per layer in the main analysis? If smaller subsets were evaluated, how sensitive is the behavioral effect to this choice?

  3. (W2) Can the paired evaluation outcomes support confidence intervals for the changes in accuracy and the first–middle gap? Are results from any independently trained retrieval checkpoints available? Stronger evidence is needed to connect this finding to middle-position retrieval failures in long-context LMs. Is there additional evidence supporting this broader interpretation, beyond the current synthetic setting?

6 comments

We thank the reviewer for the careful review. We agree that "uncover a mechanism for low-context processing" suggested more than we show (Q1). The revised abstract states the claim at the level of causal variables, i.e., what engages sink neurons and what depends on them. Beyond the contextual response (§5.2–5.4), this claim rests on three causal tests: ablation at sequence beginnings (§4.3), ablation across matched contexts (response (5)), and transplants into retrieval evidence (responses (1) and (6)). Together with the k-lag experiment (§5.3), in which only the training data differ, these results point to a data-level account rather than a distinct mechanism. We address the concerns below, beginning with new evidence from a pretrained model.

(1) Behavioral evidence in a pretrained language model (Q3, W2). In a pretrained language model, transplanting the beginning state of sink neurons into middle-position evidence improves its retrieval.

  • Setup. We use Qwen3-0.6B-Base, which shows a clear middle deficit, and keep the paper's sink population fixed. We copy the sink-neuron activations of evidence placed first, in all layers, into the same evidence placed in the middle. As the transplant targets known evidence, it serves as a causal probe, not a mitigation method; the revised paper details the donor construction, the replaced tokens, and the question filtering.
  • Controls. Random neuron sets matched in layer and count.
  • Tasks. Key–value retrieval uses 2,048-token prompts with 408 key–value pairs, with the queried pair placed 1st or 205th. For multi-document question answering (QA), we use the 10-passage Natural Questions (NQ) setup of Liu et al. (2024). We place the answer-containing Wikipedia passage 1st or 5th and evaluate questions answered incorrectly without the passages. A generated answer is correct if it contains a normalized gold-answer alias.

Accuracy is in percent; gaps and accuracy changes below are in percentage points (pp). The first–middle gap is first-target minus middle-target accuracy; for this forward transplant, its post-transplant value retains clean first-target accuracy. CIs are 95% paired-bootstrap confidence intervals; SD denotes standard deviation, here across random neuron sets. Values are rounded independently.

TaskClean first / middle (%)Middle after transplant (%; Δ [95% CI])Gap, clean → after (pp)Random Δ middle (pp; mean ± SD)
KV retrieval (2,000 prompts)99.7 / 78.792.1 (+13.4 [11.7, 15.1])21.0 → 7.6+0.1 ± 2.8 (100 sets)
NQ (2,107 questions)52.7 / 32.836.2 (+3.3 [2.3, 4.4])19.8 → 16.5+0.1 ± 0.4 (10 sets)

(2) Robustness of the controlled-model result (Q2, Q3). The toy-model effect is stable across samples and training seeds, and varies systematically with set size.

  • Paired CIs. Clean and patched runs share inputs, so we bootstrap over paired samples. On the paper's 2,000 samples:

    • the gap changes by −2.05 [−3.10, −1.05] points;
    • middle-target accuracy changes by +1.55 [0.80, 2.30] (44 errors fixed, 13 introduced; McNemar p = 4.7×10⁻⁵), and first-target accuracy by −0.50 [−1.15, 0.15].

    With 10,000 samples, the gap change is −1.71 [−2.22, −1.23].

  • Training seeds. The effect replicates in six independently trained checkpoints (gap change −2.87 ± 0.74 pp, mean ± SD across checkpoints; all CIs exclude zero). Random sets instead increase the gap (+3.11 pp) and never reach the sink effect.

  • Set size (Q2). With 64 neurons per layer, 1% would be less than one neuron, so we used 8.

    • Copying k = 1/2/4/8/16 sink neurons changes the gap by −0.43/−0.97/−1.77/−2.87/−3.27 points on average.
    • At k = 32, half the layer including weakly enriched neurons, the gap increases (+4.67), though far less than with the other half (+19.78).

(3) Novelty relative to initial-position neurons (W1). We agree that sink neurons largely coincide with the initial-position neurons of Voita et al. (2024). What our work adds is their sensitivity to context, which the training data shape, and its functional consequences.

  • Overlap. In their §5, Voita et al. report neurons that "never activate except for a few first token positions". Under our event definition, their criterion selects 84% of our sink neurons. We will acknowledge this in §2.1.
  • Context, not position. At fixed positions, the activity of these neurons follows the availability of useful context (§5.2–5.4). The training data shape this pattern: in the k-lag copy task, where only the data change and context becomes useful after the first k tokens, the sink span grows from 2.25 to 38.75 positions as k increases from 2 to 64, while an IID control shows only a brief initial elevation (§5.3). Neuron sink thus tracks how far into the sequence the training data leave predictions without useful context.
  • Function. Voita et al. left the function open, noting that the outputs "do not seem to encode meaningful distributions over vocabulary" (their §5.5). From the low-context perspective, these neurons support early prediction (§4.3), contribute more when context is missing (response (5)), and affect later retrieval through token states (responses (1) and (6)).

(4) Relation to confidence-regulation neurons (Q1). Our analysis complements that of Stolfo et al. (2024). They focus on the readout, i.e., how final-layer entropy and token-frequency neurons adjust the output distribution of the current prediction. The effects of sink neurons are not limited to this readout: they extend beyond the final layer and beyond the current prediction.

Beyond the final layer. Confidence-regulation neurons reside in the final layer, whereas the sink pattern appears in every layer.

Beyond the current prediction. Final-layer neurons at a token affect only the prediction at that token, whereas the sink state of a token also affects later predictions: transplanting this state at evidence tokens changes how a later query retrieves them (responses (1) and (6)).

Since confidence-regulation neurons also respond to useful context, we view final-layer confidence regulation as one part of low-context prediction, through which sink neurons may partly act, rather than an alternative account of it, and will discuss this relation in §2.1.

(5) Larger contribution when context is missing (Q1). A new experiment links the two results the reviewer noted: the neurons whose activity rises when context is missing also contribute more to prediction in that condition. The paper shows these results separately: missing context raises sink activity (§5.2–5.4), and ablating sink neurons harms early prediction (§4.3). If the model relies more on these neurons when context is missing, ablating them should cost more in that condition. We test this by combining the two manipulations.

  • Design. The same 1,024-token target passage follows either its own preceding text (related prefix) or an equally long passage from another document (unrelated prefix), as in §5.2, so target tokens and positions are identical. We zero the sink neurons at the first 32 target positions in all layers and measure the increase in next-token loss.
  • Result. Ablation raises the loss by 0.044 nats/token after unrelated prefixes but only 0.011 after related ones.
  • Controls. This unrelated-minus-related difference exceeds that of non-sink neurons matched on activity and projected contribution by 0.031. The matched neurons are selected on 128 separate pairs. The result replicates with same-domain unrelated prefixes (0.030).

Scope. Zero ablation removes the whole contribution of these neurons, not only the part induced by missing context. This result thus shows greater reliance on sink neurons when context is missing, not that the modulation itself mediates adaptation; the revised §5 keeps this distinction.

(6) Neuron sink contributes to position bias (W2). We claim that neuron sink contributes to position bias, not that it is the sole source. The remaining first–middle gap after the own-state transplants (§6.1–6.2) is consistent with other known sources such as causal masking and positional encoding (Wu et al., 2025).

  • Liu, Nelson F., Kevin Lin, John Hewitt, Ashwin Paranjape, Michele Bevilacqua, Fabio Petroni, and Percy Liang. (2024). [Lost in the Middle: How Language Models Use Long Contexts]
  • Stolfo, Alessandro, Ben Wu, Wes Gurnee, Yonatan Belinkov, Xingyi Song, Mrinmaya Sachan, and Neel Nanda. (2024). [Confidence Regulation Neurons in Language Models]
  • Voita, Elena, Javier Ferrando, and Christoforos Nalmpantis. (2024). [Neurons in Large Language Models: Dead, N-gram, Positional]
  • Wu, Xinyi, Yifei Wang, Stefanie Jegelka, and Ali Jadbabaie. (2025). [On the Emergence of Position Bias in Transformers]

Thank you for the additional experiments. The uncertainty estimates, independent training runs, and pretrained-model results address my concerns about robustness and the synthetic setting. The matched-context ablation also strengthens the functional evidence. Two questions remain:

  1. Intervention strength. Random controls match layer and neuron count, but do they produce comparable changes in the MLP output? For example, rescaling the middle evidence’s existing sink activations to match the transplant’s MLP-output change in norm could distinguish the specific beginning state from simply amplifying the correct evidence. Has such a control been evaluated?
  2. Connection between the effects. The new ablation shows greater reliance on sink neurons when useful context is missing, while the transplants establish an effect on later retrieval. What evidence connects the context-dependent modulation itself to the retrieval effect, beyond demonstrating both properties in the same population?

Given the acknowledged overlap with prior work and the lack of a practically applicable intervention, these clarifications would help me assess the overall contribution.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.