acceptodds
Under review as a conference paper at ICLR 2027

Lost in the Gaps: How Non-Lexical Filler Disrupts Belief Tracking in Language Models

Abstract

Unlike natural-language filler, repetitive, meaningless tokens such as a sequence of spaces can significantly degrade Language Models' (LMs) task accuracy when inserted into a prompt. Why does this happen? We investigate this question using a belief tracking task, in which a model must infer an agent's belief about the world, as a case study. Through activation patching and causal interventions on attention across three LMs, we find that filler sequences that are associated with performance degradation receive markedly high attention weights. Activation patching traces these failures to early MLP layers processing the filler: exchanging their outputs between harmful and harmless fillers suffices to remove or induce the errors. Patching individual MLP neurons further localizes these effects to a single neuron in Llama-3.1-8B and four neurons in Qwen2.5-14B, whereas in Qwen2.5-7B a few neurons suffice to remove the errors but inducing them requires a more distributed set. Our findings connect filler-induced failures to early MLP computations and downstream attention interactions, providing a mechanistic account of how semantically uninformative inputs disrupt belief tracking.

Primary area:
interpretability and explainable AI
Keywords:
Large Language Models, Mechanistic Interpretability, Token Repetition, Attention Allocation, Contextual Robustness, Activation Patching

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_uWrYOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

This paper studies the effect of repetitive task-irrelevant filler tokens. With the assumption of such fillers degrades model's retrieval capabilities, the authors generates false-belief-based ToM tasks using the CausalToM templates. The authors tested three models from the Llama3.1 and Qwen2.5 model families on the ToM task. Results show that fillers such as whitespace and dots leads to more severe performance degradation than other fillers such as irrelevant or shuffled text. Further investigation show that such fillers capture disproportionate attention weights, thereby diluting the useful information. The authors show that causal attention intervention effectively mitigate the effect of fillers and trace the root cause to abnormal computations in early layers. Further, neuron-level patching demonstrate the the effect of fillers can be largely mitigated by applying patching to a small number of neurons (1 or 4 depending on the model).

Critical strengths
  • The phenomenon of distorted attention pattern (activation value) introduced by certain fillers, if proven to be generalizable to a wide range of tasks and models, is interesting and would spark more research efforts in understanding and mitigating the phenomenon.
  • Activation patching studies are conducted to probe the causal relationship between the observed phenomenon and certain components/aspects of the models.
  • The authors included a range of fillers, covering various aspects such as whitespace, punctuation, and random words.
  • The paper is well-written and follows a good logical flow. Starting from the phenomenon, the authors then march into causal attention intervention, critical neuron localization and patching.
Critical weaknesses
  • My biggest concern is the limited scope of this study.
    • The tested models are very out-dated (e.g., Llama-3.1 and Qwen 2.5), leading to limited insights to current models. Will we observe a similar phenomenon in more recent models, particularly the large reasoning models?
    • Only one Theory of Mind task is used for analysis. Would similar phenomenon be observed in other tasks? While the authors mentioned that ToM task is suitable as it demands contextual information retrieval, many other tasks are also applicable including prevalent ones such as GSM8K, which has a shorter context, or LOCOMO which support long context. Even within ToM, data such as FANToM and OpenToM can also be used to validate the generalizability of the identified phenomenon.
  • With limited model tested, the trend of the phenomenon is unclear. For instance, Table 1 shows that Qwen2.5-14B is completely prone to the whitespace attack. Does this suggest that only smaller model is affected by repeated whitespaces?
  • While the authors have pointed many findings and phenomenon, no root cause or in-depth analysis was given, making the overall analysis superficial and insights limited. For instance, why would fillers affect specific MLP neurons in the shallow layers?
Questions for authors
  • Does the stated phenomenon generalize to ToM tasks other than CausalToM or tasks outside the domain of ToM? Is the stated phenomenon observed in more recent models. It would also be interesting to see the behavior of Large Reasoning Models with fillers.
  • Is there ablation study that test placing the fillers at a different location other than the given [Instruction] [Story] [Filler] [Question] [Answer] template?
  • If chosen examples where the model get wrong at first, would there be cases where adding fillers drive the model to give accurate answer? If so, what would be the explanation for such phenomenon?
  • For the remaining erroneous cases after module intervention or neuron patching, what are the (potential) causes of their persistent errors?
5 comments
AuthorsOpenReview ↗

Response For Q1

Q1: Does the stated phenomenon generalize to ToM tasks other than CausalToM or tasks outside the domain of ToM? Is the stated phenomenon observed in more recent models. It would also be interesting to see the behavior of Large Reasoning Models with fillers.

Thank you for raising these points, which also provide clear directions for our future work.

For other tasks: Two sets of results in our paper show that this phenomenon does not depend on a specific prompt. In Appendix E, we replaced the residual stream at the filler positions in the original prompt ([Story][Filler][Question][Answer]) with that from a prompt in which the same filler follows an irrelevant natural context ([Geology][Filler]). Accuracy still dropped (Table 7), indicating that the erroneous representations arise from the model's processing of the filler itself rather than from the preceding context. We also tested factual questions (e.g., "What does the mug contain?") that do not require tracking an agent's belief, and observed the same pattern (Appendix B, Table 4). In addition, Du et al. [1] found that whitespace filler hurts model performance on math, question answering, and coding tasks.

We chose CausalToM [2] because its internal mechanism is already well characterized (Section 1), all three models reach 100% accuracy without filler, and the answer is a single token, which makes the masking and patching experiments straightforward. In contrast, FANToM, OpenToM, GSM8K, and LoCoMo involve multi-step reasoning, which makes it difficult to align a single readout position for neuron-level localization. They are well suited to behavioral evaluation, which we leave to future work. We will revise the manuscript and expand this discussion in the future work section.

For more recent models and reasoning models: Activation patching requires reading and modifying a model's internal states, so it can only be done on open-weight models. Llama-3.1 and Qwen2.5 are common choices in mechanistic interpretability; the original CausalToM study [2] used Llama-3/3.1 and Qwen2.5. Because of limited computational resources, we did not test more recent or larger models, or reasoning models. However, existing evidence suggests that the mechanisms underlying this phenomenon are shared across models: massive activations [3] and the mechanism that forms attention sinks [4] have been found across model families and sizes. Our own results also show that Qwen2.5-7B and Qwen2.5-14B are sensitive to different fillers, which suggests that this sensitivity is learned during training. We therefore expect similar neurons to exist in newer models, although the token that triggers them may differ.

For reasoning models, under causal attention the activations at the filler positions depend only on the preceding input and are not affected by any token generated afterwards. The abnormal activations we observe will therefore form regardless of whether the model later performs explicit reasoning such as CoT. NoLiMa [5] also reports that models with reasoning capabilities struggle to maintain performance in long contexts. How multi-step reasoning interacts with these early representations remains to be tested directly. We will discuss this in Section 7 and list it as future work.

  • [1] Du et al. Context Length Alone Hurts LLM Performance Despite Perfect Retrieval. Findings of EMNLP 2025.
  • [2] Prakash et al. Language Models use Lookbacks to Track Beliefs. ICLR 2026.
  • [3] Sun et al. Massive Activations in Large Language Models. COLM 2024.
  • [4] Yona et al. Interpreting the Repeated Token Phenomenon in Large Language Models. ICML 2025.
  • [5] Modarressi et al. NoLiMa: Long-Context Evaluation Beyond Literal Matching. ICML 2025.
AuthorsOpenReview ↗

Response For Q2

Q2: Is there ablation study that test placing the fillers at a different location other than the given [Instruction] [Story] [Filler] [Question] [Answer] template?

Thank you for the question. In our experiments the filler is fixed between the story and the question, and we did not vary its position. However, prior work shows that the phenomenon does not depend on this particular position. Du et al. [1] tested the effect of filler position directly. In addition to the [Evidence] [Whitespace] [Question] layout, which is the same as ours, they moved the whitespace filler before the evidence, i.e., [Whitespace] [Evidence] [Question], so that the evidence is adjacent to the question. Model performance on math, question answering, and coding tasks still dropped under this layout.

  • [1] Du et al. Context Length Alone Hurts LLM Performance Despite Perfect Retrieval. Findings of EMNLP 2025.
AuthorsOpenReview ↗

Response For Q3

Q3: If chosen examples where the model get wrong at first, would there be cases where adding fillers drive the model to give accurate answer? If so, what would be the explanation for such phenomenon?

Our evaluation set contains only prompts that all three models answer correctly without filler, so that every model starts from the same baseline. It therefore does not include prompts that are answered incorrectly without filler. We elaborate below.

We do not expect the harmful fillers in our work to correct errors systematically. These fillers draw the attention of post-filler positions away from the story. For example, under dot filler in Llama-3.1-8B, the attention from the readout position to the correct answer token falls to 1% of its no-filler value (Section 3). For a sample that is already answered incorrectly without filler, the model would likewise read less information from the story once the filler is added. A few samples might land on the gold answer by coincidence because the output distribution shifts, but this would be chance, not an improvement in belief tracking.

Prior work has reported two cases in which fillers help, and both rely on mechanisms different from ours. First, for non-repeated irrelevant text, Cuconasu et al. [1] found that adding random documents to the prompt can improve question-answering accuracy by up to 35%. Second, on whether repeated symbols can provide extra computation, Pfau et al. [2] showed that models can be trained to use repeated dots for hidden computation, but that learning this requires specific, dense supervision. They also note, citing earlier results, that off-the-shelf models do not benefit from filler tokens. All models in our work are off-the-shelf models without such training.

  • [1] Cuconasu et al. The Power of Noise: Redefining Retrieval for RAG Systems. SIGIR 2024.
  • [2] Pfau et al. Let's Think Dot by Dot: Hidden Computation in Transformer Language Models. COLM 2024.
Reviewer_W7ZTOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies a practical problem: how repetitive, meaningless fillers disrupt language models' performance on a belief-tracking task. The authors conduct experiments across three models and find that excessively long fillers can substantially degrade model performance. They further identify that this degradation stems from a disproportionately large share of attention being allocated to filler tokens, and that blocking attention to these tokens can restore model performance. Finally, through activation patching, they attribute this phenomenon to MLPs in the early layers.

Critical strengths
  • The paper is generally well-written and easy to follow.
  • The experimental designs are clear and broad. Its logical structure is clear, with self-contained and coherent analyses throughout.
Critical weaknesses
  • The sample size is relatively small. All experiments are conducted on one self-constructed dataset consisting of only 100 samples. Extending the experiments to larger and more diverse datasets would be crucial to establish the findings of the paper. Besides, the experimental setting studied in the paper does not quite align with the framing used to motivate the study. Specifically, the repeated fillers are constructed by directly concatenating individual tokens, whereas in practice, a long sequence of repeated characters (for example, 2,000 consecutive spaces or periods) would be tokenized into only 16–32 tokens by the Qwen2.5-7B tokenizer. The controlled token-level findings remain valid, but their applicability to ordinary text inputs remains unverified.
  • The mechanistic findings are somewhat incremental. Prior work [1] has already established connections between repeated tokens, attention sinks, and early-layer MLPs. This substantially weakens the novelty of the paper, particularly contribution (3). The authors should more clearly articulate how their findings differ from and relate to prior work.

[1] Interpreting the Repeated Token Phenomenon in Large Language Models. ICML 2025.

Questions for authors

Q1: Could you more clearly explain how your findings differ from the attention-sink/repeated-token mechanistic account established in [1] and justify the novelty of your findings?

Q2: Could you report results on

  • non-repeated non-lexical sequences; and
  • more repeated lexical tokens? (Do ' hi', slash and the text fillers stay harmless at K = 4,000 or beyond?)

Q3: Could you extend the experiments to larger datasets to further justify the generalizability of your findings?

Q4: In §4, which token positions does each mask cover? Appendix F supports the readout route, but if “both” includes positions beyond the question/readout union, Fig. 5 does not establish that the question-token route alone suffices to cause errors in Llama. Could you clarify the evidence or qualify this claim?

[1] Interpreting the Repeated Token Phenomenon in Large Language Models. ICML 2025.

5 comments
AuthorsOpenReview ↗

Response For Q1

Q1: Could you more clearly explain how your findings differ from the attention-sink/repeated-token mechanistic account established in [1] and justify the novelty of your findings?

Thank you for pointing out this connection. We will add discussion at the Introduction part and refine our (3) contribution.

The key difference between our work and [1] is the role of repeated token sequences which we called filler. In [1] they required LMs to repeat specific words and figured out the outputs were broken. However, in our work, we insert the filler between the [Story]and[Question], make them as a whole prompt and let the LMs inferred(Chapter 2 Figure1).

Furthermore, [1] the mechanism of how repeated tokens would trigger attention - sink. In our work, we further examined how the filler affected the accuracy of the task through the question position and read-out position: The masking experiments identify the pathway of filler affecting the answer(Chapter 4 Figure5). Meanwhile, we found that double switching the MLP output between harmful and normal filler could increase or decrease the accuracy(Chapter5.1 Figure6d-f). Therefore, our contribution mainly focuses on establishing the causal relationship between the filler and decreasement of accuracy and identifying the pathways through position specific intervention.

  • [1] Interpreting the Repeated Token Phenomenon in Large Language Models. ICML 2025.
AuthorsOpenReview ↗

Response For Q2

Q2: Could you report results on non-repeated non-lexical sequences; and more repeated lexical tokens? (Do ' hi', slash and the text fillers stay harmless at K = 4,000 or beyond?)

Thank you for the insightful questions, which helped us clarify the scope of our findings.

For non-repetitive and non-lexical tokens: We do not test non-repeated and non-lexical sequences, but prior work and our result show a consistent expectation. Cuconasu et al. [1] found that adding irrelevant context to a prompt sometimes could improve the models' performance. Sun et al. [2] and Yu et al. [3] showed that disproportionate attention concentrates on specific tokens: massive activations appear on the initial token and on dots or newlines, and attention sinks also appear on the less meaningful tokens later in the sequence. This aligns with our observations. Yona et al. [4] further showed that this mechanism is triggered when the first attention layer handles later tokens in the same way as the first token: their cluster attack is built from tokens of the same cluster, which are processed by the same first-layer attention head, and it induces attention sinks without repeating tokens. Generally non-repeated non-lexical sequences do not meet this condition, so we expect them to behave much like anagram. We will add this discussion to the revised manuscript and clarify the scope of our experiments.

For longer length: We set 2000 as the upper bound of the filler length because our goal is to compare performance across different filler types. NoLiMa [5] reported that model performance already drops considerably at context lengths of 2K to 8K tokens. RULER [6] and Du et al. [7] also found that both natural text and whitespace degrade model performance as the context grows longer. If all fillers degraded at longer lengths, it would be hard to tell whether the degradation is caused by length itself or by the early neuron mechanism we identify. Within the range we tested, the harmless fillers show no significant change in either neuron activation or model performance: the activation of the top 1 neuron stays between 0 and 0.03 as the length increases from 1400 to 2000 (Table 17), and the three text fillers remain 90 to 100% accuracy at K = 2,000 (Table 1). We will list the effects of longer fillers, and the distinction between general length effects and token specific effects, as future work.

  • [1] Cuconasu et al. The Power of Noise: Redefining Retrieval for RAG Systems. SIGIR 2024.
  • [2] Sun et al. Massive Activations in Large Language Models. COLM 2024.
  • [3] Yu et al. Unveiling and Harnessing Hidden Attention Sinks: Enhancing Large Language Models without Training through Attention Calibration. ICML 2024.
  • [4] Yona et al. Interpreting the Repeated Token Phenomenon in Large Language Models. ICML 2025.
  • [5] Modarressi et al. NoLiMa: Long-Context Evaluation Beyond Literal Matching. ICML 2025.
  • [6] Hsieh et al. RULER: What's the Real Context Size of Your Long-Context Language Models? COLM 2024.
  • [7] Du et al. Context Length Alone Hurts LLM Performance Despite Perfect Retrieval. Findings of EMNLP 2025.
AuthorsOpenReview ↗

Response of Q3

Q3: Could you extend the experiments to larger datasets to further justify the generalizability of your findings?

Thank you for pointing that out! We agreed that the sample size is not large enough and will discuss it in our limitation section.

Sample size and generalizability: Due to computational resource constraints, our evaluation set uses 100 prompts from the CausalToM [1] template with 95% paired bootstrap intervals over 2,000 resamples (Sections 2.1, 5.1; Appendix G), matching the scale of the original template study (Prakash et al. [1] ran interventions on 80 examples). In Appendix E, we replaced the residual stream at the filler positions in the original prompt ([Story][Filler][Question][Answer]) with that from a prompt keeping the same filler preceded by an irrelevant natural context ([Geology][Filler]), and found that it still degraded performance (Table 7), showing that these erroneous representations arise from the model repeatedly processing the filler rather than from the preceding context. Still, we agree that our conclusions should be evaluated on larger benchmarks such as RULER [2] and NoLiMa [3], as we already discuss in Section 7 (Limitations and Future Work).

Tokenization (Weakness 1): Your observation is correct: we also noticed that consecutive dots and whitespace get merged by the tokenizer, making the actual token length much shorter than the character length. Precisely because we observed this phenomenon, we concatenated single token IDs directly to ensure that the exact token length K remains identical across all filler conditions.

  • [1] Prakash et al. Language Models use Lookbacks to Track Beliefs. ICLR 2026.
  • [2] Hsieh et al. RULER: What's the Real Context Size of Your Long-Context Language Models? COLM 2024.
  • [3] Modarressi et al. NoLiMa: Long-Context Evaluation Beyond Literal Matching. ICML 2025.
Reviewer_oqa4OpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper investigates how and why language models tend to degrade in performance (for example when it comes to entity binding) when the input prompt is polluted by many (like 100s-1000s) of "filler" tokens like white space, punctuations, etc. It first describes and characterizes the phenomenon on small (7B-14B) open-source models, then extracts specific mechanisms (in particular, neurons) that explain the degradation.

This is a good interpretability paper. It is clearly written, illustrated with examples, and easy to follow. The experimental design is sound, relying on common models (Qwen, Llama), datasets (CausalToM), and interp techniques (causal interventions like activation patching). The findings are interesting, in particular that only a handful of neurons can explain performance collapse. While I think this is work worth publishing, my concern is that it might be a little to narrow and incremental for ICLR. An interpretability workshop might be a better venue.

Critical strengths
  • The paper is clearly written and easy to follow.
  • The experimental process is sound and follows well-established interpretability approaches.
  • The findings are interesting, and add to our (scattered) understanding of how models process information and, in some cases, break down under unusual inputs.
Critical weaknesses

The main weakness of this paper is that the scope is rather limited and the practical implications seem modest. Specifically:

  • the paper reports performance collapse starting at around 1000 filler tokens; that's a lot, and makes it a unrealistic situation. Of course I guess the point of the paper is to understand why this collapse happens, rather than correcting a common failure mode.
  • results are limited to 3 small, rather old models. Of course using larger models requires significantly more compute, which many researchers don't have access to. So in itself this is not something to be critical of, but it contributes to the other points I am making here that suggest there is not much practical implication to this study.
  • while a robust study, it doesn't really bring anything new in terms of methodology or dataset that could be useful to the field more broadly
Questions for authors
  • Could you discuss the broader implications, fundamental and/or practical, of your findings? Eg, how does it improve our understanding of attention and failure models in LLM, beyond this one example that you investigate.
  • I find the "single-neuron" effect particularly intriguing. Maybe there would be a bit more to say about it, for example are there other known examples (in the literature) where single-neurons can cause so much disruption in a model?
4 comments
AuthorsOpenReview ↗

Response For Q1

Q1: Could you discuss the broader implications, fundamental and/or practical, of your findings? Eg, how does it improve our understanding of attention and failure models in LLM, beyond this one example that you investigate.

We appreciated the reviewer's valuable feedback and for highlighting the clarity and findings of our work.

Thank you for mentioning this issue. We agree that a sequence of 1000 spaces rarely appears in manual input in daily life. As the reviewer pointed out, our work does not aim to solve an error, but to use this controlled setting to illustrate the reason why meaningless filler degrades model performance, while natural filler does not. However, the internal patterns we discovered are meaningful in fundamental and practical ways:

Fundamental Meaning

Attention diversion rather than length: Prior work found that inserting whitespace degrades LLM performance due to length itself [1] (this indeed happens at much longer inputs). However, with the position indices matched at 2,000 tokens, inserting irrelevant natural text is fine, whereas repetitive meaningless token would influence performance. This shows that within the context window, model collapse is sometimes caused not by length itself, but by the filler attracting attention.

A few neurons grab attention: The filler itself does not contain any information; the reason why performance becomes worse is that neuron(s) in a single early MLP layer react to this input and attract attention to the filler part. These subtle variations accumulate in the early layers as the filler length increases, we explain that when exceeded a threshold causes the collapse.

The failure mechanism is not limited to a single task: In Appendix E, we replaced the residual stream at the filler positions in the original prompt ([Story][Filler][Question][Answer]) with that from a prompt keeping the same filler preceded by an irrelevant natural context ([Geology][Filler]), and found that it still affected performance (Table 7), proving that these erroneous representations arise from the model repeatedly computing the filler rather than from the preceding context. Besides, we also tested factual questions (e.g., "What does the mug contain?") that do not require the model to track an agent's belief. The results showed the same trend as ToM's (Table 4), demonstrating the generalization of this fragility.

Sensitivity is learned during training and traceable to the pretraining process: In our experiments, Qwen2.5-7B and 14B share the same tokenizer, yet fail exclusively on whitespace and dots, respectively (Table 1). In their early MLP layers, we localized the neuron with the largest difference in output contribution between harmful and harmless fillers (e.g., 7B Layer 0's #9335 and 14B Layer 4's #11696; Table 11, Table 17); when processing the harmful filler, these neurons show a large activation, whereas under the harmless filler they do not. These findings encourage future work to compare base and fine-tuned checkpoints [2] and track the development of these neurons during training (similar to how prior work has studied how attention sinks form during pretraining [3]), so that we can address these issues at their root.

Practical Meaning

Formatting issues from copy paste: It is rarely possible for a person to manually type thousands of dots or spaces, but copying formatted text from web pages or PDF files can easily raise this issue. Text in PDF files is hard to extract faithfully, which has motivated dedicated recognition models [4] (e.g., pasting multiple pages of a PDF table of contents filled with leader dots). These fillers are harmless to humans, yet they can degrade model performance, which is also supported by prior work on prompt formatting sensitivity [5]. Thus, uncovering why models collapse under such inputs is important.

  • [1] Context length alone hurts LLM performance despite perfect retrieval. Findings of EMNLP 2025.
  • [2] Fine-Tuning Enhances Existing Mechanisms: A Case Study on Entity Tracking. ICLR 2024.
  • [3] When Attention Sink Emerges in Language Models: An Empirical View. ICLR 2025.
  • [4] Nougat: Neural Optical Understanding for Academic Documents. ICLR 2024.
  • [5] Quantifying Language Models' Sensitivity to Spurious Features in Prompt Design or: How I Learned to Start Worrying About Prompt Formatting. ICLR 2024.
AuthorsOpenReview ↗

Response For Q2

Q2: I find the "single-neuron" effect particularly intriguing. Maybe there would be a bit more to say about it, for example are there other known examples (in the literature) where single-neurons can cause so much disruption in a model?

Thank you for focusing on this interesting phenomenon. Yes, some previous studies have revealed that a single neuron or a few neurons can significantly affect a model's performance, whereas our work shows the exact opposite side of this phenomenon:

Connection to prior work: Previous studies found that LLMs exhibit outlier dimensions with large magnitudes across the whole token sequence [1], and more recent work pointed out that a single "super weight" [2] in an early MLP layer produces "massive activations" [3] on a few specific tokens such as the first token, which concentrate attention on these tokens; if we set these dimensions or weights to zero, the model's performance collapses. Our experiments reach the opposite conclusion. We showed that the localized early MLP neurons have very little activation under harmless filler (Table 17). However, when processing harmful filler, these neurons become significantly activated (e.g. neuron #11696 in Layer 4 of Qwen2.5 14B rises from 0 to 267). So the key difference is: previous work found shutting down a critical component could make the model fail, whereas our work found that harmful filler activates a normal component and causes the model to collapse.

Why a single neuron has a huge effect: Out of a large number of neurons in the early MLP layer, replacing just the top 1 neuron produces 53%–64% of the output change from replacing the entire layer (Appendix H, Table 16). To further verify this finding, we randomly picked a neuron and amplified its change to match the exact same output change, or added random noise of the same size; none of these baselines could restore accuracy (Figure 7). This illustrates that this specific neuron pulls downstream attention toward the filler. For instance, in Llama 3.1 8B, when we replaced this neuron's activation with its harmless counterpart, the attention from the readout position to the filler decreased from 92% to 13% (Table 19), and accuracy recovered from 12% to 94%.

  • [1] LLM.int8(): 8-bit Matrix Multiplication for Transformers at Scale. NeurIPS 2022.
  • [2] The Super Weight in Large Language Models. arXiv preprint arXiv:2411.07191, 2024.
  • [3] Massive Activations in Large Language Models. COLM 2024.

Finally, we sincerely appreciate the reviewer for the constructive and helpful feedback. In our revised manuscript, we will expand our discussion on both the theoretical and practical implications of our findings, and discuss more about the single neuron effect in the main text. If there are any further questions, we would be glad to provide further discussion.

Thank you to the authors for a thorough response.

At this point, I stand by my original assessment: I think this paper is robust and brings a valuable contribution to the field of interpretability. It still seems too limited in scope and significance to be of interest to the broader ICLR community, which is why I maintain my (overall positive) assessment at "weak rejection" (for the main conference).

AuthorsOpenReview ↗

Thank You and a Brief Clarification on Novelty and Scope

Thank you for taking the time to read our response, and we sincerely appreciate your positive feedback on the robustness of our work and its contribution to interpretability. We fully respect your assessment! Regarding your remaining concern, we would like to give a brief clarification for you and the Area Chair to consider:

  1. Long-context degradation and abnormal internal activations are widely studied across the community. Prior work has largely attributed whitespace caused degradation to length itself [1], or focused on model collapse when critical weights or activations are removed [2, 3]. By contrast, we show that at the exact same length, whether a model fails depends on specific tokens (with different sensitive tokens even across model sizes within the same family), and that the errors come from activating a few early neurons that are nearly silent under harmless fillers.
  2. Because the failure concentrates on a few neurons, intervening on just a few neurons is enoght to imporve accuracy without changing how the model handles long inputs . This also mentioned a simple safeguard: filtering long runs of sensitive tokens before the input reaches the model and may have implications for the reliability of deployed models.
  3. We controlled the task and model scale so we could conduct our causal interventions down to individual neurons. Trading breadth for mechanistic depth in a controlled setting has several precedents at ICLR, including belief tracking on 80 examples from the same template [4], the IOI circuit study on GPT-2 small [5], modular addition on a one-layer Transformer [6], and entity tracking on 7B-scale models [7].

Thank you again for your valuable time and constructive discussion.

  • [1] Du et al. Context Length Alone Hurts LLM Performance Despite Perfect Retrieval. Findings of EMNLP 2025.
  • [2] Yu et al. The Super Weight in Large Language Models. arXiv:2411.07191, 2024.
  • [3] Sun et al. Massive Activations in Large Language Models. COLM 2024.
  • [4] Prakash et al. Language Models use Lookbacks to Track Beliefs. ICLR 2026.
  • [5] Wang et al. Interpretability in the Wild: a Circuit for Indirect Object Identification in GPT-2 small. ICLR 2023.
  • [6] Nanda et al. Progress Measures for Grokking via Mechanistic Interpretability. ICLR 2023.
  • [7] Prakash et al. Fine-Tuning Enhances Existing Mechanisms: A Case Study on Entity Tracking. ICLR 2024.

Comments on the paper

1 comment
AuthorsOpenReview ↗

General Response

We sincerely thank all reviewers for their insightful suggestions and for their recognition of our work. We have uploaded a revised manuscript and added Appendix I, which lists every part we revised. We list the key clarifications here for the reviewers and the AC.

  1. Why the model fails (oqa4-Q1). At a matched length of 2,000 tokens, natural text is harmless while specific repeated tokens cause errors; such runs arise in practice when text is copied from PDFs or web pages.
  2. How our findings differ from prior work (oqa4-Q2; W7ZT-Q1). Prior work shows that removing a critical component collapses a model, and Yona et al. (2025) explain how repeated tokens form attention sinks. In contrast, we show that harmful fillers activate one to four early MLP neurons that are nearly silent under harmless fillers, and we establish their causal effect on task accuracy and the pathway behind it.
  3. How the filler is built (W7ZT-W1). We concatenate single token IDs so that the length K is identical across filler conditions.
  4. Revised conclusion about Llama (W7ZT-Q4). We defined the positions each mask covers and revised the claim: in Llama, the effect is carried by several post-filler positions, including the readout position.
  5. How general the findings are (W7ZT-Q2/Q3; uWrY-Q1–Q3). The same pattern holds for factual questions that require no belief tracking and when the filler follows an unrelated context (Appendix B and E), and prior work reports consistent effects on other tasks and filler positions. Section 7 now clarifies the scope of our controlled setting and outlines extensions to further benchmarks and models.

Detailed responses are under each review.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.