- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
The paper investigates the problem of distractors appearing in the context of language models. The authors argue that a certain number of attention heads attend more to this distracting context while reducing their contribution to the correct answers (intrusion heads), and propose a remedy that masks these heads’ attention to the context. In simple math and question answering tasks, across 4 language models, the authors report that the remedy recovers about 57% of the accuracy drop from distractors.
Critical strengths
- How to avoid irrelevant context tokens adding noise is an interesting question.
- Some figures are nice (eg Figure 4)
Critical weaknesses
- The paper is not well written. Besides how unclear some parts are individually, it overall lacks a coherent and well thought structure and flow. The language is not polished at all, and it is frequently hard to parse. In addition, the paper is overly complex to understand given the simplicity of the main idea. Overall the paper is far from being the best version of itself from a clarity and flow point of view.
- The paper lacks clarity and rigor in the definitions, in the equations (see question about Equation 3, for an example), in the experimental setup of each experiment (almost never provided in the main paper). Overall, it is very hard from a careful read to understand what is going on, besides the overall concept of “finding a way to silence the contribution of irrelevant tokens”.
- Significant parts of the paper are not backed by results nor properly explained experimental setups within the main paper. As an example, section 3 seems to show a lot of findings without showing a single numerical result (besides the last paragraph out of 5), nor clarifying what the experimental setup is and clearly stating what the take away is. It is not very informative and also hard to verify without spending a considerable amount of time looking at the appendix. If the authors believe these paragraphs should be part of the main paper, then they should be self-contained and should not be necessary to go to the appendix to understand basic experimental setup explanations or the results themselves. Otherwise, the entire section should be moved to the appendix.
- Related to the previous point, the appendix is used way too much. Most paragraphs have references to at least one appendix paragraph. Sometimes more. The main paper refers to the appendix twenty-eight times. Overall, the main paper is far from being self-contained within the page limit.
- The paper only plays with very simple tasks, and it is unclear how much any of these findings transfer to real world settings, and how to operationalize them without these controlled settings.
- Some results reported in the main paper are quite odd: for example, Figure 7 shows a clean (no distractors) performance that changes across the number of distractors.
Questions for authors
- Why does Equation 3 need to contain the nctx quantity? Can you better explain what Equation 3 is measuring? Why is the intrusion score defined this way?
- In Figure 7, why does clean change with the x axis? If x axis means number of distractors, and clean has no distractors, it should be a horizontal line and not decrease with more distractors.
- Which models are used in Figure 7 and Table 2? How are the results put together?
4 comments
Follow-up to Reviewer P1HE
Thank you again for your detailed review. We acknowledge your concerns about the writing and the heavy use of the appendix. Each statement in the paper rests on its own result, and all of them cannot fit within the page limit. In the revised version, each analysis now gives its setup and key result in the main text, and the appendix is used only for the fuller detail. We have also rewritten large parts of the paper with more explanation and less jargon, and our response answers your three questions on Equation 3, Figure 7 and the models used.
Since the revised paper differs considerably from the version you reviewed, we would be grateful if you could take a look at it and at our response. If they address your concerns, we hope you would consider reflecting this in your assessment.
I thank the authors for their rebuttal. However, I am still not satisfied, and my opinion remains mostly unchanged.
I re-read the paper, and the abstract and introduction are now clearer, but the other presentation problems remain. Sections 3 and 4 still read like dense summaries of too many results without details, contextualization, clearer word choice, or appropriate pacing, with too many important appendix references. My view has not changed: either these results are important and should be properly self-contained within the main text, or they are not important and should be left out altogether. Currently, reading the paper and understanding all steps is still too complicated because of these issues.
Besides the presentation issues, I have a concern about the experimental side. I was looking at the code and it looks like there might be an issue with HotpotQA and PubMedQA prompts. The gold passages are always added at the end of the context. The default maximum context is 3000 characters in the provided code for the downstream tasks results (I assume to avoid the risk of going past the context limit of the models). In practice, this cuts the gold paragraphs from “distracted” contexts in many cases depending on k, and omits them altogether for high values of k. I am fairly certain the cut happened in the paper experiments because my reproduction closely matches the accuracies reported in the paper for “distracted”. Removing the limit restores most of the “clean” accuracy for the “distracted” prompts. Could the authors confirm whether the paper experiments used this limit, and what changes without it?
Thank you for re-reading the paper and for the detailed follow-up. We are glad the abstract and introduction now read more clearly. We take your point that Sections 3 and 4 are still dense and lean on the appendix, and we will keep restructuring them so that each result is either self-contained in the main text or moved out of it. We are also checking the prompt-construction point you raise for HotpotQA and PubMedQA and will act accordingly.
Response to Reviewer P1HE
Thank you for the detailed review. Your review helped us make the main text more self-contained and helped sharpen the presentation.
Q1. "Why does Equation 3 need to contain the nctx quantity? Can you better explain what Equation 3 is measuring? Why is the intrusion score defined this way?"
The score asks two things of each head. Does it read the irrelevant text, and does it lose support for the correct answer when that text is added? The nctx run is the same question with no context, so it is the reference for the second part. The difference inside the max is the support for the answer that the head loses once the irrelevant text is added, and the max keeps only losses. Without this reference, a head that never supports the answer could not be told apart from a head that lost its support. The weight a_h is the head's attention to the irrelevant text. A head therefore scores high only if it both reads that text and loses support for the answer. The text after Eq. 3 now states this for each term (Section 4). We have added this explanation in the revised main paper as well. Kindly refer to General Response Point-4.
Q2. Clean accuracy in Figure 7 changes with the number of distractors.
This follows from how GSM-DC (Yang et al., 2025) is built, which differs from the other benchmarks we use. Every GSM-DC problem is a dependency graph of reasoning steps, and irrelevant context is added as k extra nodes in that graph, not as text appended to a fixed question. Therefore, in our evaluation each value of k has its own prompts and its own clean reference, which is that k's problems without the injected nodes, so the whole prompt changes from one k to the next. The clean accuracy at a given k is the accuracy on that k's problems without the irrelevant nodes and not one fixed problem set, and hence its accuracy moves with k. For this reason each k has to be read on its own, and every comparison in Figure 7 is made within the same k. Appendix D.1 noted this in the previous version as well, and the caption of Figure 7 now states it in the revised main text.
Q3. Models and aggregation in Figure 7 and Table 2.
Figure 7 shows the mean over the four 7B-scale models (Llama-3.1-8B, Mistral-7B-v0.1, Qwen2.5-7B and Pythia-6.9B) at each k, with binomial standard errors. Figure D.1 gives each model separately. In Table 2, GSM-DC, HotpotQA and PubMedQA use the same four models, and GSM-IC uses Llama and Mistral with chain of thought. Each entry is the share of the lost accuracy that the intervention recovers, and the last row is the mean of the four tasks. Both captions now state the models.
Simple tasks. We use the simple tasks only to find the mechanism: tracing needs one predicted token and prompts that differ only in the added text. The fix is tested on harder tasks (Section 5): multi-hop QA (HotpotQA), biomedical QA (PubMedQA) and multi-step arithmetic (GSM-DC, GSM-IC).
Use outside controlled settings. Controlled prompts are needed once per model, to find the heads. Applying the mask needs three things: access to the model's attention, the head list, and the position of the context. A system that assembles its own prompt already has that position. No clean run, no retraining [and no label of which text is irrelevant] is needed. These are still benchmarks with injected distractors; tool outputs and very long contexts are untested, and the Limitations section now says so.
Experimental setup and results in the main text.
Thank you for raising this point, which made the main text stronger. Section 2 gives the setup shared by all experiments, that is the prompts, tasks, models, metrics and notation. Sections 3 and 4 follow the irrelevant text through the model one step at a time, so they give the setup of each analysis next to its result instead of in a separate methods section. The introduction now ends with a short passage that says so. Following your comment, we made the setup of each analysis explicit in the text and added the key result as a number for most of the analyses in these two sections.
Use of the appendix.
The paper traces a pathway, and each step of it rests on several analyses and their controls. In the main text we gave priority to the chain of reasoning, so that a reader can follow each step and the checks that rule out other explanations for it. The complete results behind every step, that is the full tables, per-model values and controls, do not all fit in the main text, so we placed them in the appendix and pointed to them where they are used. The main text now also carries the setup and the key numbers, as described above, and the pointers lead to the full detail behind them.
Thank you again for the close reading. We would be glad to clarify any point that remains unclear.