acceptodds
Under review as a conference paper at ICLR 2027

When Neurons Collide: Why Independently Successful Computations Fail Together

Abstract

Large language models routinely carry out multiple computations within a single forward pass. Because these computations read from and write to the same residual stream, they can interact with one another. We study neural collisions: cases in which computations that succeed independently fail when processed together, even when both share the same answer. Across Llama-3.1-8B-Instruct, Gemma-4-31B-it, and Qwen3-32B, we find that neural collisions are the extreme case of a broader phenomenon: concurrent computations routinely influence one another during inference, sometimes enough to overturn an otherwise correct prediction. In a collision, one computation corrupts another, pushing its internal representation away from the state it reaches in isolation. Using activation patching, we show that this corruption emerges abruptly, at a small number of token positions over just a few layers. To quantify this corruption, we introduce residual drift, the distance between a computation's joint and isolated trajectories. Across all three models, residual drift strongly predicts how much a collision changes the model's answer.

Primary area:
foundation or frontier models, including LLMs
Keywords:
representational interference, mechanistic interpretability

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_sF59OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper investigates how additional context with same label can disrupt a model’s answer. The authors identify pairs of statements that the model evaluates correctly in isolation but incorrectly when presented together, even when both statements have the same ground-truth label. Using activation patching, they find that intermediate target-token states can carry changes sufficient to reproduce the erroneous behavior. They further introduce a residual-drift metric that correlates with the magnitude of the margin shift at later layers. However, the analysis provides limited insight into why these changes arise. Given existing research on irrelevant-context interference, the contribution appears to be an additional experimental characterization of a known phenomenon. The interpretation that these failures reflect neural collision is not sufficiently established.

Critical strengths
  1. The problem is worth investigating, and the same-label pairing setting provides an interesting way to study contextual interference.

  2. The authors go beyond behavioral evaluation and attempt to investigate the model’s internal processing through causal interventions.

Critical weaknesses
  1. The activation-patching experiments localize states that can transmit the erroneous behavior, but do not sufficiently explain the mechanism that produces it. They do not establish what information from the partner statement disrupts the target computation, or which model components cause this disruption. The additional mechanistic insight over prior work therefore remains unclear.

  2. The interpretation of these failures as neural collision is insufficiently supported. The model is asked to evaluate only one statement, so the presence of another statement does not by itself establish that two independent evaluation processes are being executed and interfering with each other.

Questions for authors
  1. The paper rules out answer competition via same-label pairing (Sec. 2, 4), but this does not show that two separate evaluation processes actually run in the joint prompt. What evidence supports the ‘collision’ interpretation over a single computation being distracted by the partner?

  2. Could the results be partly explained by changes in input length, token position? Please provide appropriate controls and analyze whether errors are concentrated in particularly vulnerable target statements or particularly disruptive partner statements. What properties of the target–partner relationship determine whether interference occurs?

7 comments
AuthorsOpenReview ↗

Response to Reviewer sF59

We have highlighted all changes to the paper in light red to make them easy to identify.

1. Mechanism of destructive collisions

Reviewer comment:

“The activation-patching experiments localize states that can transmit the erroneous behavior, but do not sufficiently explain the mechanism that produces it. They do not establish what information from the partner statement disrupts the target computation, or which model components cause this disruption.”

Our response:

We thank the reviewer for this question. We added three experiments. The first identifies which model components contribute to collisions. The second investigates why information from partner A causes disruption. The third examines what information in statement B's representation is affected.

Experiment 1: Which component causes the disruption?

Within a transformer layer, self-attention is the only operation that transfers information between token positions. We therefore test whether attention from B's tokens to A's causes interference. For up to 100 destructive collisions per model, we prevent B's tokens from attending to A's tokens across progressively more layers. Both statements remain in the prompt, so statement A can still influence the final answer, but cannot directly alter B’s representation in the masked layers.

Results (Figure 6): Blocking this attention corrects most collisions:

  • Llama-3.1-8B: 76.1% corrected (first 16 layers).
  • Qwen3-32B: 99.1% corrected (first 48 layers).
  • Gemma-4-31B: 57.2% corrected (first 48 layers).

Attention from the target's tokens to the partner's tokens therefore contributes causally to collisions. This also suggests a simple, training-free remedy: block attention between independent statements while still letting the answer draw on both.

Experiment 2: Does the harm come from contamination or attention dilution?

Attention from B to A can harm B in two different ways:

  1. Contamination: B's tokens read information from A, and this information disrupts B's computation.
  2. Dilution: attention is a fixed budget shared across tokens, so attention given to A is taken away from B's own tokens, which may weaken retrieval of the information B needs.

Blocking attention removes both effects together. To distinguish them, we exploit the additive structure of attention to intervene on each separately:

  • Remove contamination: Remove A's contribution while leaving attention to other tokens unchanged.
  • Undo dilution: Restore attention to other tokens while retaining A's contribution.

The mathematical decomposition and interventions are detailed in Section 6.2.

Results (Figure 7):

  • Llama-3.1-8B: Removing contamination corrects substantially more collisions than undoing dilution, which has almost no effect.
  • Qwen3-32B: Both mechanisms contribute in the earliest layers. In later layers, undoing dilution has little effect, while removing contamination continues to correct a substantial proportion of collisions.

Collisions are therefore not simply caused by attention being pulled away from the target’s own tokens. The information B reads from A is itself a major source of the damage.

Experiment 3: What task-relevant information in B is affected?

Experiments 1 and 2 show that information from A, carried by cross-statement attention, contributes causally to collisions, but they leave open what in B's representation is affected. One possibility is that interference disrupts information about whether B is true or false.

To test this, we train a linear probe to predict whether B is true or false from its internal state at B's final token. If interference affects this information, we would expect the probe to be less accurate on pairs that collide. We test this prediction by applying the fixed probe to held-out collision and non-collision pairs.

Results (Figure 21): Across all three models, probe accuracy is substantially lower on collision pairs. This is consistent with interference affecting information about B's truth value before the question is read.

Summary.

Together, these experiments go beyond localising where collision damage appears and show how it arises. Information from the partner enters the target’s representation through cross-statement attention (Figure 6). The damage comes mainly from this information itself, not from the attention it takes away from the target’s own tokens (Figure 7).

Manuscript reference: Section 6.1, “Does blocking attention between statements prevent collisions?” (Figure 6); Section 6.2, “What makes cross-statement attention harmful?” (Figure 7); and Appendix D, “Probing truth values before the question” (Figure 21).

AuthorsOpenReview ↗

Response to Reviewer sF59

2. Is this really a collision between two computations?

Reviewer comment:

"The interpretation of these failures as neural collision is insufficiently supported. The model is asked to evaluate only one statement, so the presence of another statement does not by itself establish that two independent evaluation processes are being executed and interfering with each other."

Our response:

We agree that the presence of two statements does not establish that the model evaluates both. To investigate whether two evaluations are taking place, we test whether the model represents the truth values of both statements before it knows which one will be queried.

Experiment: Are both truth values represented in the same hidden state? In the joint prompt, both statements appear before the question:

A: [statement A]
B: [statement B]
Is statement B true?

At the final token of B, we train two probes on the same residual-stream activation: one to decode A's truth value and one to decode B's. We use pairs balanced across all four truth-label combinations (TT, TF, FT, FF), ensuring that knowing one label does not reveal the other.

Results (Figure 20): Both truth values are decodable from this single state across all three models, while shuffled-label controls remain near chance (50%). This shows that information about both statements' truth values is represented concurrently, even before the model knows which statement will be queried. Crucially, information about A's truth value is available at B's final token, indicating that information from the partner has already entered the target's representation.

Manuscript reference: Appendix D, “Probing truth values before the question” (Figure 20).

 

3. Can prompt length or token position explain collisions?

Reviewer comment:

"Could the results be partly explained by changes in input length, token position? Please provide appropriate controls and analyze whether errors are concentrated in particularly vulnerable target statements or particularly disruptive partner statements. What properties of the target–partner relationship determine whether interference occurs?"

Our response:

We thank the reviewer for these suggestions and have added a new appendix section addressing each question.

Adding a partner changes the prompt in two simple ways: it makes the prompt longer, and it moves the target and question to later positions. We test each with a separate control.

Control 1: Partner length (Figures 12–13). If longer prompts caused the effect, longer partners should change the target's margin more. They do not: in all three models, the average size of the change shows no consistent increase with partner length.

Control 2: Length-matched whitespace (Figures 14–15). We replace each partner with the same number of whitespace tokens. The prompt length and the positions of all other tokens stay exactly the same; only the partner's content is removed. Across all three models and both target positions, matched whitespace produces far fewer collisions than a real partner.

Together, these results show that input length and token position alone cannot explain the observed collisions. The partner's content plays an important role.

Manuscript references: Appendix A.2, “Can changes in input length or token position explain collisions?” (Figures 12–15).

AuthorsOpenReview ↗

Response to Reviewer sF59

4. Are destructive neural collisions concentrated around particular statements?

Reviewer comment:

"Please provide appropriate controls and analyze whether errors are concentrated in particularly vulnerable target statements or particularly disruptive partner statements."

Our response:

We conducted additional analyses to determine whether collisions are driven by a few disruptive partners (statements that cause many targets to fail) or vulnerable targets (statements that fail with many different partners).

Experimental setup: We arrange all evaluated statement pairs in a matrix, with partners as rows and targets as columns. Each cell indicates whether that particular pairing causes a destructive collision. A few influential contexts would show up as rows full of collisions, and vulnerable targets as columns full of collisions.

Results (Figures 9–11):

  • Collisions concentrate on vulnerable targets (Figure 9). In all three models, collisions line up in a small number of columns: a few targets fail with many different partners.

  • These targets are close to the decision boundary. A collision occurs when a partner reduces the target’s margin by more than its solo margin, so less confident targets need only a small reduction to fail. Consistent with this, a target’s solo margin is negatively correlated with the fraction of partners that cause it to collide (Qwen: ρ = −0.435; Gemma: ρ = −0.654; Llama: ρ = −0.503).

  • Changes in the model’s margin are much less concentrated (Figures 10–11). Beyond collisions, we measure how much each partner reduces the target’s margin, even when the answer stays correct. To test for concentration, we compare against a permutation baseline that shows how much concentration chance alone would produce. Some targets receive somewhat larger reductions than others, most clearly in Gemma.

Conclusion: We find little evidence that a few influential contexts drive collisions. Almost every partner shifts the target’s margin to some degree. Collisions concentrate on particular targets mainly because those targets start closer to the decision boundary, so even an ordinary shift is enough to flip their answer.

Manuscript references: Appendix A.1, “Are collisions concentrated around particular statements?” (Figures 9–11).

 

5. What properties of the target–partner relationship determine whether interference occurs?

Reviewer comment:

"What properties of the target–partner relationship determine whether interference occurs?"

Our response:

One possibility is that collisions arise from confusion between semantically similar statements. For example, two statements about capital cities might interfere more strongly because they involve the same factual relation. If this explains collisions, we would expect statements from the same semantic category to collide more frequently.

Experimental setup: Our statement bank contains facts from different semantic categories (capital cities, authors, sports, etc.) alongside arithmetic statements. We compare destructive collision rates for pairs from the same category against pairs from different categories, and examine the full category-by-category breakdown.

Results (Figure 16):

  • Qwen3-32B: Same-category and different-category pairs have identical collision rates (1.7% vs. 1.7%).
  • Llama-3.1-8B: Same-category pairs collide slightly less often (5.2% vs. 6.5%).
  • Gemma-4-31B-it: Same-category pairs collide roughly twice as often (13.1% vs. 6.6%).

Figure 16 further shows how collision rates vary across individual category pairings.

Conclusion: Semantic category similarity does not consistently predict collisions across models, although it appears to matter for Gemma. More specific properties of the relationship between statements may still matter, and identifying these remains an open question.

Manuscript reference: Appendix A.3, “Are collisions more likely between statements from the same semantic category?” (Table 1, Figure 16).

Reviewer_pUiUOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

The paper investigates 'neural collisions' - prompts where irrelevant additional information in the prompt causes the model to behave differently. The authors neatly isolate true 'neural collisions' from other potential explanations through careful dataset design. Experiments on a range of models reveal that this is a surprisingly common phenomenon, and activation patching experiments demonstrate that the phenomenon is remarkably localised.

Critical strengths
  • The experiments are clear and well-designed
  • The results show that the phenomenon is surprisingly widespread, at least in simple prompts
Critical weaknesses
  • The broader significance of these results is unclear, especially given the small models and simple prompts studied in the paper
Questions for authors

How might we find neural collisions at larger scales and in more complex domains?

4 comments
AuthorsOpenReview ↗

Response to Reviewer pUiU

1. Generalisation to more complex tasks and larger scales

Reviewer comments:

"How might we find neural collisions [...] in more complex domains?"

Our response:

To address this concern, we extend our experiments to three multi-step reasoning benchmarks with open-ended answers: GSM8K, Compositional Celebrities, and MuSiQue.

Because these tasks require open-ended answers rather than binary Yes/No predictions, we can no longer measure collisions using logit margins. Instead, we compare the model's greedily decoded answer against the gold answer. A collision occurs when the model answers correctly in isolation but incorrectly when paired with another problem. We also compare against length-matched whitespace controls to account for changes in prompt length and token position.

Results (Figure 4): Collisions occur across all nine model–dataset combinations, with real partners consistently causing more failures than length-matched whitespace controls. Collision rates range from approximately 3.2% to 15.6%, compared with 1.8% to 6.2% in our binary experiments.

These findings show that collisions extend to multi-step reasoning with open-ended answers. The higher observed failure rates suggest that interference may be particularly relevant to more complex tasks.

Manuscript reference: Section 4.2, “Collisions extend to multi-step reasoning” (Figure 4); evaluation protocol in Appendix B.”

AuthorsOpenReview ↗

Response to Reviewer pUiU

2. Fine-tuning to reduce collisions

Reviewer comment:

"The broader significance of these results is unclear"

Our response:

We have added a new section showing that collisions can be substantially reduced through fine-tuning, without impairing the model's ability to combine statements when a task requires it.

Motivation. Blocking attention between statements corrects most collisions without any training (Section 6.1). However, this approach is only suitable when the statements are independent. In tasks where information from both statements is needed, blocking attention could prevent the model from combining them. We therefore ask whether fine-tuning can reduce interference between independent statements while preserving the ability to combine dependent information.

Method. We fine-tune each model using LoRA to preserve the target's original confidence when another statement is added. We then compare collision rates before and after fine-tuning on held-out statement pairs.

Results (Figure 8): Fine-tuning substantially reduces collision rates in all three models. To check that this improvement does not come at the cost of useful information sharing, we also evaluate the models on dependent-statement problems that require combining information from both statements. Accuracy on these problems remains essentially unchanged after fine-tuning.

Conclusion. Collisions are not a fixed property of these models: a simple objective derived from our analysis removes most of them, and the models still combine statements when the question requires it.

Manuscript references: Section 7, "Reducing collisions through fine-tuning" (Figure 8); Appendix E, "Fine-tuning details" (Figure 22).

As the discussion period closes, thank you again for your helpful feedback. We hope the added multi-step reasoning results and fine-tuning experiments clarify the broader significance of neural collisions. We would be grateful for a brief acknowledgment of our rebuttal and, if the reviewer feels that our responses and revisions have addressed their concerns, reconsideration of the score.

AuthorsOpenReview ↗

Closing comment

As the discussion period closes soon, we would appreciate your thoughts on whether our new multi-step reasoning and fine-tuning results address your concerns about broader significance. If so, we would be grateful if you could reconsider your score.

Reviewer_WCTNOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies the influence of irrelevant context on predictions. Specifically, this paper focuses on True/False questions and constructs two sets of problems that the model can answer both correctly and confidently when presented alone. It then pairs these questions with each other, retaining only pairs with the same ground-truth label, and observes that they interfere with each other’s predictions. The authors call cases where an otherwise correct prediction becomes incorrect “destructive neural collisions.”

To understand how this interference happens, the authors perform a series of activation patching experiments and observe that the representations of the target-statement tokens are indeed influenced by the distractor statement, confirming the occurrence of neural collision. Moreover, they localize the influence to a few critical tokens within a handful of layers. Finally, they propose residual drift, measured by the relative L2 distance between readout states with and without the partner statement, as an indicator of the severity of the influence, which is supported by correlation analyses across sampled destructive collisions.

Critical strengths
  1. The paper is very clear and easy to follow. It is well motivated, has clear visualizations, and is very coherent.

  2. The problem is important and timely. As large language models process longer and longer contexts, understanding how irrelevant context interferes with predictions will become increasingly important.

  3. The experimental setup is very clean and thoughtful. The authors filter out low-confidence statements and pair statements with the same ground-truth label. The use of true/false questions also helps isolate the effect of neural collisions clearly.

  4. The activation patching experiments are also well designed. The interventions cover all target-statement tokens and layers, which is very comprehensive. This helps convincingly localize the influence to a few critical tokens and layers.

Critical weaknesses
  1. The depth of the explanation is not fully satisfactory. Specifically, this paper localizes the positions where destructive neural collisions happen. However, the exact mechanism by which the irrelevant statement changes the computation of B is not yet clear. For example, do the target tokens read irrelevant information, making the computation less clean? Or is the relevant information not retrieved because the model’s capacity is taken up by processing the irrelevant context?

  2. Related to the first point, the explanations remain largely descriptive. It is not clear how to use these findings to improve model training or decoding to avoid the negative influence of destructive neural collisions. Such guidance would be especially useful given the question posed in the introduction: “What determines whether concurrent computations coexist successfully or collide?” I am not sure I have a clear answer to this question after reading the paper.

  3. The variety of tasks and the complexity of the predictions are relatively limited. The experiments mainly cover factual recall and basic arithmetic. However, it is unclear whether these observations generalize to other types of tasks and reasoning that require multiple steps. Moreover, although the true/false setup is clean, it covers only binary predictions. It would be interesting to see whether the same observations hold for predictions involving numerical or other types of answers.

Questions for authors
  1. Could you include more experimental details? For example, how were the arithmetic statements generated, and how complex are they? I am also curious about how the false statements were constructed.

  2. Are destructive neural collisions concentrated around a few highly influential contexts?

6 comments
AuthorsOpenReview ↗

Response to Reviewer WCTN

We have highlighted all changes to the paper in light red to make them easy to identify.

1. Mechanism of destructive collisions

Reviewer comment:

"The depth of the explanation is not fully satisfactory. Specifically, this paper localizes the positions where destructive neural collisions happen. However, the exact mechanism by which the irrelevant statement changes the computation of B is not yet clear. For example, do the target tokens read irrelevant information, making the computation less clean? Or is the relevant information not retrieved because the model’s capacity is taken up by processing the irrelevant context?"

Our response:

We thank the reviewer for this question. We added three experiments. The first identifies the mechanism by which statement A changes B's computation. The second experiment directly tests the reviewer's two explanations: whether B is disrupted by information read from A, or by reduced attention to relevant information. The third examines whether B's representation of its own truth value is affected.

Experiment 1: Does cross-statement attention cause collisions?

Within a transformer layer, self-attention is the only operation that transfers information between token positions. We therefore test whether attention from B's tokens to A's causes interference. For up to 100 destructive collisions per model, we prevent B's tokens from attending to A's tokens across progressively more layers. Both statements remain in the prompt, so statement A can still influence the final answer, but cannot directly alter B’s representation in the masked layers.

Results (Figure 6): Blocking this attention corrects most collisions:

  • Llama-3.1-8B: 76.1% corrected (first 16 layers).
  • Qwen3-32B: 99.1% corrected (first 48 layers).
  • Gemma-4-31B: 57.2% corrected (first 48 layers).

Attention from the target's tokens to the partner's tokens therefore contributes causally to collisions. This also suggests a simple, training-free remedy: block attention between independent statements while still letting the answer draw on both.

Experiment 2: Does the harm come from contamination or attention dilution?

Experiment 1 shows that cross-statement attention causes interference, but does not explain why this attention is harmful. Following the reviewer's suggestion, we consider two mechanisms:

  1. Contamination: B's tokens read information from A, and this information disrupts B's computation.
  2. Dilution: attention is a fixed budget shared across tokens, so attention given to A is taken away from B's own tokens, which may weaken retrieval of the information B needs.

Blocking attention removes both effects together. To distinguish them, we exploit the additive structure of attention to intervene on each separately:

  • Remove contamination: Remove A's contribution while leaving attention to other tokens unchanged.
  • Undo dilution: Restore attention to other tokens while retaining A's contribution.

The mathematical decomposition and interventions are detailed in Section 6.2.

Results (Figure 7):

  • Llama-3.1-8B: Removing contamination corrects substantially more collisions than undoing dilution, which has almost no effect.
  • Qwen3-32B: Both mechanisms contribute in the earliest layers. In later layers, undoing dilution has little effect, while removing contamination continues to correct a substantial proportion of collisions.

These findings directly address the reviewer's alternatives: the information B reads from A is itself an important source of disruption, rather than failures arising solely because attention is diverted away from B.

Experiment 3: What information in B is affected?

Experiments 1 and 2 show that information from A, carried by cross-statement attention, contributes causally to collisions, but they leave open what in B's representation is affected. One possibility is that interference disrupts information about whether B is true or false.

To test this, we train a linear probe to predict whether B is true or false from its internal state at B's final token. If interference affects this information, we would expect the probe to be less accurate on pairs that collide. We test this prediction by applying the fixed probe to held-out collision and non-collision pairs.

Results (Figure 21): Across all three models, probe accuracy is substantially lower on collision pairs. This is consistent with interference affecting information about B's truth value before the question is read.

Manuscript references: Section 6.1, “Does blocking attention between statements prevent collisions?” (Figure 6); Section 6.2, “What makes cross-statement attention harmful?” (Figure 7); and Appendix D, “Probing truth values before the question” (Figure 21).

AuthorsOpenReview ↗

Continued Response to Reviewer WCTN

2. Generalisation to multi-step reasoning and open-ended answers

Reviewer comment:

“The variety of tasks and the complexity of the predictions are relatively limited. The experiments mainly cover factual recall and basic arithmetic.”

Our response:

To address this concern, we extend our experiments to three multi-step reasoning benchmarks with open-ended answers: GSM8K, Compositional Celebrities, and MuSiQue.

Because these tasks require open-ended answers rather than binary Yes/No predictions, we can no longer measure collisions using logit margins. Instead, we compare the model's greedily decoded answer against the gold answer. A collision occurs when the model answers correctly in isolation but incorrectly when paired with another problem. We also compare against length-matched whitespace controls to account for changes in prompt length and token position.

Results (Figure 4): Collisions occur across all nine model–dataset combinations, with real partners consistently causing more failures than length-matched whitespace controls. Collision rates range from approximately 3.2% to 15.6%, compared with 1.8% to 6.2% in our binary experiments.

These findings show that collisions extend to multi-step reasoning with open-ended answers. The higher observed failure rates suggest that interference may be particularly relevant to more complex tasks.

Manuscript reference: Section 4.2, “Collisions in multi-step reasoning” (Figure 4); evaluation protocol in Appendix B.”

 

3. Additional experimental details

Reviewer comment:

"Could you include more experimental details? For example, how were the arithmetic statements generated, and how complex are they? I am also curious about how the false statements were constructed."

Our response:

We have expanded the experimental setup in the revised manuscript to clarify how our statements are constructed.

Arithmetic statements are generated synthetically, each involving a single addition, subtraction, or multiplication of two integers, with operands up to 600. For each expression, we create a true statement using the correct result and a false statement by shifting the result by a small amount (up to 10 in either direction). For example:

  • True: 247 + 318 = 565
  • False: 247 + 318 = 567

Factual statements are drawn from a preprocessed version of PopQA containing both true and false propositions (Mallen et al., 2023). False propositions replace the correct answer entity with an incorrect alternative while preserving the underlying factual relation.

“Manuscript reference: Section 4, “Detecting neural collisions” (Data paragraph).”

 

4. Are destructive neural collisions concentrated around a few highly influential contexts?

Reviewer comment:

"Are destructive neural collisions concentrated around a few highly influential contexts?"

Our response: We thank the reviewer for this question. Our new analyses suggest that collisions are not primarily driven by a few unusually disruptive partners. Instead, they concentrate on a small number of vulnerable targets, partly because these targets are closer to the decision boundary.

To investigate this, we plot all evaluated statement pairs in a matrix, with partners as rows and targets as columns. Red cells indicate collisions: rows with many red cells would reveal disruptive partners, while columns with many red cells would reveal vulnerable targets.

Results (Figures 9–11):

Across all three models, collisions cluster in a few columns (Figure 9), showing that certain targets fail with many different partners. These targets tend to have smaller solo margins, meaning that even modest interference can flip their predictions. Consistent with this, we find negative correlations between solo margin and collision frequency (Qwen: ρ = −0.435; Gemma: ρ = −0.654; Llama: ρ = −0.503).

To determine whether this concentration reflects unusually strong interference or simply greater target vulnerability, we also examine continuous margin shifts (Figure 10). These shifts are much more broadly distributed across statement pairs.

Together, these results suggest that interference is broadly distributed, but targets closer to the decision boundary are more easily pushed into failure.

Manuscript reference: Appendix A.1, “Are collisions concentrated around particular statements?”, Figures 9–11; summarised in Section 4.3.”

AuthorsOpenReview ↗

Continued Response to Reviewer WCTN

5. Fine-tuning to reduce collisions

Reviewer comment:

"It is not clear how to use these findings to improve model training or decoding to avoid the negative influence of destructive neural collisions."

Our response:

We thank the reviewer for this suggestion. We have added a new section showing that collisions can be substantially reduced through fine-tuning, without impairing the model's ability to combine statements when a task requires it.

Motivation. Blocking attention between statements corrects most collisions without any training (Section 6.1). However, this approach is only suitable when the statements are independent. In tasks where information from both statements is needed, blocking attention could prevent the model from combining them. We therefore ask whether fine-tuning can reduce interference between independent statements while preserving the ability to combine dependent information.

Method. We fine-tune each model using LoRA to preserve the target's original confidence when another statement is added. We then compare collision rates before and after fine-tuning on held-out statement pairs.

Results (Figure 8): Fine-tuning substantially reduces collision rates in all three models. To check that this improvement does not come at the cost of useful information sharing, we also evaluate the models on dependent-statement problems that require combining information from both statements. Accuracy on these problems remains essentially unchanged after fine-tuning.

Conclusion. Collisions are not a fixed property of these models: a simple objective derived from our analysis removes most of them, and the models still combine statements when the question requires it.

Manuscript references: Section 7, "Reducing collisions through fine-tuning" (Figure 8); Appendix E, "Fine-tuning details" (Figure 22).

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.