acceptodds
Under review as a conference paper at ICLR 2027

How Do LLMs Change Predictions Under Negation?

Abstract

Negation is an essential feature of human language, yet large language models (LLMs) remain unreliable in processing it. We evaluate recent open-source and closed-source LLMs on our negation benchmark and find that, in 37—71% of cases, they repeat the same answer under negation (e.g., "Madrid" for "What is not the capital of Spain?"). To understand and address this brittleness, we mechanistically examine how models operate under negation. Our main finding is that specialized attention heads and MLP neurons jointly implement negation by (1) suppressing retrieval of the original answer (e.g., "Madrid") while (2) promoting a favored candidate within the answer category (e.g., "Paris"). This contrasts with accounts of human negation processing, in which information about the original answer helps to determine what should be excluded. Furthermore, we find that this difference from human processing is a key source of negation failures: the model's mechanism relies on suppressing the original answer rather than using it to determine what to exclude, so the model can repeat the original answer when suppression is too weak or when a bias toward particular answers prevents it from selecting an alternative. To address this weakness in the model's negation mechanism, we propose a training objective that requires larger shifts in answer preference for more confident original predictions, and show that it reduces negation failures with less degradation of general capabilities than standard fine-tuning baselines. Together, our results demonstrate how mechanistic analysis can reveal why a linguistic capability fails and guide training that targets the underlying limitation.

Primary area:
interpretability and explainable AI
Keywords:
Large Language Models, Mechanistic Interpretability, Negation

Reviews

Among:
Shape:
1234this paper, Top 4%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.33 over 3 reviews, above 96% of the others.
Reviewer_pc8xOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

The paper investigates mechanistically how LLMs handle negation. The paper first finds that LLMs (including frontier ones) in general are brittle on negated questions across several tasks. To understand this behaviour, the authors then focus on negated questions where LLMs answer correctly, and through mechanistic interventions find that (among many observations): (1) negation appears to introduce a consistent negation direction; (2) there exists attention heads that suppress original answers and MLP neurons that promote alternative answers but answer category is generally preserved; and (3) most interestingly, negation masks the answer token and encourages the model to find alternative candidates, but this second step of alternative search appears to be question- and answer-agnostic - that is, across different questions with different answers (but in the same answer category), the model has similar preference for the alternative candidates. This last finding explains why LLMs sometimes fail to answer negated questions correctly, because it ISN'T using the original answer to exclude this alternative candidate search. In other words, when the alternative candidate that LLMs prefer matches the original answer, they will generate an answer that will not change under negation. Given these findings, the authors propose a post-training process to fix negation handling, and they found positive results.

Critical strengths
  • The authors found clear mechanisms how LLMs handle negation and why they fail - this is perhaps the most important contribution of this work.

  • The writing is generally clear.

Critical weaknesses
  • A lot of important intuitions are not fleshed out and many technical details are buried in Appendix; this makes it difficult to readers/reviewers to carefully scrutinise them. That said, at a high level most of the design choices (particularly the intervention approaches and metrics) appear sound to me.

  • The findings of the suppression vs. support mechanism isn't entirely new and there are other papers that found similar observations in other tasks (e.g. https://aclanthology.org/2024.acl-long.458.pdf).

  • It's not a big surprise that the post-training method works (given that the metric is closely related to the training objective). It'd be interesting to see whether it would generalise to other types of negations, beyond the somewhat restricted forms of negation in the training data.

Questions for authors
  • Answer generation: Are answers generated with sampling? Asuming that's the case, what's the sampling configuration, and do you run multiple times? if there is variation, how do you select the final answer?

  • Figure 2 right: Isn't it given that when the model's most frequent negated answer = original answer, that we're not going to see answer change? In fact what is strange is that why isn't the answer change% not always 0 for these cases?

  • How do you find the prompt tokens (answer tokens, answer category tokens, etc) given a question?

  • Normalised log-prob gap shift (line 233) missing x_i in the numerator.

  • normalised log-prob gap shift formulation: What is the intuition of the formula? Presumably, if the value is small, that means the intervention didn't do anything, and if the value is large, the intervention has shifted the model to give the original answer for negated questions? But this assumes expect g_{i,f}(x^neg) to be small or close to zero when we look at instances where the model generates an appropriate answer given the negated question? Did you validate that? It might be that there is still a big gap between them (e.g. if the model is a lot less confident in generating the answer for the negated question). It seems like it may be possible to use a simpler formulation that looks at probability shift for the original answer?

  • Figure 9 should be in the main, given its importance

Other relevant papers:

6 comments
AuthorsOpenReview ↗

Official Rebuttal (1/4)

Thank you for your careful reading and thoughtful review. We especially appreciate how well your review summary captures the intent of our work.

Below, we address the weaknesses (W) and questions (Q) you raised and explain how we have strengthened the paper in response. We note the changes in our responses and highlight them in blue in the revised manuscript


W1. A lot of important intuitions are not fleshed out and many technical details are buried in Appendix; this makes it difficult to readers/reviewers to carefully scrutinise them. That said, at a high level most of the design choices (particularly the intervention approaches and metrics) appear sound to me.

TL;DR: We have expanded Sections 4.1–4.2 to explain the intuition and procedures behind our core interventions in the main text.

Thank you for pointing this out. We are glad that you found our intervention approaches and metrics broadly sound. In covering several aspects of the problem, we left some methodological explanations too brief. We have therefore expanded these explanations in Sections 4.1 and 4.2, which present our core mechanistic analyses, to provide clearer intuitions and more technical detail in the main text.

Specifically, Section 4.1 now explains how we rank heads and neurons, fix the selected components before validation, and test their causal effects against matched random controls. Section 4.2 clarifies how we learn the answer-category and candidate-preference spaces through interventions that favor another category or another answer within the same category, respectively.


W2. The findings of the suppression vs. support mechanism isn't entirely new and there are other papers that found similar observations in other tasks (e.g. https://aclanthology.org/2024.acl-long.458.pdf).

Q7. Other relevant papers:

TL;DR: Our novelty lies in explaining how support and suppression mechanisms jointly implement negation and, more importantly, using this understanding to explain negation failures.

Thank you for pointing us to these relevant papers. We would like to take this opportunity to clarify the novelty of our findings.

We agree that support and suppression mechanisms have already been reported in prior work [1–3]. However, our analysis goes beyond examining the effects of individual mechanisms to explain how these relatively simple mechanisms work together to implement a more complex behavior such as negation.

Furthermore, our main contribution is to use this mechanistic understanding to explain why the model’s handling of negation is unreliable. Specifically, we find that the way the suppression mechanism operates is closely linked to negation failures. The model suppresses retrieval of the original answer and promotes candidates within the same category, but does not reliably use information about the original answer to determine what should be excluded. A favored candidate can therefore be the original answer itself, leading the model to repeat it under negation.

We have revised the related works section (Appendix A.2) to discuss these connections to prior mechanistic studies [1–3] and clarify the distinction from our findings. We have also incorporated FLUKE [4] into Appendix A.1 alongside prior evaluations of robustness to negation.

[1] Ortu et al. (2024). Competition of Mechanisms: Tracing How Language Models Handle Facts and Counterfactuals. ACL 2024.

[2] Sun et al. (2025). ReDeEP: Detecting Hallucination in Retrieval-Augmented Generation via Mechanistic Interpretability. ICLR 2025.

[3] Li, Chen, and Tong (2025). Taming Knowledge Conflicts in Language Models. ICML 2025.

[4] Otmakhova et al. (2026). FLUKE: A Linguistically-Driven and Task-Agnostic Framework for Robustness Evaluation. Findings of EACL 2026.

AuthorsOpenReview ↗

Official Rebuttal (2/4)

W3. It's not a big surprise that the post-training method works (given that the metric is closely related to the training objective). It'd be interesting to see whether it would generalise to other types of negations, beyond the somewhat restricted forms of negation in the training data.

TL;DR: ALiT outperforms all evaluated baselines on both accuracy and consistency across all three models on CondaQA, which tests broader negation forms and reasoning within passages.

Thank you for the suggestion! To examine whether our findings extend to more varied forms and uses of negation, we evaluated the post-trained models on CondaQA [5]. CondaQA covers over 200 distinct negation cues and requires reasoning about negation in passages. It also provides three types of passage edits: paraphrasing the negated statement, changing its scope, and reversing the negation.

For illustration, consider the following constructed example with the fixed question, “Could the owner retrieve the cash stored in the safe after the fire?”

Passage versionPassageAnswer
OriginalThe fire destroyed all possessions except the cash in the safe.Yes
ParaphraseThe cash in the safe survived the fire, while all other possessions were destroyed.Yes
Scope changeThe fire destroyed all possessions except the jewelry in the safe.No
Negation reversalThe fire destroyed all possessions, including the cash in the safe.No

Answering requires inferring whether the cash remains recoverable from what the passage says was destroyed or spared. The model must preserve its answer under paraphrasing and adjust it when the passage’s implications change.

To evaluate generalization to this setting, we directly tested the checkpoints already trained in our original benchmark setting, without any additional training on CondaQA. We evaluated all three models (Gemma-3-12/27B-IT, Qwen3.5-9B) including those added in this revision. For baselines, we evaluated checkpoints trained with SFT, unlikelihood training, and DPO, alongside the corresponding base checkpoints. Each cell reports test-set accuracy / overall consistency (%), where consistency requires correct answers to the same question for the original passage and all three edited versions.

MethodGemma-3-12B-ITGemma-3-27B-ITQwen-3.5-9B
Base44.25 / 18.7635.44 / 12.1331.10 / 7.28
SFT41.70 / 17.6211.08 / 0.4339.27 / 10.20
Unlikelihood43.80 / 18.121.01 / 0.0037.20 / 10.70
DPO42.10 / 17.4017.07 / 1.6433.85 / 7.99
ALiT45.59 / 18.9037.38 / 13.5545.30 / 16.05

For each of the three models, the checkpoint trained with ALiT achieves higher accuracy and consistency than both the base checkpoint and those trained with the other methods. Even though CondaQA’s evaluation metrics are less closely tied to our training objective, ALiT achieves these gains without any additional training. These results suggest that ALiT’s benefits extend beyond the training setting to more varied negation forms and to reasoning about negation within passages.

We also found evidence that the negation mechanism identified in our original benchmark generalizes to CondaQA. For the corresponding analyses, please see our response to Reviewer sRo6 (W3).

We have revised the manuscript to include the above findings (Section 5.2).

[5] Ravichander, A., Gardner, M., and Marasović, A. (2022). CONDAQA: A Contrastive Reading Comprehension Dataset for Reasoning about Negation. EMNLP 2022.


Q1. Answer generation: Are answers generated with sampling? Asuming that's the case, what's the sampling configuration, and do you run multiple times? if there is variation, how do you select the final answer?

We use greedy decoding and generate a single response for each input. Thus, we do not run multiple sampling trials or select a final answer from multiple generated responses.

We have explicitly stated these decoding settings in Appendix C (“Evaluation Details”) of the revised manuscript.

AuthorsOpenReview ↗

Official Rebuttal (3/4)

Q2. Figure 2 right: Isn't it given that when the model's most frequent negated answer = original answer, that we're not going to see answer change? In fact what is strange is that why isn't the answer change% not always 0 for these cases?

TL;DR: The most frequent negated answer is defined across different negated questions in the same category, so individual questions can still receive another answer and the answer-change rate need not be zero.

We apologize for the ambiguity in our description of this setup. To clarify, the “most frequent negated answer” is NOT the most common response obtained by repeatedly asking the same negated question. Instead, it is the answer generated most frequently across different negated questions that share the same answer category (e.g., questions asking about people).

For example, if the model most frequently answers “Red” across different negated questions asking about colors, then “Red” is the most frequent negated answer for that category. The model may still generate a different answer for a particular question whose original answer is “Red”. Therefore, the answer change rate can be greater than zero even when the original answer matches the category’s most frequent negated answer.

We have changed the manuscript to more clearly explain the definitions (Section 2).


Q3. How do you find the prompt tokens (answer tokens, answer category tokens, etc) given a question?

TL;DR: We identify answer and category tokens using source metadata for factual questions and document/image and question tokens, respectively, for reasoning tasks; the remaining groups are the inserted negation tokens and the last input token.

Thank you for asking about this. We had not clearly explained how these token groups were identified in the manuscript. Below, we describe how we identify the four groups: (1) answer tokens, (2) answer category tokens, (3) negation tokens, and (4) the final prompt token.

(1) Answer tokens and (2) answer category tokens. We identify these groups using the metadata from the benchmarks used to construct ours. The procedure differs across the three benchmark types.

For factual association (PopQA and RippleEdits), the questions are typically single sentences, such as “What is the capital of Spain?” The source datasets provide information identifying both “capital”, which indicates the answer category, and “Spain”, which is used to retrieve the original answer, “Madrid”. We use the corresponding tokens as answer category tokens and answer tokens, respectively.

For logical reasoning (PhantomWiki and SynthWorlds), the input consists of a document and a question about it. The document contains the information needed to answer, while the question indicates what is being asked. We therefore treat all document tokens as answer tokens and all question tokens as answer category tokens.

Finally, for visual reasoning (GQA and PTR), the input consists of an image and a question about it. Here, the information needed to answer comes from the image, while the question indicates what is being asked. We therefore treat the image tokens as answer tokens and all question tokens as answer category tokens.

(3) Negation tokens. These correspond to the negation expression introduced during benchmark construction. As described in Appendix B.1, we use spaCy’s syntactic analysis to identify the relevant predicate and determine where to introduce negation. We then identify the prompt tokens corresponding to that expression.

(4) Final prompt token. This is the last token of the input prompt, whose representation is used to predict the first answer token.

We have added this explanation to Section 3 and Appendix D.1.


Q4. Normalised log-prob gap shift (line 233) missing x_i in the numerator.

We have corrected the numerator of the normalized log-probability gap shift to include the missing xix_i. Thank you for catching this omission.

Reviewer_sRo6OpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper presents insights into how LLMs make predictions under negation, specifically, how their predictions change with respect to the input (questions) with and without negation. The authors target QA (with questions derived from several existing benchmarks) and mainly explore whether answers change when negation is in the question (they should) and whether the answers belong to the same category than answers to the question without negation (they should). The main conclusion is that the LLMs they try are generally bad (or at least not that good). The paper proceeds with a substantial effort to interpret how negation is processed by LLMs and how answers are generated (by looking at the model internals).

Critical strengths
  • The paper argues that negation is an “essential feature of human language”, and LLMs generally underperform (these points are well-supported with citations). Thus, this is clearly an interesting problem.

  • According to the appendices, the validation of their benchmark (adding negation to questions) as well as the evaluation of the models relies on LLMs as a judge. This is fine and supported by the human validation. That said, they really need to include kappa agreement (discounting the chances of agreeing by chance) in the corpora validation (page 20). The numbers reported (% agreement) appear high, but given the heavily unbalanced distribution, I can’t tell if they are better than chance. I do understand that what they do is rather difficult, they don’t want to do it by hand, and they are doing a good job by reporting human validation rather than just take for granted than what LLMs do is good enough.

  • The paper provides an analysis of several LLMs. The main paper presents the analysis with one LLM, the rest are in the appendices. I didn’t check the results in the appendices carefully, and I would recommend that the authors briefly mentioned whether the main trends are present in all the LLMs or not. If they are, that is certainly better.

  • One could argue that the analyses (figures, etc.) are not tremendously surprising. I disagree. For example, I didn’t know that answer categories don’t change regardless of whether there is a negation in the question (Figure 5). It is true that this paper for the most part observes and analyzes parameter values and intermediate representations (as opposed to build a “better” model), but I do believe there is some interesting analysis.

Critical weaknesses

Negation is no doubt complicated (and the authors are aware based on the references). There are few examples (and most are in the examples) of what kind of texts (questions) this paper is about, but it is limited to a very narrow slice of negation: answering “trivia” questions with and without negation (Table 3, other examples in the paper). It is not true that they cover “diverse surface forms of negation” (134): they only have four and one (n’t) is a contraction of another (not). CondaQA, a paper they cite, includes hundreds of negations forms (or cues).

Additionally, for the kind of questions the authors target, wouldn’t it be enough to always answer with a concept that belongs to the same category (and there is no reason to understand negation). Based on the examples in the paper, questions with negation can be answered by choosing randomly a co-hyponym of the answer to the question without negation. Am I missing something?

Following with the above, the negations they work with are automatically introduced into QA corpora. In other words, the texts they work with are not natural language written by some human, but a carefully curated set of trivia questions to examine what kind of outputs LLMs generate. I can’t help but wonder why they don’t work with open-ended QA instead of the kind of trivia question they work with (e.g., CondaQA and other corpora they cite). I get that their task is narrower in scope (and thus easier to dig into the internals of the LLM), but the concept of “answer category” is not clear to me in “open” texts and open-ended QA. Take, for example a passage describing that a house burnt down and all possession were destroyed except the contents inside a fireproof safe. Answering “Could the owner retrieve the cash he stored in the safe?” should be answered with “yes”, and the idea of answer category is not clear here. Reasoning with passages that have negation (and harder questions / answers) would be much more interesting (and corpora exist for this, the paper cites several).

The paper is about providing interpretations / explaining what LLMs do when processing their questions (specifically, comparing answers to questions with and without negation). Section 5 is an attempt to make empirical improvements, but it is a failed attempt. SFT does as well or better than their new loss. Improving diversity in this case is not beneficial because you are working with trivia questions. If I asked “What is not the capital of Spain” (the first example in the paper), what is the benefit of having diversity in the answer? Any major European city (or major Spanish cities other than the capital) is fine, and I see no reason to prioritize (or reward) diversity. I would suggest removing this section (it is a negative result, isn’t it?) and bringing more analysis to the appendices into the main paper. Alternatively, an actual solution would be better. The way I look at it, SFT is as good as their approach, so there is no reason to use their confidence-adaptive training.

I am surprised about the following:

  • The process to ensure quality of their benchmarks (138) include checking for “semantically unambiguous” questions. That’s just impossible unless one works with very simple questions. The name of any person is ambiguous, for example (e.g., “Mr. Bush”, but which one?). Surprisingly, page 20 says that the humans basically agree 100% with the LLM in terms of whether a question is ambiguous.
  • The quality assurance also ensures that “preserves the meaning of the original question under negation.” This is nonsensical? A question without negation (original) and the question after adding negation obviously won’t have the same meaning. Adding negation just doesn’t preserve meaning (ever?).

Other comments (minor; I think these are easier to fix):

  • Abstract: it mentions “our benchmark”, but it turns out their “benchmark” is a bunch of existing benchmarks (047; the do add negation to the questions). Am I missing something? I think it is a strength that they use (and modify) existing benchmarks rather than creating yet another new benchmark from scratch, but the wording should be clear.

  • The first two pages (and elsewhere in the paper) claim a few times that LLMs process negation differently than humans, They provide as “support” that the LLMs suppress / ignore the answer to the questions without negation. If this is true, how are they getting the “answer category”? (Just from the context? If so, where is the evidence?)

Questions for authors
  • Am I mistaken believing that there is no special analysis of the two datasets that require image understanding (CQA, PTR)? Wouldn’t it be reasonable narrow down anything that belongs to the answer category down to what can be “confused” from the image? For example, not any color makes a lot sense (only colors of other pieces of clothing from the image)?

  • I couldn’t find any details about the identification of tokens into the 4 categories introduced in 200-2024. How is it done?

  • Figure 9 is in an appendix but described in page 5? The paper in general is very heavy in the appendices. This issue could be minimize if the authors chose to remove Section 5.

  • I don’t get the title of Section 5. The LLMs fail, not negation (I guess negation “makes” the LLM fail?)

  • I can’t really figure out what “identified above” refers to in 399.

  • Can you clarify if the input never includes a passage to extract answers from? In other words, when the input does not include an image, are you only checking for parametric knowledge?

10 comments
AuthorsOpenReview ↗

Official Rebuttal (1/7)

Thank you for the detailed review. We are especially grateful for your comment that our mechanistic analysis is surprising and interesting.

Below, we address weaknesses (W), questions (Q), and suggestions in the strengths (S) section. We note the changes in our responses and highlight them in blue in the revised manuscript.


S2. ...they really need to include kappa agreement (discounting the chances of agreeing by chance) in the corpora validation (page 20)...

We would like to clarify that we already include chance-corrected agreement measures for dataset validation (Appendix B.2). Specifically, we report Fleiss’ κ = 0.440 among the three human annotators and Cohen’s κ = 0.334 between the human majority labels and the LLM judgments. As discussed in the manuscript, the labels are heavily skewed toward PASS, which increases expected chance agreement and can lower κ despite high percentage agreement. Both κ values are positive, supporting better-than-chance agreement.

We apologize for not mentioning these measures in the main text, which may have made them difficult to find. We have now revised the main text to explicitly mention them and refer readers to Appendix B.2.


S3. The paper provides an analysis of several LLMs. The main paper presents the analysis with one LLM, the rest are in the appendices. I didn’t check the results in the appendices carefully, and I would recommend that the authors briefly mentioned whether the main trends are present in all the LLMs or not. If they are, that is certainly better.

Thank you for the suggestion. The main trends are largely consistent across the models examined, and we have added a brief summary of these cross-model findings to the main text.

Our original behavioral evaluation covered six open-source models, with localization analyses conducted on four of them in the main paper (Sections 2–3). We have now also extended the mechanistic analysis and post-training experiments (Sections 4–5) from Gemma 3-12B-IT to Gemma 3-27B-IT and Qwen3.5-9B. In both additional models, we identify the same three types of components—original-answer suppression heads, category-promoting heads, and alternative-promoting neurons—and ALiT also reduces negation failures (Appendix G).


W1-1. Negation is no doubt complicated (and the authors are aware based on the references). There are few examples (and most are in the examples) of what kind of texts (questions) this paper is about, but it is limited to a very narrow slice of negation: answering “trivia” questions with and without negation (Table 3, other examples in the paper).

TL;DR: Our evaluation includes reasoning over provided documents and images, so it not limited to "trivia" questions about factual knowledge.

If you mean questions that rely solely on recalling factual knowledge by “trivia” questions, we would like to clarify that our evaluation is not limited to "trivia" questions: it also includes questions requiring in-context reasoning over supporting passages or visual information.

Of the three domains in our benchmark, factual association (PopQA and RippleEdits) relies on parametric knowledge and may therefore fit the description of “trivia QA”. However, logical reasoning (PhantomWiki and SynthWorlds) and visual reasoning (GQA and PTR) require the model to derive answers from the provided documents or images. These questions cannot be answered from parametric knowledge alone. Together, these domains allow us to examine how models handle negation across different sources of information. We have clarified these input formats and the scope of our evaluation in the manuscript.

Nevertheless, we agree that our original benchmark covers a restricted set of question formats. We have therefore extended our evaluation to CondaQA to examine reasoning about negation within passages (see our response to W3).


W1-2. It is not true that they cover “diverse surface forms of negation” (134): they only have four and one (n’t) is a contraction of another (not). CondaQA, a paper they cite, includes hundreds of negations forms (or cues).

We agree that negation takes a wider range of forms than those evaluated here. We have revised the wording to explicitly identify the expressions used (Section 2).

To examine negation beyond the expressions used in our original benchmark, we have also evaluated selected negation types from CondaQA (see our response to W3).

AuthorsOpenReview ↗

Official Rebuttal (2/7)

W2. Additionally, for the kind of questions the authors target, wouldn’t it be enough to always answer with a concept that belongs to the same category (and there is no reason to understand negation). Based on the examples in the paper, questions with negation can be answered by choosing randomly a co-hyponym of the answer to the question without negation. Am I missing something?

TL;DR: Choosing a co-hyponym still requires identifying the relevant category and excluding the original answer, which is the negation requirement that models often fail to satisfy.

You are correct that randomly choosing a co-hyponym is sufficient to solve our task. However, choosing a co-hyponym requires both identifying candidates within the relevant category (hyponyms) and selecting one that is not the original answer. Performing these two processes indeed amounts to carrying out negation. Even if the alternative is selected randomly, the original answer must still be excluded, which requires applying negation.

For example, “Paris” is a valid answer to “What is not the capital of Spain?”, but must be excluded for “What is not the capital of France?”. The answer category remains the same, but which candidate must be excluded changes. Simply choosing a city independently of the original answer does not ensure that this requirement is met.

Even for this seemingly simple task, LLMs frequently fail to exclude the original answer. This makes it a useful controlled setting for studying how LLMs process negation and why they fail. Our analysis examines how models select same-category alternatives and finds that they do so without reliably using information about the original answer to guide exclusion.

AuthorsOpenReview ↗

Official Rebuttal (3/7)

W3. Following with the above, the negations they work with are automatically introduced into QA corpora. In other words, the texts they work with are not natural language written by some human, but a carefully curated set of trivia questions to examine what kind of outputs LLMs generate. I can’t help but wonder why they don’t work with open-ended QA instead of the kind of trivia question they work with (e.g., CondaQA and other corpora they cite). I get that their task is narrower in scope (and thus easier to dig into the internals of the LLM), but the concept of “answer category” is not clear to me in “open” texts and open-ended QA. Take, for example a passage describing that a house burnt down and all possession were destroyed except the contents inside a fireproof safe. Answering “Could the owner retrieve the cash he stored in the safe?” should be answered with “yes”, and the idea of answer category is not clear here. Reasoning with passages that have negation (and harder questions / answers) would be much more interesting (and corpora exist for this, the paper cites several).

TL;DR: Our CondaQA interventions provide causal evidence that the negation signal and components identified in our original setting also contribute to processing negation within passages.

We acknowledge that our original experiments cover a limited range of negation phenomena. We deliberately chose this controlled setting to examine how models exclude the original answer while selecting an appropriate alternative within the same answer category. Suppressing the original answer alone does not determine what the model should produce instead; keeping the requested answer category unchanged allows us to study how suppression and alternative selection work together. This notion of an answer category also applies to other negation tasks when the question or context constrains the answer type; for example, in your fireproof-safe example, the answer category includes affirmative and negative responses, such as "yes" and "no".

Following your suggestion, we examined whether the negation mechanism identified in our setting generalizes to reasoning about negation within passages. We tested two key elements of this mechanism on CondaQA: (1) the negation signal, a shared activation direction, and (2) the downstream attention heads and MLP neurons that suppress original-answer retrieval and promote alternatives within the same answer category.

We compared CondaQA passages with and without negation while keeping the question fixed. All interventions used the signal and components identified from our original benchmark. We tested whether removing this signal or restoring these components to their non-negated values makes the model favor the answer to the non-negated passage again, which would support the mechanism’s generalization to CondaQA.

(1) Negation signal. We removed the signal extracted from our original benchmark’s training split from the prompt token representations of negated CondaQA inputs, and compared its effect with removing a random vector.

Removed directionNormalized log-probability gap shift (%)
Transferred negation signal42.6
Random vector3.1
Removing this signal also shifts answer preference back toward the non-negated answer on CondaQA, much more strongly than the random control. This supports its causal role beyond the questions used to identify it.

(2) Identified components. We replaced the identified head outputs or neuron activations in the negated run with their values from the paired non-negated run.

Restored componentsNormalized log-probability gap shift (%)Random-control gap shift (%)
Attention heads36.44.2
MLP neurons29.12.8
Restoring either set produces a larger shift than random controls, indicating that the same heads and neurons also contribute to the model’s answer change on CondaQA.

Logit-lens illustration. We also examined whether these components retain similar functional roles. One CondaQA question asks whether a tourist in Sarawak would be likely to see Dutch colonial architecture. The affirmative passage describes prior Dutch rule (answer: “yes”), whereas the negated passage states that Dutch rule was never established (answer: “no”).

ComponentOutput examinedIllustrative logit-lens scores
Original-answer suppression headNegated − original“yes”: −1.2
Category-promoting headNegated output“yes”: +1.5; “no”: +1.8
Alternative-promoting neuronsNegated − original“no”: +1.1; “yes”: −0.4
The results are consistent with original-answer suppression, support for the answer category (yes/no), and promotion of the alternative answer.
Together, the analyses support our mechanistic account of negation to negation within passages. We have revised the manuscript to include the above experiments (Sections 3.2, 4.1, and Appendix H.2).
Reviewer_MXJPOpenReview ↗
Rating
4
Confidence
1(−1)
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

This is a comprehensive, well-structured interpretability paper. The paper starts with behavior evaluation and finds that LLMs repeat the original answers in a high proportion when negating. Then, it uses activation patching, component intervention, and DAS analysis mechanisms, and finally proposes the ALiT training objective based on the mechanism diagnosis. The paper does not merely provide "explanations" but converts the mechanism discovery into a training method and provides ID/OOD and general ability assessment, with a good closed loop.

Critical strengths

The problem is significant, and the phenomenon is clear.

The benchmark is large in scale and covers a wide range.

The mechanism analysis has a hierarchical structure.

The failure diagnosis is closely linked to the training method.

Critical weaknesses

The model universality of the mechanism conclusion is limited. The core mechanism analysis mainly focuses on Gemma 3-12B-IT, and other models mainly involve partial replication. Although there is an expansion in the appendix, the conclusion that "LLMs generally achieve negation" should still be stated more cautiously.

The training and validation range is narrow. ALiT is only trained on Gemma 3-12B-IT + PopQA within the domain, and OOD is the test set of five benchmarks. Lack of validation for other models, larger models, multilingualism, or full fine-tuning, and the extrapolation is limited.

The benchmark validation relies on an LLM judge. Although 240 cases were manually verified with high consistency, GPT-5.6 Luna still performed the main filtering; the fidelity of the never-negation scope filtering was relatively high, which may introduce systematic bias.

The statement "Different from human negation processing" is slightly stronger. The paper compares with human negation processing literature, but there is no direct human experiment. Saying that LLM "contrasts with human accounts" is acceptable, but it should be more explicit that this is an indirect comparison based on literature.

The target construction of ALiT may amplify the model's own negation preference. ALiT extrapolates from the model's own negated logits; although the confidence-adaptive margin is targeted, it still needs to be analyzed whether it amplifies the negation bias; the answer diversity is still lower than the base, and the sensitivity of α and β_max is not discussed adequately.

Questions for authors
  1. Does ALiT still maintain the trade-off between category preservation and diversity when applied to models such as Llama/Qwen/OLMo?

  2. Can you conduct a small-scale manual semantic evaluation of the negative answers generated by ALiT, rather than relying solely on LLM judgment?

  3. How sensitive are α and β_max? When the original answer's confidence is extremely high, is the target extrapolation still reliable?

  4. Can the causal effects of answer category heads and candidate-preference neurons in the mechanism analysis be replicated in more models?

  5. Can the comparison with human negative processing be strengthened through a small-scale human behavior experiment or a more direct literature argument?

6 comments
AuthorsOpenReview ↗

Official Rebuttal (1/5)

Thank you for the detailed and insightful feedback on our work!

We especially appreciate your description of our work as “a highly complete and well-structured interpretability paper” and your recognition of the value of using mechanistic findings to guide training.

Below, we address the weaknesses (W) and questions (Q) you raised. We note the changes in our responses and highlight them in blue in the revised manuscript.


W1. The model universality of the mechanism conclusion is limited. The core mechanism analysis mainly focuses on Gemma 3-12B-IT, and other models mainly involve partial replication. Although there is an expansion in the appendix, the conclusion that "LLMs generally achieve negation" should still be stated more cautiously.

Q4. Can the causal effects of answer category heads and candidate-preference neurons in the mechanism analysis be replicated in more models?

TL;DR: We have replicated the main mechanistic findings and the causal effects of the identified components in additional models, with conclusions limited to the models and settings tested.

To examine whether our findings generalize across models, we have now extended our core mechanistic analyses and causal intervention experiments from Gemma 3-12B-IT to Gemma 3-27B-IT and Qwen3.5-9B. In both additional models, selected heads and neurons causally affect answer preference, and their logit-lens readouts are consistent with original-answer suppression, category promotion, and alternative promotion (Appendix G.1).

(1) Negation signal. Following Section 3.2, we estimate each model’s negation direction from paired non-negated and negated inputs in the training split. We remove this direction from the final prompt-token representation of negated inputs with steering strength β=1\beta=1. The table reports the normalized log-probability gap shift toward the non-negated answer at the best-performing layer in each evaluated sweep.

ModelSelected layerNegation-signal removal (%)Random-vector removal (%)
Gemma 3-12B-ITL2532.154.51
Gemma 3-27B-ITL3431.942.98
Qwen3.5-9BL1820.455.45

Removing the signal shifts answer preference back toward the non-negated answer in all three models, more strongly than the random control.

(2) Identified components. We replace the selected attention-head outputs or MLP neuron activations in the negated run with their values from the paired non-negated run. We intervene on five attention heads and/or 256 MLP neurons and compare their effects with layer- and count-matched random component sets across six benchmarks. Each cell reports the normalized log-probability gap shift for selected / random components (%).

Restored componentsGemma 3-12B-ITGemma 3-27B-ITQwen3.5-9B
Attention heads21.43 / 2.1719.12 / 8.1225.98 / 4.62
MLP neurons24.76 / 0.1126.94 / -0.0428.54 / 0.32
Both sets jointly35.21 / 2.2032.95 / 9.6433.12 / 5.58

For all three models, restoring either selected component set produces a substantially larger shift than random restoration, and restoring both produces a larger effect than either alone.

(3) Functional roles. To examine what these components promote or suppress, we measure negated-minus-non-negated changes in their summed logit-lens contributions. We examine original-answer support and category support from selected heads, and preference for the generated alternative from selected neurons. Each cell reports selected / random components.

ComponentsGemma 3-12B-ITGemma 3-27B-ITQwen3.5-9B
Original-answer support-0.960 / -0.083-1.478 / +0.041-0.575 / -0.127
Same-category versus other-category support+0.489 / +0.062+1.207 / -0.114+0.656 / +0.019
Generated alternative versus other same-category candidates+1.126 / -0.036+2.846 / +0.138+1.005 / -0.071

The interventions establish causal contributions of the selected component sets to answer preference, while the readouts are consistent with original-answer suppression, category promotion, and alternative promotion in all three models.

These results support our core interpretation across the models examined: models suppress retrieval of the original answer and promote alternatives within the same category, without reliably using information about the original answer to determine what to exclude. We limit this interpretation to the tested models and settings.

AuthorsOpenReview ↗

Official Rebuttal (2/5)

W2. The training and validation range is narrow. ALiT is only trained on Gemma 3-12B-IT + PopQA within the domain, and OOD is the test set of five benchmarks. Lack of validation for other models, larger models, multilingualism, or full fine-tuning, and the extrapolation is limited.

Q1. Does ALiT still maintain the trade-off between category preservation and diversity when applied to models such as Llama/Qwen/OLMo?

TL;DR: ALiT’s benefits extend across models, negation tasks, and fine-tuning settings, with the added models retaining a favorable trade-off between category preservation and diversity.

(1) Generalization across models. We have additionally applied ALiT to Qwen3.5-9B and Gemma 3-27B-IT, extending our evaluation to another model family and a larger model (Appendix G.2).

We report overall answer change, category preservation, answer diversity, original accuracy, and average general capability. In addition, to measure exclusion more directly, we report conditional answer change when the original answer matches the model's most frequent negated answer within its category (a=afreqa=a_{\mathrm{freq}}). In these cases, repeatedly producing that frequent answer would fail to exclude the original answer.

Gemma 3-27B-IT

SplitMethodOverall answer change (%)Change: a=afreqa=a_{\mathrm{freq}} (%)Category preservation (%)Diversity(%)Original accuracy (%)
IDBase68.256.0099.989.8824.28
ALiT90.3781.0099.0479.1822.17
Unlikelihood100.00 (+9.63)N/A*45.88 (-53.16)92.10 (+12.92)22.82 (+0.65)
SFT99.31 (+8.94)17.86 (-63.14)99.45 (+0.41)7.29 (-71.89)23.72 (+1.55)
DPO100.00 (+9.63)100.00 (+19.00)38.72 (-60.32)96.39 (+17.21)24.14 (+1.97)
OODBase69.0056.4295.1274.8528.85
ALiT81.6470.4498.1173.6828.86
Unlikelihood88.74 (+7.10)71.85 (+1.41)46.45 (-51.66)85.77 (+12.09)29.55 (+0.69)
SFT74.87 (-6.77)36.59 (-33.85)99.12 (+1.01)48.82 (-24.86)29.30 (+0.44)
DPO99.03 (+17.39)89.87 (+19.43)44.35 (-53.76)86.16 (+12.48)29.71 (+0.85)

Qwen3.5-9B

SplitMethodOverall answer change (%)Change: a=afreqa=a_{\mathrm{freq}} (%)Category preservation (%)Diversity(%)Original accuracy (%)
IDBase55.414.8899.379.7817.64
ALiT87.0466.44100.0078.3117.96
Unlikelihood99.91 (+12.87)N/A*31.23 (-68.77)85.22 (+6.91)17.15 (-0.81)
SFT98.46 (+11.42)0.00 (-66.44)99.61 (-0.39)7.94 (-70.37)17.69 (-0.27)
DPO100.00 (+12.96)100.00 (+33.56)50.63 (-49.37)92.57 (+14.26)17.73 (-0.23)
OODBase59.5736.1196.166.0139.20
ALiT67.4140.4698.4464.2939.32
Unlikelihood97.41 (+30.00)88.48 (+48.02)38.87 (-59.57)85.56 (+21.27)39.50 (+0.18)
SFT82.07 (+14.66)16.22 (-24.24)96.40 (-2.04)36.96 (-27.33)38.55 (-0.77)
DPO99.73 (+32.32)99.47 (+59.01)47.23 (-51.21)89.84 (+25.55)38.70 (-0.62)

*N/A indicates that no originally correct examples satisfy a=afreqa=a_{\mathrm{freq}}, so the conditional answer-change rate is undefined.

ALiT maintains a favorable trade-off between category preservation and diversity in both additional models. Although its diversity remains below the base model, it retains substantially higher diversity than SFT while maintaining category preservation of 98.11–100.00% across ID and OOD settings. Unlikelihood and DPO achieve higher overall answer change and diversity, but their category preservation is substantially lower (31.23–50.63%).

ALiT also achieves higher conditional answer change than SFT: 81.00% / 70.44% versus 17.86% / 36.59% for Gemma 3-27B-IT, and 66.44% / 40.46% versus 0.00% / 16.22% for Qwen3.5-9B (ID / OOD). It therefore more reliably excludes the original answer even when that answer is the model’s most frequent response under negation. Across both additional models, ALiT improves exclusion without the large diversity reduction observed with SFT or the large category-preservation loss observed with Unlikelihood and DPO.

AuthorsOpenReview ↗

Official Rebuttal (3/5)

(2) Generalization beyond our benchmark.

We have also evaluated the post-trained models on CondaQA [1], which covers over 200 distinct negation cues and requires reasoning about negation within passages. It provides three types of passage edits: paraphrasing the negated statement, changing its scope, and reversing the negation.

To evaluate generalization to this setting, we directly tested the checkpoints already trained in our original benchmark setting, without any additional training on CondaQA. We evaluated all three models, including those added in this revision. Each cell reports test-set accuracy / overall consistency (%), where consistency requires correct answers to the same question for the original passage and all three edited versions.

MethodGemma 3-12B-ITGemma 3-27B-ITQwen3.5-9B
Base44.25 / 18.7635.44 / 12.1331.10 / 7.28
SFT41.70 / 17.6211.08 / 0.4339.27 / 10.20
Unlikelihood43.80 / 18.121.01 / 0.0037.20 / 10.70
DPO42.10 / 17.4017.07 / 1.6433.85 / 7.99
ALiT45.59 / 18.9037.38 / 13.5545.30 / 16.05

For each of the three models, the checkpoint trained with ALiT achieves higher accuracy and consistency than both the base checkpoint and those trained with the other methods. These results suggest that ALiT’s benefits extend beyond the training setting to more varied negation forms and to reasoning about negation within passages.

We have revised the manuscript to include these findings (Section 5.2 and Appendix H.3).

[1] Ravichander, A., Gardner, M., and Marasović, A. (2022). CONDAQA: A Contrastive Reading Comprehension Dataset for Reasoning about Negation. EMNLP 2022.

(3) Generalization to full fine-tuning. We have also evaluated ALiT with full fine-tuning of Gemma 3-12B-IT, comparing it with SFT under the same training setting. We report answer change, category preservation, and answer diversity to examine whether ALiT’s benefits also hold under full fine-tuning (Appendix J.1).

MethodOverall answer changeChange: a=afreqa=a_{\mathrm{freq}}Change: a≠afreqa\ne a_{\mathrm{freq}}Category preservationDiversity
ID / OODID / OODID / OODID / OODID / OOD
Base58.14 / 46.9353.97 / 33.7458.21 / 51.0699.71 / 96.6490.90 / 80.05
SFT96.84 / 87.1543.07 / 19.1597.10 / 94.00100.00 / 99.0691.54 / 79.68
ALiT97.45 / 84.6775.31 / 62.5098.03 / 86.4499.86 / 97.6392.32 / 80.88

Even under full fine-tuning, ALiT achieves higher answer change than SFT when a=afreqa=a_{\mathrm{freq}} in both ID and OOD settings, while maintaining high category preservation and answer diversity.

(4) Multilingualism. Our benchmark is currently available only in English, and it would be interesting to examine whether our findings extend to other languages. Unfortunately, constructing and validating multilingual versions is not feasible within the rebuttal period. We therefore leave this extension to future work and limit our conclusions to the English settings evaluated here.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Review Summary

We thank the reviewers for their constructive and insightful feedback. They highlighted several strengths:

  • (1) Significance of the problem. “The problem is significant, and the phenomenon is clear.” (MXJP); “this is clearly an interesting problem.” (sRo6)
  • (2) Clear mechanistic explanation of negation failures. “clear mechanisms how LLMs handle negation and why they fail” (pc8x).
  • (3) Actionability of the analyzed mechanisms. “The failure diagnosis is closely linked to the training method.” (MXJP)
  • (4) Soundness and scale of the methodology. “The benchmark is large in scale and covers a wide range.” (MXJP); “at a high level most of the design choices … appear sound to me.” (pc8x)
  • (5) Well-structured analysis and clear writing. “The mechanism analysis has a hierarchical structure.” (MXJP); “The writing is generally clear.” (pc8x)

The reviewers also raised several concerns, which we addressed as follows:

  • (1) Generalizability of mechanistic findings (MXJP, sRo6).

    • Concerns: Reviewers expressed concern about mechanistic analysis mainly focused on one model and a limited set of negation forms (MXJP, sRo6).
    • Response:
      • We showed model generalizability by replicating the core causal findings in two additional models (Gemma 3-27B-IT and Qwen3.5-9B).
      • We also showed task generalizability by testing the causal effects of our core mechanisms(negation signal and components that we identified) on diverse forms of negation in CondaQA passages (Appendices G.1, H.2).
  • (2) Effectiveness of the training algorithm (sRo6, pc8x, MXJP).

    • Concerns: The reviewer asked why ALiT is needed when SFT achieves similar or higher overall answer-change rates. They also questioned why producing more diverse answers is beneficial for this task (sRo6). Reviewers requested additional tests across models, tasks, and hyperparameter settings (pc8x, MXJP).
    • Response:
      • We showed ALiT’s advantage in excluding the original answer, a core requirement of negation. When the original answer was also the most frequent response to negated questions in the same category, ALiT achieved 62.74% OOD answer change versus 18.48% for SFT (Gemma 3-12B-IT; Table 2). This comparison directly measures reliable exclusion, rather than diversity among otherwise valid answers.
      • We reproduced this exclusion benefit in additional models and under full fine-tuning. We also evaluated ALiT on CondaQA without additional training, where it achieved higher accuracy and consistency than all evaluated baselines across all three models (Appendices G.2, H.3, J.1).
      • We added hyperparameter sensitivity analyses and showed improved answer change over the base model on high-confidence inputs (Appendices J.2–J.3).
  • (3) Soundness of dataset and evaluation (sRo6, MXJP).

    • Concerns: Low agreement raised concerns about noise in the LLM-filtered dataset (sRo6). Direct human evaluation of ALiT’s answers was also requested (MXJP).
    • Response:
      • For dataset validation, we directly assessed the quality of the retained data to address the low-agreement concern. LLM-filtered data were not significantly noisier than human-filtered data (2.13% vs. 1.80%; p = 0.823). Additionally, most LLM–human disagreements involved the LLM rejecting human-accepted examples and therefore were not included in the dataset.
      • For answer evaluation, we conducted additional annotation of 120 ALiT predictions with three annotators and reported Cohen’s κ = 0.9272 between the LLM judge and human-majority labels (96.67% agreement), and Fleiss’ κ = 0.9259 among annotators (94.81% unanimous agreement).
  • (4) Novelty compared to prior work (pc8x).

    • Concerns: Reviewers expressed concerns about individual mechanisms (suppression and support) not being entirely new.
    • Response:
      • We clarified that our contribution is not identifying individual mechanisms, but explaining how they jointly implement a complex behavior (negation) and why this process fails (Appendix A.2).
  • (5) Task design and scope (sRo6).

    • Concerns: The reviewer asked whether the task can be solved simply by choosing a random answer from the requested category, without processing negation. They also questioned whether the benchmark only tests factual recall.

    • Response:

      • We clarified that the task requires processing negation: choosing from the right category is not sufficient, as the model must also exclude the answer to the question without negation.
      • We also explained that our benchmark is not limited to trivia questions, as it includes reasoning over provided documents and images.
  • We also clarified the literature-based human comparison and English-only scope, expanded the main-text explanations, and moved the key figure into the main text. Revisions are highlighted in blue.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.