acceptodds
Under review as a conference paper at ICLR 2027

A Common Scale for Contextual Override: What Survives a Change of Wording

Abstract

Retrieval systems are vulnerable to attacks embedded within the documents they retrieve. Presenting a claim as fiction, attributing it to a specific source, or repeating it across multiple documents are all known exploit vectors. However, since each finding originates from a different experimental setup, these attacks have never been systematically ranked against one another. Furthermore, they have never been measured against the baseline control required to make such a ranking interpretable: the same claim, embedded in the same document, but explicitly denied. In this work, we aim to construct this unified scale. A controlled document generator maintains the fact, counterfactual object, prompt question, sentence count, and mention count as strict invariants, changing only the framing of a single sentence. The resulting behavior is then scored as a signed margin against each model’s measured parametric prior. Because each rung of this scale uses a distinct sentence, a surface-level ranking of framings risks reducing the analysis to a simple ranking of arbitrary wordings. To mitigate this, we rebuild both evaluation ladders using an independent set of claim templates and re-assess all 11 models. Accumulated corroboration emerges as the most effective attack we tested. All models are overridden on a majority of facts by at least one accumulation condition (achieving an 89% override rate across 185 facts at eight sources, compared to just 26% when the claim is denied). Instruction tuning halves the efficacy of single-mention attacks while simultaneously sharpening the impact of accumulation. This asymmetry is critical for real-world deployment, where a retrieval ranker dictates the exact number of corroborating passages presented to the model. When matched on their baseline parametric priors, models differ primarily in susceptibility (the magnitude by which a single mention shifts their confidence), not in memorization or polarity sensitivity. Ultimately, while the hierarchical ordering between framing tiers survives a change of wording, the assumption that denial constitutes the weakest possible framing does not.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Contextual Override, Retrieval Systems, Knowledge Conflict, Language Models, Instruction Tuning

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_vpd9OpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

This paper presents the challenge that current contextual override attacks are tested using different assumptons and experimental setups, which makes it hard to compare the effects or even rank against one another. To solve this problem, the paper presents a unified framework that standardises the documents, facts and corresponding metrics that are robust to challenges facing existing approaches. The paper finds that accumulated corroboration is the most effective attack tested, with all models getting overriden by at least one accumulation.

Critical strengths
  1. The paper evaluates the proposed unified framework across a wide range of LLMs which improves the validity of the experiments conducted.
  2. The paper tackles an important problem i.e. the lack of a unified framework to compare the effect of different contextual override attacks.
Critical weaknesses

1. The significance of the contribution remains unclear.

The intro claims that the effect of different attacks remains incomparable because of a lack of a unified framework that allows for comparison. However, it remains unclear what the new finding this unified framework presents that previous work has not already surfaced. There is very little engagement on how the results and findings from the experiments complement or differ from existing work making it hard to determine the utility of the proposed instrument. I would suggest the authors improve the current discussion section by moving beyond just reporting the findings to reflecting on how it answers the problem posed on the lack of a unified framework.

2. Core experimental details are unclear

L248, the paper uses 185 facts to test the instrument proposed, however, it is unclear how this dataset was constructed. Is this sampled from an established dataset or were the facts created for this task? Please provide clearer details on the dataset.

3. The paper omits the Appendix section which makes it hard to cross-check most parts of the paper

The following sections: L116, L160, L381, L502, L503, L504, L505, make references to an Appendix section that is non-existent in this submission. I am unable to cross-check any claims presented in the main sections.

Questions for authors
  1. Could you provide references for the claims made in L044 - L046, what previous literature supports the claim being made i.e. "The prevailing view, which holds that parametric and contextual evidence each carry an instrinsic strength and the stronger wins...". This claim remains unsubstantiated.
  2. This framing here needs to be clarified (L041) - "retrieval system does not filter what it surfaces". This sentence may appear misleading as retrieval systems in practice include multiple stages, first-stage retrieval followed by reranking e.g. using a LLM which may filter depending on the requirements of the task. Please clarify what type of retrieval systems you are referring to.
  3. The notation used in defining the instrument in Section 3.1 is unclear and makes it difficult to follow, I would recommend providing a clear example of a fact, question and document, as well as providing clear definitions of the notation before use.
  4. How was the size of D selected and did you run any ablations across different document sizes?
  5. How is this claim substantiated - "allocating more words to deny a claim provides the model with a stronger signal that a denial is occurring"?
  6. The current presentation format makes it hard to follow through different parts. I would suggest the authors to provide clear section titles that tell the reader where the important parts of the paper e.g. Methodology, Experimental Setup, Results etc.
Reviewer_g5RWOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

The paper investigates how different presentations of false claims in retrieved documents influence language models' factual predictions. It proposes a common measurement framework to compare denial, fiction, source attribution, and repeated claims across 11 language models.

Critical strengths
  1. The paper evaluates several contextual framing conditions across multiple language models and attempts to control experimental variables, including document length and mention count.
Critical weaknesses
  1. The abstract and introduction discuss contextual override, counterfactual objects, parametric priors, and a common scale without first providing a concrete explanation of the experimental task. The reader only learns in Sections 3.1–3.2 that a model receives a constructed document and a factual question, and that the evaluation compares its preference for the true and counterfactual answers. A simple running example illustrating the input, expected output, and different framing conditions would make the paper considerably easier to understand. Even after reading the introduction carefully, I found it difficult to establish precisely what problem the authors intended to solve.

  2. The introduction (Section 1, pp. 1–2) argues that previous studies evaluate contextual manipulations using different experimental setups and that these manipulations have not been systematically ranked. However, the paper does not explain why establishing such a ranking is itself an important research problem. For instance, what scientific question would be answered by knowing that repeated corroboration is more effective than fictional framing or explicit denial?

  3. The abstract initially presents the problem as a security concern involving attacks embedded in retrieved documents. The introduction then emphasizes the surprising failure of models under explicit denial, citing override rates of 31% and 84% for OLMo-2-7B and Qwen2.5-7B. Subsequently, Sections 3.4 and 4.2 frame the robustness of the scale to wording changes as the central methodological contribution. Finally, Section 4.3 describes the instruction-tuning analysis as the most consequential result. These are different research directions, and their relationship is not clearly articulated.

  4. Section 2 acknowledges that Basmova et al. (2024) have already studied fictional and non-affirmative contexts, Schuster et al. (2026) have examined credibility hierarchies and their reversal under repetition, and Jin et al. (2024a) have investigated majority-rule behavior. Nevertheless, the paper does not convincingly explain what fundamental limitation of these findings necessitates the proposed framework. Furthermore, relevant studies such as ClashEval (Wu et al., 2024), which investigates reliance on incorrect retrieved evidence, and PoisonedRAG (Zou et al., 2025), which evaluates attacks involving multiple malicious retrieved documents, are missing from the related work.

  5. The introduction motivates the work through real-world retrieval systems that may surface fact-checks, rumors, fictional accounts, and conflicting evidence. However, Section 3.2 constructs synthetic eight-sentence documents using templated claims and filler sentences, and no actual retrieval system is evaluated.

  6. Section 3.6 evaluates 185 facts from only four relations: capitals, currencies, authorship, and element symbols. The authors acknowledge that relation type is confounded with corpus frequency, but no broader evaluation is provided. Similarly, the experiments cover 11 open-weight models ranging from 7B to 32B parameters, without evaluating larger contemporary or frontier reasoning models. Given the paper's broad claims about retrieval-system vulnerability and instruction tuning, the evaluation does not establish whether the findings generalize beyond these particular facts, model families, and prompting conditions.

  7. The paper frequently alternates between terms such as attack, manipulation, framing, condition, intervention, and dose, as well as scale, ladder, rung, and instrument, without consistently defining their relationships. Several statements are also confusing. For instance, Section 4.2 describes the second template set as weakening every condition while reporting changes ranging from −1.28 to +0.25 log-odds, which includes a positive shift. The results section also introduces multiple secondary analyses without consistently relating them to the central research question. A clearer problem formulation, a running example, consistent terminology, and a more focused organization would substantially improve the manuscript's readability.

Questions for authors
  1. The paper frequently alternates between terms such as attack, manipulation, framing, condition, intervention, and dose, as well as scale, ladder, rung, and instrument, without consistently defining their relationships. Several statements are also confusing. For instance, Section 4.2 describes the second template set as weakening every condition while reporting changes ranging from −1.28 to +0.25 log-odds, which includes a positive shift. The results section also introduces multiple secondary analyses without consistently relating them to the central research question. A clearer problem formulation, a running example, consistent terminology, and a more focused organization would substantially improve the manuscript's readability.

  2. Why is establishing a unified ranking of contextual framings important?

  3. What is the main novelty relative to previous knowledge-conflict studies?

  4. How do the synthetic experiments support the motivation concerning real-world RAG systems?

  5. Where are the appendices referenced throughout the manuscript? The submitted PDF ends with the references on page 11, but the paper repeatedly refers to Appendices A–L.

Reviewer_icnYOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper develops a unified measurement framework for studying context-induced overrides of parametric knowledge in retrieval-augmented generation (RAG). The document generator holds the factual subject, counterfactual object, question, number of sentences (eight), and number of mentions fixed, while varying only the framing of the sentence containing the counterfactual claim.

The primary metric is a signed log-probability margin, defined as (logp(o∘)−log⁡p(o)log p(o^\circ)-\log p(o)) at the first token following "Answer:", interpreted relative to each model's own no-evidence prior. An override rate is then computed using a margin threshold of zero.

The study examines five dimensions: polarity (assertion, denial, refutation, debunking, and fiction), source credibility (from rumor to consensus), cumulative evidence (0–8 sources), plausibility, and document position. Two additional controls are introduced: explicit denial as an anchor condition and a prior-matched subset (MAB(τ)M_{\mathrm{AB}}(\tau)).

To assess whether the observed rankings are merely artifacts of wording, the authors independently construct a second set of templates (Tb), recreate the full experimental ladders, and repeat the evaluation on 11 open-source models using 185 facts across four relations.

The main findings are as follows. Cumulative corroboration produces the strongest override effect, reaching an 89% override rate with eight sources, compared with 26% under explicit denial. Fiction and rumor outperform denial in 9–10 models, while the credibility ladder is monotonic in 7–8 models. However, denial is not consistently the weakest framing condition, and the relative ordering of denial and debunking changes depending on whether override rate or margin is used.

Critical strengths
  1. The measurement design is more rigorous than that of many related studies. The number of sentences and mentions is enforced as an invariant in the generation code, reducing several obvious sources of confounding. Using explicit denial as an anchor also makes attack effectiveness a relative quantity rather than relying solely on absolute model behavior. The paper also validates the relationship between the proposed margin metric and actual generated responses using 60 facts across seven conditions, reporting 98.3% agreement in the direction of the response and a correlation of (r=0.989). And the authors openly report that one of their headline ordering claims does not survive robustness testing: denial is not reliably the weakest framing condition.

  2. The prior-matched subset (MAB(τ)M_{\mathrm{AB}}(\tau)), together with the threshold sweep from (τ=0.25\tau=0.25) to (4.0), provides a useful way to distinguish a model's resistance to contextual evidence from differences in its initial preference for the correct answer. The reported effect remains relatively stable around −3.80 across the threshold sweep. This is a stronger design than simply measuring correlations between prior confidence and override behavior over an unmatched population of facts.

Critical weaknesses
  1. Two of the paper's main conclusions depend on cross-model comparability of log-odds margins. The paper presents several conclusions as findings revealed by a "validated scale," but the underlying margin is measured in log-odds, which are not directly comparable across models with differently calibrated output distributions. In particular, models can differ substantially in the sharpness of their predictive distributions, or their effective temperature. Instruction tuning often produces sharper distributions, so an equivalent change in the model's relative preference for two answers can result in a larger log-odds shift after instruction tuning.

Also, the paper's own results seem to provide a different picture when using the scale-independent override rate. In Table 3, the difference in override rate between Gemma-27B and OLMo-7B is -1.96;[-13.73,,+9.80] which is not statistically significant.

Similarly, the claim that instruction tuning makes cumulative attacks more effective is supported by increased log-odds margin shifts in 10 of 12 comparisons, while the override rate remains at 100% both before and after instruction tuning.

  1. A central practical claim is that presenting a debunking document can reinforce or inject the very claim it attempts to refute. However, the denial conditions used in the experiments appear to negate the counterfactual claim without providing the correct answer. For example, a sentence such as "It is false that the capital of France is Sucre" rejects the incorrect entity but does not identify the correct capital. Real-world fact-checking documents typically include both a denial and an explicit correction. Without a "denial + correct answer" condition, the experiments cannot distinguish between two substantially different mechanisms: The model fails to process the negation and treats the denied claim as evidence; The model reproduces the only salient answer-like entity present in the retrieved context because no alternative answer is provided. The second behavior may reflect a general tendency of base language models to copy salient contextual entities under raw-completion prompting rather than a vulnerability specific to deployed RAG systems.

  2. There are also several limitations in the experimental design and statistical analysis: First, the study considers only 185 facts across four relations. The relation types are confounded with corpus frequency, as the authors acknowledge, making it difficult to disentangle their effects. Second, the analysis does not apply a multiple-comparison correction, and the reported confidence intervals do not account for variance arising from document randomization. fInally the first-token scoring results in model-dependent subsets of evaluated facts, because facts whose correct and counterfactual objects share the same first token are excluded. So the reported override rates across models are not necessarily computed over identical sets of facts. The number of exclusions for each model is reportedly available only in the appendix.

Questions for authors
  1. Were Appendices A–L included as part of the submission? They are not present in the PDF available to me.
  2. Could the authors recompute the results in Table 3, as well as the claim that instruction tuning increases cumulative margin shifts, using a calibrated or scale-invariant metric? For example, please consider model-specific temperature scaling before computing log-odds margins, or an appropriate rank-based effect measure.
  3. Please provide the exact sentence templates used for the denial, refuted, and debunked conditions.
  4. Section 3.5 reports a generation-based validation using 60 facts across seven conditions. Do these seven conditions include the eight-source cumulative-evidence condition and the chat-template prompting regime?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.