acceptodds
Under review as a conference paper at ICLR 2027

Decided While Reading: Why a Language Model Stops Using a Fact Placed Before Its Rule

Abstract

Language models are often asked to apply a rule to a few facts and give an immediate verdict, as guardrails, classifiers and judges do. Such prompts place the rule before, among or after the data. They assume that a model with the whole prompt in view applies the rule to the fact it names, wherever the rule appears. We find that this assumption fails. Across 18 open models, moving one unchanged rule sentence just past the fact it names, or after all the facts, sharply reduces that fact's influence on the answer, and the answer follows the facts read after the rule instead. Yet the models still know the fact. Asked separately, they mostly answer correctly what the rule compares and whether the fact matches. Causal experiments explain the gap. As a model reads each fact after the rule, it forms the rule's outcome at that fact's own position, and its direct answer later reads the outcome mainly from those positions. A fact read before the rule has no outcome there, and the answer, which sees both the rule and the fact, does not make up for it. In short, the verdict is decided while reading. For systems that answer directly, the rule must come before the data it governs, and explaining before answering restores the lost fact.

Primary area:
interpretability and explainable AI
Keywords:
rule following, instruction placement, prompt order, activation patching, mechanistic interpretability, large language models

Reviews

Among:
Shape:
1234this paper, Top 32%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.50 over 2 reviews, above 68% of the others.
Reviewer_WHQAOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

The paper studies prompts in which a model applies a rule to a few facts and answers directly, as guardrails, classifiers, and judges do. In a synthetic task with three facts and one rule sentence that moves among them, the authors enumerate all eight truth assignments to obtain complete answer tables and define the influence of each fact on the answer. Across 18 open models, the influence of the named fact falls from 76 to 34 (medians) when the rule moves from just before it to just after it, while the influence of the facts read after the rule rises, and the models still answer separate questions about the rule and the fact mostly correctly. Re-reading the fact after the rule restores its use, and hidden-state transplants move the answer more when written into the fact's tokens than into the rule's tokens, which the authors interpret as the rule's outcome being formed at each fact's tokens during reading and read from there by the answer.

Critical strengths
  1. The behavioral design isolates rule placement, since only one sentence moves between layouts, six formatting factors are counterbalanced, complete eight-row answer tables are recorded, and the full placement grid argues against distance to the answer as the cause (Fig. 2b). The step appears in 18 of 18 models and in both vocabularies, which supports the placement effect independently of the mechanistic account.
  2. The paper separates knowing a fact from using it, as the models answer the attribute and match questions while the action fails (Fig. 3). A copy of the fact restores its use only when read after the rule, not before it and not when another fact is copied (22 to 80 versus 23 and 21, Fig. 4a), which locates the failure between the fact and the answer.
Critical weaknesses
  1. The observed drop in influence may also be due to an instruction-following fault under an unusual prompt structure. In standard pretraining and instruction-tuning distributions, rules almost always precede the data they govern. Placing a rule after the fact does not follow this prior, and the authors do not fully disentangle whether the failure is driven by distribution mismatch. In fact, Llama-3.3-70B and smaller models are less robust than closed models like GPT and are easy to break, but experiments on stronger, more robust frontier models are missing.
  2. Experiments use short prompts with only three facts and one rule (Sec. 2), so it is unclear whether a fact placed before its rule is still ignored in longer prompts with more facts.
  3. Main prompts come from a single template , so the results may not hold for other prompt wordings and tasks.
Questions for authors
  1. Does the step persist when the number of facts grows or when unrelated text of several hundred tokens separates the rule from the named fact? A result of this kind would address the first weakness. (clarification or short experiment)
  2. Could the behavioral test be run on one or two closed models through sampled answers, to show whether the placement law holds there?
Reviewer_YEMnOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2(+1)
Presentation
2
Soundness
3
Read review
Paper summary

The paper studies how language models apply a rule to a few facts when they must give an immediate verdict, as guardrails, classifiers and judges do. Using a synthetic task in which only the position of one rule sentence changes, the authors show across 18 open models that placing the rule after the fact it names sharply reduces that fact’s influence on the direct answer, even though the models still know the fact when asked separately. The finding is validated with complete answer tables, counterbalanced prompts and templated support tickets. To explain it, the authors propose that the rule’s outcome is computed at each fact read after the rule and that the direct answer reads it from there. They support this account with re-reading experiments and hidden-state transplants, and they present Llama-3.3-70B as an exception that takes the outcome from the rule itself. They also show that explaining before answering restores the fact, and they recommend placing the rule before the data in systems that answer directly.

Critical strengths

The paper studies one clear observation carefully. It uses a clean manipulation, moving a single rule sentence while keeping everything else fixed, and measures the effect thoroughly across many models with controlled variations. The effect it reports is large and consistent, which makes the core observation credible and reproducible, and this is the main reason I see value in the work.

The comparison between explaining before answering and answering before explaining is the most interesting part of the paper for me. Explaining first restores the use of the fact. When the answer comes first, the explanation that follows often reasons correctly even though the answer was wrong. This gives a controlled example of a post-hoc explanation that does not reflect how the answer was produced, which matters both for work on explanation faithfulness and for judges and classifiers that answer directly.

Critical weaknesses

The effect is shown on a synthetic task (three facts, one rule template, two value vocabularies) and only for direct answers. Reasoning before the answer removes it (influence 97 vs 28 with the letter first, Sec. 5), and a policy in the system prompt behaves like a rule placed first. The configuration at risk is therefore a direct answer with the rule after the data, and the paper does not show that this is common in deployed guardrails or judges. The paper’s one realistic test does not support its prescription either. In the support tickets, a policy after the whole ticket costs nothing when the facts are written as free prose (median 0, lower in 9 of 18 models). With form fields, the median cost of 14 points hides several models that score higher with the policy last (e.g., Llama-3.3-70B, 93 vs 62; Table 11). This mirrors Liu et al. (2024a), where placing the query before and after the documents made the synthetic key-value task near-perfect but barely changed multi-document QA. The headline effect is also anticipated by cited work (Ok & Lee, 2026; Sen Sharma et al., 2026). This is a well-documented observation, but in a narrow regime.

The paper claims that “the verdict is decided while reading” and that the direct answer reads it mainly from the fact’s own tokens. Its own test for separating a stored action from a stored comparison, donors whose rule swaps the two actions, gives inconsistent results across models (5, −8, 63). The limitations section concedes the question is open, yet the title, abstract and Figure 1 present the stored quantity as the verdict. The contrast between transplants into the fact’s tokens and into the rule’s tokens (−62 vs −23) cannot show where the answer reads from. In the recipient the rule follows the fact, so a one-layer patch into the fact also changes what the rule and every later token read at higher layers. A one-layer patch into the rule, by contrast, can be overwritten when the rule’s tokens re-read the unpatched fact, and several models were tested at only two layers. Reading the prompt twice, with the rule removed from the first copy, raises the named fact’s influence from 22 to 73 (Qwen3-32B: 7 to 82; Table 12), although the fact is never read after the rule. The mechanism is what the paper adds beyond the behavioral observation, so without it the contribution reduces to the observation above.

Questions for authors
  1. When the rule is removed from the first copy of a repeated prompt, the named fact’s influence reaches 73 (vs 22 for a single copy), although it is never read after the rule. How does fact-side reading account for this?

  2. How do you reconcile “the rule must come before the data” with the support tickets? There, a policy after the whole ticket costs nothing in free prose, and with form fields, Llama-3.3-70B, Phi-4, Gemma-3-12B and Gemma-3-27B score higher with the policy after the ticket than at the start.

  3. Can you point to realistic settings where a guardrail or judge must give a direct verdict with the rule placed after the data it governs? Policies usually sit in the system prompt, which behaves like a rule placed first in your results.

  4. Could the authors conduct their placement study on a well-established benchmark from a realistic setting? Showing that the observation holds there would make it considerably more practical and impactful, and would help change my assessment.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.