acceptodds
Under review as a conference paper at ICLR 2027

PRIOR: PRECEDENT RECORDS INVALIDATE OPERATIVE RULES

Abstract

An agent's past decisions can weaken its adherence to an explicit policy that remains in context. We call this phenomenon precedent erosion. We introduce Ledger, a controlled suite of binary governance decisions with synthetic histories, and measure actions, stated-policy identification and reports of the operative policy. Length-matched controls, sham subtraction and five validity checks address measurement confounds. In the primary low-risk setting, one permissive record after five compliant records yields an action-level sham-subtracted double difference of in Qwen3-4B, while the recovery criterion is first met after five corrective records. Qwen3-14B shows no positive erosion under the same conditions. Violations coexist with correct identification of the stated policy: recognising a rule does not ensure following it. In Qwen3-4B, late-layer precedent shifts align with the activation direction defined by removing the policy sentence. Injecting this direction induces or reverses violations, whereas norm-matched controls do not produce these flips at the tested dose. Controlled post-training experiments on Qwen3-4B-Base produce substantial erosion after response distillation from Qwen3-14B, while token-matched SFT, preference optimisation and self-distillation controls show little or none. Together, these results characterise precedent erosion, establish a causal handle and show that a teacher's resistance to precedent does not guarantee a resistant student.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Precedent erosion, LLM agents, Instruction following, Activation steering

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_9aVrOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

The paper studies whether synthetic histories of earlier policy violations make language models disregard an explicit current rule. It measures decisions and policy reports, then uses activation steering and post-training comparisons to investigate the effect.

Critical strengths

The research question is relevant to agents whose earlier decisions remain in the conversation: an explicit approval rule should continue to govern subsequent actions.

Critical weaknesses
  • The paper is exceptionally difficult to read and understand. In the abstract, the setup, control comparisons, and recovery claim are compressed into terminology that makes the basic experiment difficult to follow. I found the paper exceptionally frustrating to read. In Figure 2, the main results are attributed to paraphrased records, but in Table 4, the same treatment is described as literal. This leaves the experimental condition unclear, even after consulting the appendix, and directly affects the claim that the result goes beyond copying answer tokens.

  • Claims and experiment results do not align. In Appendix I.1, rewording the prompt reduces the Qwen3-14B-distilled student's action effect from prose records from +0.94 to 0.00, while the control remains at 0.00. The effect subsequently reported with bare answer tokens does not reproduce that original comparison. In Appendix B, recovery is declared when the confidence interval includes zero. This means the remaining effect cannot be reliably distinguished from zero; it does not show that compliance has been restored or establish the claimed recovery asymmetry.

  • The training and steering experiments do not establish the proposed explanation. In Table 19, the UltraChat control is described as "human-written," although the original UltraChat paper by Ding et al. describes generating the conversations with ChatGPT. This comparison cannot establish that human-written supervision prevents the failure. In Appendix I.5, the distilled student also shows generation problems, raising the unresolved question of whether the training causes broader instruction-following failures. In Section 3.5 and Table 7, the steering experiments manipulate activations near the output using directions obtained from prompts that favor different answers. The intervention could therefore change the CALL/HOLD preference without showing that the model uses the policy differently.

Questions for authors
  1. Which records produced the main result? Figure 2 describes paraphrased records, while Table 4 labels the treatment literal. Which description is correct, and does Table 4 contain the raw results underlying Figure 2?

  2. What supports the recovery claim? Which readout gives the five-record recovery threshold? Does the existing analysis show a return close to the compliant baseline on that same readout, beyond the confidence interval merely including zero?

  3. What was the actual source of the training control? Does "human-written supervision" in Table 19 refer to a separately verified human-authored dataset or to the original UltraChat responses? Which existing evaluations establish that the distilled students retain ordinary instruction-following ability despite their reported generation problems?

  4. What distinguishes policy steering from answer steering? Which existing control separates a change in how the model uses the policy from a change in its general preference for CALL versus HOLD? This distinction determines whether the intervention explains the failure or simply changes the selected answer.

Reviewer_mRTqOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper analyzes the phenomenon in which LLM agents exhibit weakened policy compliance due to their past decision-making history, even though they are aware of explicit policies. The authors constructed the LEDGER evaluation benchmark to analyze the impact of history on decision-making across multiple LLMs. The experiments show that while Qwen3-4B exhibits a significant decline in policy compliance due to a small number of violation histories, a similar phenomenon is not observed in Qwen3-14B. Furthermore, the authors report that behavioral changes resulting from interventions in internal representations, as well as distillation from a teacher model, introduce vulnerabilities in the student model.

Critical strengths
  • The issue of LLM agents prioritizing past behavioral history over explicit policies is a highly significant problem for the safety of agents operating over the long term.
  • It is interesting to observe that agents do not simply forget rules, but rather violate them while correctly recognizing them.
  • It verifies factors such as history length matching, sham subtraction, policy validity checks, token length of options, and the scope of internal representation analysis.
  • The measurement design is more meticulous than in studies that simply compare violation rates. It combines behavioral experiments, internal representation analysis, and distillation experiments.
Critical weaknesses
  • The scale and realism of the benchmark are insufficient.
  • The role of Precedent Erosion is unclear.
  • Based on current experimental results, it is difficult to determine whether distillation causes vulnerabilities.
Questions for authors
  1. This paper demonstrates that when a model is provided with a history of past decisions, its output changes in accordance with that history. However, this phenomenon is considered to be very similar to the way in which contextual examples in LLMs and agents influence the model’s output. In other words, it is unclear whether presenting the history of past actions as the model’s own decisions is the decisive factor in changing the output of an LLM agent.

  2. While benchmark designs are useful as controlled experiments, I think there is a significant gap between them and real-world LLM agents. In real-world agents, tasks such as executing tools, receiving feedback from the external environment, and searching memory occur simultaneously.

  3. The paper also reports that this effect disappears when the history is presented in a retrieval-based format, raising concerns about its generalizability to real-world applications.

  4. The answers generated by the teacher model and those from standard SFT data may differ in terms of sentence structure, reasoning patterns, and the distribution of answers. Therefore, the observed differences may not be due to the general properties of the distillation learning method, but rather to adaptation to a specific teacher distribution.

  5. Is it possible that this phenomenon could occur in models other than Qwen? I think it would be helpful to have further discussion on this.

Reviewer_xos2OpenReview ↗
Rating
3
Confidence
2
Informativeness
3
Presentation
2
Soundness
3
Read review
Paper summary

The paper considers that an agent’s past decisions to weaken its coherence to the policy used in the context in a erosion problem. Then, it introduces LEDGER, a synthetic binary-decision suite, and a sham-subtracted double-difference estimator with separate readouts for action, stated-policy identification, and reported operative policy. In Qwen3-4B, one permissive record after five compliant records produces an action SDD of +0.75; violations can coexist with correct identification of the stated policy. Qwen3-14B shows no positive erosion in the primary setting. The authors also report that bidirectional steering along a late-layer policy-removal direction changes compliance, and that response distillation from Qwen3-14B makes a Qwen3-4B student susceptible while several matched training controls show little erosion. Validation includes length-matched histories, sham subtraction, headroom, comprehension, and option-token checks, as well as seed and held-out-scenario replications. Of 17 off-the-shelf checkpoints, six pass the headroom gate. The evidence is promising, but its scope is limited by synthetic scenarios, small per-tier coverage, and template-dependent effects.

Critical strengths

• S1. A careful measurement framework makes the central behavioral finding credible within the tested setup. Equal-length controls, sham subtraction, the headroom gate, and comprehension and option-length checks address plausible sources of false positives. Separating action from stated-policy identification also lets the authors distinguish rule recognition from rule adherence. These features are consequential to my assessment because they make the main result more than a raw change in model choices; they support a weak-accept recommendation despite the limited scenario coverage.

• S2. The paper supports its behavioral finding with controlled intervention and training evidence. In Qwen3-4B, injecting and reversing the policy-removal direction changes compliance in opposite directions at the tested site and dose. Separately, response distillation from Qwen3-14B produces substantial erosion in a Qwen3-4B student, while several matched training controls show little or none, even though the teacher itself shows no positive erosion on the primary suite. This combination makes the paper more informative than a single benchmark observation and is a major reason I find it worth accepting within the scope of its experiments.

Critical weaknesses

• W1. The prevalence of the phenomenon is not established beyond a small synthetic suite. The main behavioral evidence uses 16 binary scenarios per tier, with records presented as the model’s own completed turns; only six of seventeen checkpoints pass the headroom gate. This matters because the paper’s broader governance motivation concerns agents using histories in more varied settings. The current evidence does not establish whether the measured effect persists across realistic multi-step tasks, tool use, or external memory systems. This limits the general claim and is a substantial reason to withhold a clear acceptance.

• W2. A key semantic-channel result changes under a different prompt template. For the Qwen3-14B-distilled student, the preregistered prose contrast falls from +0.94 to +0.00 on the second template, while a bare-token contrast reaches +0.67. This matters because showing that meaningful decision records affect behavior beyond literal option-token copying is central to the paper’s conceptual contribution. The main-template result remains substantial, but the template-dependent channel weakens the evidence for a general semantic precedent effect. A larger preregistered template evaluation would be needed to establish when this interpretation holds.

Questions for authors

• Q1. Appendix I.4 reports that dilution through unrelated turns leaves the effect intact, while retrieved records null it. Are the retrieved records presented in the same wording and position as the replayed prior turns, and does the retrieval condition pass the same headroom and comprehension checks? This distinction affects whether the retrieval result bears on external-memory agents or primarily on how retrieved content is framed.

• Q2. In Appendix I.1, does the +0.00 preregistered prose contrast for the Qwen3-14B-distilled student hold across the full registered dose grid with the headroom gate passed? If so, does the second-template result indicate that semantic paraphrase records no longer produce erosion in this condition, while bare option tokens do? This clarification would affect how broadly the semantic-record result can be interpreted.

• Future work suggestion A preregistered evaluation across several independently designed templates and more naturalistic memory-retrieval tasks would directly address the two limitations above. Evidence that the effect persists across these settings would materially strengthen my assessment of the paper’s broader governance claim.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.