acceptodds
Under review as a conference paper at ICLR 2027

Lesson or Noise? Do Language Models Know Which Corrections Belong in Memory?

Abstract

Persistent memory requires assistants to decide how far a user's correction should extend beyond the current task. We study this decision in SCOPE, a controlled protocol in which a fixed coding answer is followed by corrections that vary the error, stated scope and tone, before the model generates its own persistent notes. Across fifteen models and up to five memory requests, whether a correction becomes a standing rule depends strongly on how the write is elicited. Under the default request, Claude Sonnet 5 writes a standing rule in 39 of 40 sessions even when the user explicitly calls the typo a one-off. Rephrasing the request as a decision reduces these writes to 7 while retaining the stated codebase convention in all 40 sessions. In exploratory tests, eliciting a recurrence judgment before writing has opposite effects across models: it reduces Sonnet 5's rules after a plain typo correction from 40 to 12, but increases them in nine of the ten other models tested. GPT-6 Luna calls the typo a one-off in all 40 sessions and then writes a rule in 26. An exasperated correction (“This doesn't even run … Ugh.”) that asks for no change in future behavior also raises Haiku 4.5's typo rules from 3 to 38. These findings identify memory writing as a request-sensitive decision: verbal judgments alone do not reliably constrain the rules that persist. We release the protocol and all model transcripts.

Primary area:
foundation or frontier models, including LLMs
Keywords:
language model memory, user feedback, over-generalization, calibration, multi-turn interaction, LLM agents, evaluation benchmark

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_jdXLOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper looks into the question about whether LLMs know which corrections go to persistent memory. Extensive experiments are conducted to search for the answers. And there are some helpful conclusions, e.g., a valid correction does not determine the scope of the rule an assistant will save, changing the user tone can substantially change the rule, explicitly eliciting a recurrence judgment can increase rule writing immediately after a one-off judgement, and so on.

Critical strengths
  • The task and topic itself is timely, interesting and meaningful.
  • The experiments are extensive.
  • The explicit codebase-convention arm is a useful positive control.
  • The paper separates pre-specified from exploratory tests, stating the limitations honestly.
Critical weaknesses
  • The protocol tests only two synthetic error types, this cannot fully represent the wider types of real-world bugs.
  • The experiments are mostly limited to single-function python code generation tasks from HumanEval, many other scenarios are not considered, e.g., non-coding conversations, complex engineering projects, etc.
  • Only commercial models are considered, such as Claude and GPT family models, it can be also valuable to access the open-sourced models.
  • There are many other factors or parameters which might also influence the performance, such as whether open reasoning, the temperature, those details are not clearly demonstrated or controlled.
Questions for authors

See above.

1 comment

Thank you for the review. Two of the four weaknesses concern information that was in the submission but easy to miss; The other two are real limits, which we state.

Weakness 3 (only commercial models). The paper evaluates eight open-weight models next to five Claude and two GPT-6 models: Qwen3-14B, gpt-oss-20b, Gemma-3-12B, Gemma-2-9B, DeepSeek-R1-14B, Llama-3.1-8B, Qwen2.5-7B and Qwen2.5-Coder-7B (Table 2, lower block; Appendix F). The abstract did not say so; it now names the three groups. The open-weight models differ from one another as much as the commercial ones do. The two Gemma models write a rule after most typo corrections (37 and 30 of 40 sessions), DeepSeek-R1-14B in 23 of 39, and the other five rarely. Asking for a judgment first raises typo rules in six of the seven open-weight models on which we ran it, significantly in four (Table 15).

Weakness 4 (reasoning and temperature). The settings were in Section 3; a new Table 7 lists the identifier, sampling and reasoning setting of every model. Claude Sonnet 5, Opus 5, Opus 5.5 and the GPT-6 models do not accept sampling parameters and run at the API defaults; all other models are decoded greedily. The main contrast does not follow these settings. Sonnet 4.5 and Haiku 4.5 share the same settings (greedy, no thinking) and write a typo rule in 40 and 3 of 40 sessions. Among the models run at API defaults with reasoning, Sonnet 5 and Opus 5 write it in 40 sessions and GPT-6 Sol and Luna in 0 and 1. We did not vary reasoning or temperature within a model, and the paper says so.

Weaknesses 1 and 2 (two synthetic error types, single-function tasks). We agree, and the limitations say so. The design is deliberate. An injected misspelling that raises NameError under the official tests gives every model the same verifiable correction, and it is the clearest case of an error a user would call a one-off. Holding the first answer fixed removes first-turn differences between models. The cost is coverage: the paper describes SCOPE as a controlled coding setting, not a sample of real use, and makes no claim about other error types or multi-file tasks.

Reviewer_EnGkOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary
  • The paper introduces SCOPE, a controlled protocol for testing which user corrections models turn into persistent notes. It varies the error type, stated scope, tone, and memory-writing request while keeping the initial coding answer fixed.
  • Across fifteen models, memory writes depend strongly on how the request is phrased. Some models save rules despite an explicit one-off instruction. Eliciting a recurrence judgment can also disagree with, or increase, the later rule write.
  • The study includes tone controls, alternative memory requests, note-labeling audits, and limited carry-over experiments. Its clearest finding is that explicit scope can be lost when feedback is converted into memory.
Critical strengths
  • The paper asks a specific question than whether memory helps retrieval. Reflexion and memory systems such as MemGPT show why carrying information forward can help, but that does not establish what deserves to persist. Here the correction is valid, and the explicit one-off condition gives a clear test of whether the model respects its scope. That makes the result more convincing than treating every generic reminder as an error.
  • The controls make the request and tone effects worth the research community to note. The paper separates exasperation from an explicit request to be more careful, holds the correction reply fixed in additional tests, and shows that asking for a judgment is not a universal remedy. I also appreciate that it reports when a proposed intervention suppresses useful convention notes along with unwanted ones.
Critical weaknesses
  • The central measurement still has meaningful label uncertainty. The lexical classifier agrees with the human labels on 81% of non-empty sampled notes, and disagreements often concern exactly the generic typo reminders being counted. The human check uses one author, and the request variants do not have independent human validation. Since several main comparisons depend on those variants, I would like the strongest results checked with blinded annotators using a scope-based rubric.
  • The model coverage is broader than the task coverage. The study uses forty short coding tasks, two correction types, and an injected first answer rather than the evaluated model's own mistake. The decision request is also one paraphrase, not a family of matched requests. This supports a useful controlled failure case, but leaves open how reliably the same behavior appears in ordinary memory-writing situations.
Questions for authors
  • Do the main one-off/default/decide contrasts remain under independent, blinded human annotation? Please separate a lasting instruction, a record of the episode, and a generic quality reminder.
  • Among the convention notes retained by the decide and tool settings, how many preserve the stated codebase boundary rather than applying the rule to every future project? This would affect whether those settings improve selectivity rather than only the number of writes.
  • How stable is the separation across additional neutral paraphrases of the memory request? Is there evidence that the reported improvement is more than a particular wording that works for a particular model?
1 comment

Thank you for the careful review. We answer the three questions in order.

Question 1 and Weakness 1: do the contrasts remain under blinded human annotation that separates a lasting instruction, a record of the episode and a generic reminder?

Human labels. We hand-labelled a stratified sample of 150 notes from the default request without seeing the model or the condition (five sessions per model and condition; an empty file counts as no rule). Under these human labels alone:

  • Sonnet 5 has a lasting rule in 3 of 5 sampled sessions after the explicit one-off and in 4 of 5 after the plain typo correction.
  • Haiku 4.5 has one in 0 of 5 and 0 of 5, and in 4 of 5 after the harsh correction.
  • Opus 5 has one in 2 of 5 and 3 of 5.
  • All three record the stated convention (5, 5 and 4 of 5). So the default-request contrasts hold under human labels, with small counts per cell. The lexical label agrees with the human label on 81% of the 116 non-empty notes (kappa 0.63). These results are now in Appendix C and the labels are in the supplement. The sample was labelled by us, and it was drawn before the decide and tool runs, so it does not cover those requests.

The decide and tool contrasts do not need an annotator. They are mostly about whether any note is written (new Table 10): for Sonnet 5 after the one-off, a note in 39 of 40 sessions by default, 7 under decide and 1 with the tool; after the plain correction, 40 by default and 5 under decide.

Question 2: do the convention notes kept under decide and the tool preserve the codebase boundary? We coded every stated-convention note for how far it says the convention reaches (new Table 11).

  • Sonnet 5: by default 35 of 39 notes limit the convention to the user's codebase and 4 extend it across projects; under decide all 40 limit it to the codebase; with the tool 34 of 39 do and 3 extend it.
  • Opus 5: by default 31 of 40 limit it and 7 extend it; under decide all 40 limit it.
  • It is not uniform. For Haiku 4.5, decide keeps the notes but more of them state the convention with no boundary (26 of 39, against 20 by default). The default and tool requests describe the file as loaded "on any project", which may invite the extension; the paper now says so.

Question 3 and Weakness 2: paraphrases and task coverage. We have one wording of the decide request and did not run further neutral paraphrases. Two existing results bear on the question. The tool setting is a different elicitation with no request at all, and gives the same separation for Sonnet 5 (a typo rule in 6 sessions, 1 after the one-off, the stated convention in 38 of 39). Opus 5 replicates the decide pattern. We agree that model coverage is broader than task coverage: the paper now describes SCOPE as a controlled coding setting with one misspelling mechanism, one validation convention and a fixed first answer, chosen so that the correction is identical across models and verifiable.

Reviewer_rzxdOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper identifies memory writing as a request-sensitive decision: verbal judgments alone do not reliably constrain the rules that persist. It provides convincing examples of prompt-sensitive memory writing.

Critical strengths
  1. The paper presents comprehensive experiment over a wide model family to demonstrate that memory-writing outputs can change substantially with request wording and user tone (although I think this is very much expected).
Critical weaknesses
  1. The primary “standing rule” label does not distinguish an unjustified generalization from a generic quality reminder, a correctly scoped preference, or a rule that preserves the user’s restriction. Consequently, a high rule-writing rate does not by itself establish that the model has stored an inappropriate lesson.

  2. The central measurements rely on a lexical classifier with only moderate human agreement, and disagreements concentrate on the generic typo reminders central to the paper. The paper’s own semantic analysis shows that lexical rule counts can change across requests even when the number of notes containing a semantic rule does not change. Thus, I have concerns over the soundness of this paper.

Questions for authors

Are you able to respond to the weakness specified above?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.