acceptodds
Under review as a conference paper at ICLR 2027

Rewriting Is Forgetting: Zero Rewrite Capture and Budgeted Reading for Long Horizon Agent Memory

Abstract

Memory is a core component of large language model agents, enabling them to store and retrieve information across sessions. We systematically study the common practice of rewriting conversations before storage in memory systems. We build a controlled framework with three history-length regimes (no truncation, truncation, and long horizon) and evaluate this practice under a unified protocol. We find that the value of memory is truncation-dependent. When the full dialogue history fits in the model context, memory mechanisms provide no additional gain. When history is truncated, memory systems that rewrite degrade substantially, while the no-memory baseline of full-context injection remains almost unaffected. Rewriting before storage also causes a net information loss: the median storage size of rewriting systems is less than one-tenth that of verbatim storage. We term this irreversible loss the rewriting-is-forgetting effect. We therefore propose PW-MLM: the write side stores raw messages verbatim, and the read side controls query cost through temporal resolution and fixed-budget scheduling. Across three QA benchmarks spanning different history lengths, PW-MLM uses only about one-third of the injected context under moderate truncation and matches the quality of full-context injection. In the long-horizon regime, it degrades far less than rewriting-based baselines, a result that delineates the applicability boundary of zero-rewrite, budgeted reading. Our findings offer testable principles for memory design: when the full history can be injected into the context, memory mechanisms are unnecessary; when it cannot, verbatim fidelity outperforms intelligent rewriting.

Primary area:
foundation or frontier models, including LLMs
Keywords:
LLM Agents, Agent Memory, Verbatim Storage, Budgeted Reading, Long Horizon Question Answering

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_goK3OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies whether language model based rewriting improves long-term memory or irreversibly removes information. It compares full-context injection, FIFO storage, several rewriting strategies, and a proposed method called PW-MLM across AMemGym, LoCoMo, and two LongMemEval settings. PW-MLM stores user messages verbatim and adds an extracted fact tier. At query time, it ranks entries, hides older semantically similar entries, fills a fixed token budget, and answers several related questions together when possible. The paper reports that aggressive rewriting produces much smaller memory stores and lower question answering performance. PW-MLM generally approaches full-context quality while injecting fewer tokens.

Critical strengths
  1. The paper asks a specific and testable question about the write side of agent memory. The strongest evidence appears in Table 3. Removing the verbatim tier while retaining the fact tier reduces the judge score from 62.0 to 49.8 on AMemGym and from 54.9 to 35.6 on LoCoMo. This controlled ablation supports the narrower conclusion that an accessible raw archive helps recover details lost during extraction.

  2. The reporting is unusually auditable. The paper includes prompts, fixed hyperparameters, storage sizes, query tokens, category breakdowns, and code. In the roughly 120k token setting, PW-MLM obtains a judge score of 48.8 using about 3,909 injected tokens, compared with 25.2 for FIFO and at most 22.8 for the Qwen rewriting baselines. This is a meaningful result even if the paper's broader interpretation requires revision.

Critical weaknesses
  1. The experiment does not isolate rewriting from the read policy, information scope, or backbone. PW-MLM uses a 3,900 token budget, top 30 retrieval, recency weighting, temporal conflict resolution, batched answering, and a dedicated answer prompt. The Qwen rewriting baselines inject only 54 to 684 tokens per question. A fair write-side comparison would keep this read pipeline fixed and replace only raw entries with rewritten entries. The stored information also differs. PW-MLM's extraction prompt explicitly includes information stated by the assistant. The AWE prompt explicitly excludes assistant responses, while AWI only extracts user facts. Some of the assistant-side performance gap therefore follows directly from the prompt rather than from rewriting. The backbone comparison is incomplete. Qwen3.7-plus evaluates all methods except A-Mem. GLM-5.3-Flash evaluates only A-Mem and PW-MLM. The paper says A-Mem uses its official GLM backbone, but this does not explain why A-Mem cannot be run with Qwen or why the remaining baselines are absent under GLM. Backbone and memory strategy effects are therefore inseparable.

  2. This omission is consequential because A-Mem outperforms PW-MLM under GLM on every external benchmark. Its judge scores are 61.5 versus 59.0 on LoCoMo, 84.6 versus 72.1 on LME-S, and 78.0 versus 53.2 on LME-Oracle. On LoCoMo, A-Mem also uses fewer query tokens. Reclassifying A-Mem as near-verbatim after observing these results does not resolve the contradiction. It instead suggests that compression ratio or retained information, rather than rewriting as a binary property, is the relevant variable. The history regimes and cost comparison do not support several headline claims. On LoCoMo, the No Memory baseline injects 12,160 tokens from a store of 12,221 tokens. It is therefore not truncated. The experiment shows that other methods use smaller retrieval budgets, not that full-history injection is infeasible. Each history tier is also a different benchmark with different question types, history construction, and metrics. Cross-tier differences cannot be attributed only to history length. PW-MLM amortizes one context over batches of questions. On AMemGym, ten questions share a 3,900 token context, producing the reported cost of about 397 tokens per question. The paper does not say whether the No Memory and rewriting baselines receive equivalent batching. If full-context questions were batched in the same way, much of the reported 13 times cost reduction would disappear. Write-time costs are also excluded, including PW-MLM fact extraction every six turns and the construction cost of competing memory systems. Store size alone is not a measure of forgetting. Mem0 uses a much smaller store on LME-S but reaches 64.6 judge points with 584 query tokens, compared with 64.5 and 736 for PW-MLM. AWI also ties PW-MLM at 62.0 on AMemGym while using 117 rather than 397 query tokens. These results contradict the claim that PW-MLM lies on the unique Pareto frontier. All experiments use one seed. Qwen3.7-plus is simultaneously the main answering backbone, the LLM judge, and the generator of AMemGym. There is no independent human evaluation of the open-ended results.

Questions for authors
  1. Why is A-Mem not evaluated with Qwen3.7-plus, and why does the GLM-5.3-Flash panel contain only A-Mem and PW-MLM? Can the authors provide a complete model by method comparison so that backbone and memory strategy effects can be separated?

  2. Can the authors compare a verbatim store and each rewriting store under exactly the same read policy? This would require the same 3,900 token budget, top 30 retrieval, conflict resolution, batching, answer prompt, and information scope. A plain verbatim top-k baseline is also needed.

  3. Are No Memory, FIFO, AWI, AWE, Mem0, and A-Mem batched across questions in the same way as PW-MLM? Please report context tokens per model call before amortization and include write-time extraction, consolidation, embedding, and retrieval costs.

  4. On what basis is A-Mem classified as a rewriting baseline when it stores 123,472 tokens in the long setting and substantially outperforms PW-MLM? Which consistency, consolidation, eviction, and MERGE prompts were active in the reported PW-MLM runs?

Reviewer_FELCOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper compares rewriting conversations before storage with keeping the original text and selecting what to read when a question arrives. PW-MLM keeps user messages verbatim alongside extracted facts, then uses relevance, time information, conflict handling, and a fixed reading budget to select the answer context. The paper also argues that information lost during rewriting is a main cause of poor long-horizon performance. Its query-cost metric is the injected context per question, amortized when questions are answered in a batch.

Critical strengths

The paper tests a useful alternative to early compression. The interesting question is not simply whether compression loses information. It is whether selection should happen before future questions are known, or later when the system knows what it needs. Keeping the original text does not require reading the whole history every time. On LoCoMo, PW-MLM reaches a Judge score of 54.9 versus 55.7 for full-context injection, using 4,047 rather than 12,160 injected tokens. This directly supports a useful design choice: preserve details in storage and control cost at reading time.

Critical weaknesses

1. The experiments do not isolate the effect of rewriting.

Both writing and reading policies differ across systems, so the quality gaps cannot be attributed to rewriting alone. Table 3 supports retaining the raw layer within PW-MLM, but does not establish retention as the main cause of the other systems’ failures.

2. The LongMemEval settings are unclear.

Table 2 reports 4,731 No Memory tokens for LME-S and 119,761 for LME-Oracle, whereas the official definitions assign approximately 115k-token histories to S and evidence-only histories to Oracle. The dataset mapping and preprocessing need clarification because the paper’s interpretation of long-horizon degradation depends on them.

3. Table 9 contradicts the Pareto-frontier claim.

On LoCoMo with the same GLM backbone, A-Mem scores 61.5 using 3,697 injected tokens, versus PW-MLM’s 59.0 using 4,064 tokens. A-Mem is better on both measures, contradicting the claim that PW-MLM consistently lies on the Pareto frontier.

4. The batching argument overlooks the shared budget.

Pooling candidate sets before applying budget BB does not guarantee that every question retains its required evidence afterward. Table 3 reports unbatched quality but omits the corresponding token costs, leaving the contribution of batching to the reported savings unclear.

Questions for authors

For a small set of representative failures, could the authors check whether the needed evidence was removed during writing, remained in the store but was not retrieved, or was retrieved but not used correctly? This would help support or narrow the proposed explanation.

Reviewer_n7JfOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

The paper investigates whether transforming conversations into compact memories discards evidence needed for future questions. It proposes PW-MLM, which stores user messages verbatim alongside periodically extracted facts, then applies semantic retrieval, temporal filtering, and token-budgeted context selection.

Critical strengths
  1. The verbatim-layer ablation (Table 3) isolates the cost of replacing raw text with LLM-extracted facts. It is also consistent with prior findings, such as LongMemEval’s comparison of raw versus summarized memory values.
Critical weaknesses
  1. The paper says the gap between rewriting and verbatim storage “remains unobserved”, but LongMemEval already found that storing summaries or extracted facts instead of raw sessions lowers QA accuracy. LongMemEval’s recommended design, raw values plus fact-augmented keys, is essentially PW-MLM’s two tiers.
  2. For the experiments, full context never becomes infeasible, it injects up to 120k tokens and is the best or second-best strategy in every tier (Tables 1–2) therefore the authors claim that “when it cannot, verbatim fidelity outperforms intelligent rewriting” is never tested
  3. “Truncation” is never defined, and the “long-horizon” LME-S has the shortest history of all (5,495 tokens).
  4. Mem0 scores 33.3 on LoCoMo here vs. 66.88 (92.5 on April 2026) in its own paper, and AWI’s “0” store is never explained.
  5. PW-MLM gets more reading budget and batching and a different write policy, so its wins can’t be credited to the write policy alone. The paper wants to show that storing raw text (verbatim) keeps information that LLM rewriting throws away. To show that cleanly, only how memory is written should differ between PW-MLM and the baselines.
Questions for authors
  1. Can you measure retention and conflict resolution directly?
  2. How much of the store is rewritten or deleted?
  3. Is the judge human validated? 
  4. Please report LLM calls and input/output tokens for writes and reads, plus latency per conversation.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.