acceptodds
Under review as a conference paper at ICLR 2027

CoEM: Empowering Long-Context Reasoning with Commit-on-Evidence Memory

Abstract

Long-context reasoning is essential for complex and long-horizon tasks, yet the performance of large language models (LLMs) degrades as context length increases. Recent approaches address this by processing input chunk by chunk while maintaining a bounded textual memory in model context. However, premature information compression can discard critical details essential for subsequent reasoning. In this paper, we introduce Commit-on-Evidence Memory (CoEM), which learns when to convert source evidence into compact memory facts. Specifically, under a fixed context-memory budget, CoEM preserves potentially useful source excerpts verbatim in a pending set, allowing subsequent context to clarify their relevance before irreversible compression. As new context arrives, a learned policy revisits each pending excerpt and decides whether to promote it to the committed memory, retain it for further consideration, or discard it. A frozen verifier ensures proposed facts are accepted only if supported by retained excerpts and current context. To further guide effective memory management, we train this policy using reinforcement learning by combining fine-grained, step-level evidence rewards with final answer rewards.Extensive experiments demonstrate that CoEM consistently improves long-context reasoning. When evaluated on 6,400 documents long-context input, CoEM outperforms the strongest memory baseline by 10.4–11.4 F1 points on Qwen3.5-9B. We submit the code with the paper for reproducibility, and it will be publicly available.

Primary area:
foundation or frontier models, including LLMs
Keywords:
large language models, long-context reasoning, memory agent, long-context agent, reinforcement learning, multi-hop question answering

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_AyfXOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

To address the issue of premature memory compression in long-context streaming inference—which can lead to the loss of potentially critical evidence—this paper proposes CoEM. Within a fixed memory budget, CoEM maintains both committed facts and pending raw text segments. It employs reinforcement learning to decide whether to retain, commit, or discard pieces of evidence, while utilizing a frozen natural language inference (NLI) verifier to check the source support for any facts proposed for commitment. The core contribution lies in integrating raw text buffering, deferred commitment, and source verification into a recurrent memory management framework, supported by a training strategy that combines final-answer rewards with step-level evidence feedback. The authors evaluate the method using Qwen3.5-4B and Qwen3.5-9B models across three multi-hop question-answering benchmarks and document streams of varying lengths. The results demonstrate that CoEM outperforms memory-based baselines with equivalent budget constraints; specifically, the 9B model achieves an improvement of 10.4–11.4 F1 points when processing a stream of 6,400 documents. Furthermore, ablation studies, evidence-interval testing, and verifier-based audits provide a detailed analysis of the roles and limitations of each component.

Critical strengths

The paper proposes a reasonable and feasible memory mechanism designed to address critical failure modes. Within a fixed memory budget, CoEM retains segments of the original text whose utility is not yet clear, allowing this information to inform subsequent decision-making. Furthermore, it employs source verification to mitigate the risk of incorporating unsupported facts into memory. This design directly tackles the issue where the significance of certain evidence only becomes apparent later—after the original details have already been compressed or lost—which is the primary reason I find the technical approach valuable.

The performance improvements are consistent, and the key findings are well-supported by experimental evidence. Across two model scales, three multi-hop question-answering benchmarks, and various input lengths, CoEM consistently outperforms the memory-based baselines (which are matched for budget). Notably, the 9B model achieves an improvement of 10.4–11.4 F1 points on the longest input sequences. The inclusion of statistics from three different random seeds, paired confidence intervals, and additional comparisons against the original ReMemR1 further bolsters the credibility of these results. This evidence confirms the system's practical effectiveness for the tasks evaluated and serves as a significant positive factor in the overall assessment.

Critical weaknesses

The independent contribution of the core mechanism has not been sufficiently validated. While the paper emphasizes "learning when to commit," the ablation study—which removes the pending set—simultaneously alters the retention method and the allocation of the memory budget. Furthermore, a fixed two-step delay is inadequate as a representative baseline for competitive commitment strategies. Consequently, it is difficult to discern whether performance gains stem from learning the optimal timing for commitment or simply from retaining a larger volume of uncompressed information. The absence of comparisons against strong extractive caching baselines—under equivalent budget and validator constraints—as well as the lack of comparisons with well-tuned waiting strategies, leaves the necessity and incremental value of the core innovation unclear.

The evaluation fails to adequately address the critical challenge of capacity competition. Each dataset is assessed using only 128 questions, with the full supporting text capped at 4,096 tokens and distractor passages truncated to 96–128 tokens. Increasing the input length primarily serves to increase the volume of distractors, without simultaneously testing the system’s ability to handle a greater total amount of supporting information or more complex chains of reasoning. Moreover, while the stress tests involving dense source evidence report capacity-related events and discard rates, they fail to provide the corresponding answer accuracy. As a result, the current evidence is insufficient to determine whether CoEM can maintain its advantage when a large body of relevant evidence—or multiple concurrent, pending reasoning chains—compete for limited memory. This limits both the scope of the core conclusions and their overall persuasiveness.

Questions for authors

Given identical memory budgets, verifiers, and training conditions, what performance gains can be achieved by learning the timing of "commit" actions, as opposed to using robust extractive caching or a tuned waiting strategy?

As the amount of supporting evidence increases—and multiple pending chains of reasoning compete for limited memory—how do the Answer F1 score and the loss rate of critical evidence for CoEM evolve? Does the model maintain its advantage relative to the baseline?

5 comments
AuthorsOpenReview ↗

Response to Reviewer AyfX (1/4): W1 & Q1

Thank you for the careful review and for recognizing the soundness of the mechanism and the consistency of the gains. Both of your questions ask for tests of the core mechanism under equal conditions, and we added the controls and stress tests below. Because the discussion period is short, all new experiments use Qwen3.5-4B, and there was not enough time to repeat them with Qwen3.5-9B. All changes are marked in red in the revised PDF.

W1 & Q1. Gains from learning commitment timing under identical budgets, verifiers, and training

Reviewer AyfX, Weakness 1

The independent contribution of the core mechanism has not been sufficiently validated. While the paper emphasizes “learning when to commit,” the ablation study, which removes the pending set, simultaneously alters the retention method and the allocation of the memory budget. Furthermore, a fixed two-step delay is inadequate as a representative baseline for competitive commitment strategies. Consequently, it is difficult to discern whether performance gains stem from learning the optimal timing for commitment or simply from retaining a larger volume of uncompressed information. The absence of comparisons against strong extractive caching baselines, under equivalent budget and validator constraints, as well as the lack of comparisons with well-tuned waiting strategies, leaves the necessity and incremental value of the core innovation unclear.

Reviewer AyfX, Question 1

Given identical memory budgets, verifiers, and training conditions, what performance gains can be achieved by learning the timing of “commit” actions, as opposed to using robust extractive caching or a tuned waiting strategy?

Response.

We agree that two controls were missing. The eager variants were designed as ablations inside the CoEM controller. They remove the pending set and give its 256 tokens to committed memory (BPB_P = 0, BMB_M = 1,024), so that the total budget stays at 1,024 tokens. This keeps the comparison budget-matched, but, as you point out, it changes the retention method and the budget split at the same time. The fixed two-chunk delay was meant as the simplest timing rule, not as a tuned waiting strategy. To separate learned timing from these choices, we added the following controls. The extractive cache and the waiting strategies use the same budget split as CoEM (BPB_P = 256, BMB_M = 768), the same frozen verifier (η = 0.90), and the same RL training with Qwen3.5-4B. The cache changes how verbatim evidence is retained, and the waiting strategies change only when promotion is first attempted.

  • Robust extractive cache. At every chunk, source sentences are rescored by their similarity (BGE-M3) to the question and to committed memory. The highest-scoring sentences are kept within 256 tokens and the lowest-scoring ones are evicted. The policy writes facts from the cache and the current chunk through the same verifier.

  • Tuned waiting strategies. These keep the retention, retry, and capacity rules of CoEM and fix only when promotion is first attempted. We use a fixed delay tuned on the development set over 1, 2, 4, and 8 chunks (selected: 4 chunks), and an entity-triggered wait that attempts promotion when the current chunk mentions an entity that also appears in the pending excerpt. After this first attempt, each policy makes its own KEEP, retry, and DROP decisions under the same limits as CoEM.

  • Reference variants. We also include ReMemR1† with a 256-token verbatim buffer, which keeps the unverified memory updates of ReMemR1†, the two-chunk delay with matched retry, and the eager variant of Table 2, which has no pending set.

AuthorsOpenReview ↗

Response to Reviewer AyfX (2/4): W1 & Q1 (continued)

Table R10. Learned timing against extractive caching, tuned waiting strategies, and reference variants (Qwen3.5-4B). Revised Table 2 and new Table 18 in Appendix F.5.

Variant (answer F1)HotpotQA 8002Wiki 800HotpotQA 6,400Pending tokens
Component ablation (Table 2)
Eager + verifier (BPB_P = 0)77.266.070.60
Verbatim-retention alternatives
ReMemR1† + verbatim buffer82.171.976.0251
Robust extractive cache81.871.676.9247
Matched timing controls (waiting strategies)
Fixed two-chunk delay + matched retry82.872.879.2203
Tuned delay (4 chunks) + matched retry83.073.179.6221
Entity-triggered wait + matched retry83.273.480.1196
CoEM (learned timing)85.576.184.2138

Pending tokens is the mean occupancy of verbatim text at 6,400 documents on HotpotQA. The extractive cache and the waiting strategies use the same budget split, verifier, and RL training as CoEM. ReMemR1† + verbatim buffer keeps the unverified memory updates of ReMemR1†.

Answer to Q1.

  • Against robust extractive caching, learning commitment timing gains 3.7 / 4.5 points (HotpotQA / 2WikiMultiHopQA, 800 documents) and 7.3 points at 6,400 documents.

  • Against the best tuned waiting strategy, it gains 2.3 / 2.7 points at 800 documents and 4.1 points at 6,400 documents.

  • The gains do not come from keeping more uncompressed text. At 6,400 documents, the pending set of CoEM holds 138 tokens on average, fewer than the extractive cache (247) and the best waiting strategy (196).

Why learned timing helps. Table R11 tracks supporting evidence in the source-to-bridge gap intervention, where 32 or 128 chunks separate a source excerpt from the passage that links it to the question.

Table R11. Retention and commitment timing of supporting evidence (Qwen3.5-4B, HotpotQA, 6,400 documents). New Table 19 in Appendix F.5.

VariantSupporting excerpts still verbatim when the bridge arrives (gap 128)Committed at the bridge step or the next step (gap 32)
Robust extractive cache52.7%23.8%
Fixed two-chunk delay + matched retry18.6%6.1%
Tuned delay + matched retry27.9%10.4%
Entity-triggered wait + matched retry71.2%54.6%
CoEM89.8%81.3%

The extractive cache scores evidence by its similarity to the question, but delayed evidence has low similarity when it arrives, so about half of it is evicted before the bridge. Fixed delays promote supporting excerpts long before the bridge and compress them while their relevance is unclear. The entity trigger misses links through aliases. CoEM keeps 89.8% of supporting excerpts until the bridge and commits 81.3% of them at the bridge step or the next step. It therefore compresses evidence close to the moment when later context makes its role clear, which is what a fixed rule cannot do.

Effect of the budget split. We also retrained all memory agents at total budgets of 512, 1,024, and 2,048 tokens. CoEM leads at every budget, and CoEM with 512 tokens (80.1 / 69.4) exceeds every baseline with 2,048 tokens (best 78.9 / 68.5). This further shows that the advantage of CoEM does not come from a larger amount of retained memory.

Revision. Table 2 adds the matched controls, Section 4.4 limits the claim to the matched comparisons, and Appendices F.4 and F.5 give the details.

AuthorsOpenReview ↗

Response to Reviewer AyfX (3/4): W2 & Q2

W2 & Q2. More supporting evidence and multiple pending reasoning chains competing for limited memory

Reviewer AyfX, Weakness 2

The evaluation fails to adequately address the critical challenge of capacity competition. Each dataset is assessed using only 128 questions, with the full supporting text capped at 4,096 tokens and distractor passages truncated to 96–128 tokens. Increasing the input length primarily serves to increase the volume of distractors, without simultaneously testing the system’s ability to handle a greater total amount of supporting information or more complex chains of reasoning. Moreover, while the stress tests involving dense source evidence report capacity-related events and discard rates, they fail to provide the corresponding answer accuracy. As a result, the current evidence is insufficient to determine whether CoEM can maintain its advantage when a large body of relevant evidence—or multiple concurrent, pending reasoning chains—compete for limited memory. This limits both the scope of the core conclusions and their overall persuasiveness.

Reviewer AyfX, Question 2

As the amount of supporting evidence increases—and multiple pending chains of reasoning compete for limited memory—how do the Answer F1 score and the loss rate of critical evidence for CoEM evolve? Does the model maintain its advantage relative to the baseline?

Response.

Our evaluation protocol is as follows. Each dataset has 128 questions, the full supporting text is at most 4,096 tokens, and each distractor is the longest complete-sentence prefix of 96 to 128 tokens. We follow the protocol of ReMemR1 so that results are directly comparable with prior work. It also guarantees that the full-context baseline sees all supporting text, and it makes each shorter context a subsequence of every longer one, which allows paired comparisons across lengths. We agree that under this protocol a longer input mainly adds distractors. We therefore added the following tests with Qwen3.5-4B at 6,400 documents.

We measure the loss rate of critical evidence as the share of annotated supporting relations, already observed when the last chunk has been read, that cannot be recovered from the retained text. For CoEM it is measured before final resolution.

(1) Answer F1 under pending-set stress. Our stress test reported capacity events and forced drops under three orderings of the 2WikiMultiHopQA stream: the ordinary ordering of the main evaluation, a gap of 128 chunks between each source excerpt and its bridge evidence, and a source-dense ordering that places many unresolved source excerpts close together so that they compete for the pending set at once. Table R12 adds answer F1 and the loss rate of critical evidence.

Table R12. Answer F1 under pending-set stress (Qwen3.5-4B, 2WikiMultiHopQA, 6,400 documents). New Table 20 in Appendix F.6.

ConditionMemAgentGRU-MemReMemR1†CoEMCoEM forced dropsCritical-evidence loss: CoEM / best baseline
Ordinary ordering54.661.164.473.90.5%15.1% / 33.6%
Gap of 128 chunks37.841.943.065.61.8%19.6% / 52.3%
Source-dense stress46.152.355.466.97.6%23.7% / 44.0%
Reviewer_ta1sOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

This paper introduces Commit-on-Evidence Memory (CoEM), a recurrent memory framework for long-context question answering. Its central motivation is that evidence encountered early in a document stream may become relevant only after later information arrives, making immediate lossy compression potentially harmful. CoEM divides a 1,024-token memory between committed facts and pending verbatim excerpts, using an RL-trained policy to manage evidence and a frozen NLI verifier to validate updates. Experiments with Qwen3.5-4B and Qwen3.5-9B on HotpotQA, 2WikiMultiHopQA, and MuSiQue show consistent improvements across 50–6,400-document contexts, with larger gains at longer lengths and extensive supporting analyses.

Critical strengths
  • The paper is well-presented and easy to follow.
  • The distinction between retaining unresolved source material and storing compact committed facts is useful. Recovering a historical summary cannot restore information omitted when that summary was first created, whereas retaining a verbatim excerpt preserves the opportunity to make a better compression decision later.
  • Experimental gains are substantial. CoEM achieves statistically supported gains of 10.4-11.4 F1 points over budget-matched baselines at 6,400 documents with Qwen3.5-9B, transfers to two QA datasets without additional training, and outperforms native ReMemR1 by 2.3-4.3 points in length-averaged F1.
Critical weaknesses
  • The novelty should be framed more precisely. The distinction between recovering overwritten summaries and preserving details never included in a summary is a useful contribution. However, identifying delayed evidence relevance itself is not new: ReMemR1 already explicitly discusses early evidence whose significance becomes apparent only after later context, describing this as premature pruning of latent evidence. It also combines final-answer rewards with intermediate memory feedback.

  • The ablations do not fully isolate adaptive commitment timing because fixed-step pending also changes retention and retry rules, while the eager variant underperforms stronger existing baselines. Matching retry behavior and adding competitive baselines that retain verbatim excerpts alongside compact facts would clarify whether CoEM’s gains arise from learned timing, source retention, or other controller choices.

Questions for authors
  • Does the fixed-step baseline receive the same opportunities as full CoEM to retain and retry rejected or capacity-infeasible promotions? If not, can the authors provide a matched comparison, or qualify the claim that its performance gap isolates adaptive commitment timing?

  • Which policy checkpoints, datasets, and context lengths generate the 2,000 audit proposals? Can the authors report faithfulness for final retained entries and for multi-source claims after policy optimization, particularly at 6,400 documents?

6 comments
AuthorsOpenReview ↗

Response to Reviewer ta1s (1/5): W1

Thank you for the positive assessment and the precise suggestions. Your distinction between recovering overwritten summaries and preserving details never included in a summary helped us state our contribution more sharply. Because the discussion period is short, all new experiments use Qwen3.5-4B, and there was not enough time to repeat them with Qwen3.5-9B. All changes are marked in red in the revised PDF.

W1. Framing the novelty more precisely

Reviewer ta1s, Weakness 1

The novelty should be framed more precisely. The distinction between recovering overwritten summaries and preserving details never included in a summary is a useful contribution. However, identifying delayed evidence relevance itself is not new: ReMemR1 already explicitly discusses early evidence whose significance becomes apparent only after later context, describing this as premature pruning of latent evidence. It also combines final-answer rewards with intermediate memory feedback.

Response.

We agree that ReMemR1 first discussed early evidence whose importance appears only after later context, which it calls premature pruning of latent evidence, and that it combines final-answer rewards with intermediate memory feedback. The revision credits ReMemR1 for this observation in Sections 1 and 2. We would like to clarify what CoEM adds, because it targets a different loss and introduces a different decision.

  1. A loss that callbacks cannot recover. ReMemR1 treats the loss as a retrieval problem. Evidence was written into an earlier memory and later overwritten, so a callback can bring it back. We identify that much of the loss happens one step earlier, when a chunk is first compressed. A detail whose relevance is still unknown is simply not written, so no saved memory ever contains it and no callback can return it. As you note, this distinction is useful, and it is the starting point of CoEM. Reviewer AyfX also names it as the main reason the approach is valuable, namely that it handles evidence whose significance appears only after the original details have been compressed or lost, and Reviewer xXBR finds keeping the original excerpt until that point sensible. We measured this loss directly (new Appendix F.8). For native ReMemR1 with its full archive of past memories (Qwen3.5-4B, HotpotQA, 6,400 documents), 73.6% of the supporting relations missing at answer time never appeared in any archived memory, so the callback had nothing to return.

  2. A new decision: when to compress. Existing recurrent memory agents, including ReMemR1, decide what to write when a chunk arrives. To our knowledge, CoEM is the first recurrent memory agent that keeps unresolved evidence verbatim within the same fixed context-memory budget and learns when to compress it into a committed fact. The commitment time of each excerpt is a stopping time chosen by the policy (Equation 4), not a fixed rule.

  3. Support and relevance are judged separately. The frozen verifier only checks whether a proposed fact is supported by its cited excerpts. Whether the evidence matters, and when to commit it, is decided by the learned policy.

  4. Feedback without the reference answer. ReMemR1 scores memory updates and callbacks by how much of the reference answer the memory recovers. The step-level evidence reward of CoEM does not use the reference answer. It penalizes unsupported facts and invalid operations, and its return-to-go passes these penalties back to earlier admission and KEEP decisions. This gives a learning signal even when the final answer is wrong (Theorem 3), and return-to-go outperforms a current-chunk penalty by 2.4 / 2.9 points (Table 10).

AuthorsOpenReview ↗

Response to Reviewer ta1s (2/5): W1 (continued)

Table R6. How CoEM differs from the most related methods.

MethodKept after a chunk is readCan a detail left out of a summary be recovered later?When evidence is compressedIntermediate feedback
MemAgentOverwritten textual memoryNoWhen the chunk arrivesNone (final-answer reward only)
ReMemR1Current memory and archived past memories for callbackNo. A callback returns only what was writtenWhen the chunk arrivesHow much of the reference answer the memory recovers
CoEMCommitted facts and verbatim pending excerpts in one fixed budgetYes, while the excerpt is pendingLearned, once later context clarifies relevanceSource support checked by a frozen verifier, without the reference answer, passed back by return-to-go

Evidence that this difference matters. On the 49 of 128 HotpotQA questions that contain delayed evidence relevance under the annotation rule of Appendix A.5, CoEM leads native ReMemR1 by 12.6 F1 points, compared with 4.8 points on the other questions (Qwen3.5-4B, 6,400 documents). Table R7 shows the trend over context length. Even with its full archive, native ReMemR1 loses 8.4 to 10.3 points from 50 to 6,400 documents, whereas CoEM loses only 1.0 to 2.1 points. Recall over saved memories slows the decline only a little, because the missing details were never saved.

Table R7. Answer F1 as the context grows (Qwen3.5-9B). New Table 23 in Appendix F.8.

MethodHotpotQA 50 / 800 / 6,400Drop2Wiki 50 / 800 / 6,400DropMuSiQue 50 / 800 / 6,400Drop
ReMemR1†88.3 / 84.7 / 78.79.680.9 / 74.5 / 69.611.372.6 / 66.8 / 61.611.0
ReMemR1 (native)89.9 / 86.8 / 81.58.482.6 / 76.8 / 72.310.374.3 / 68.9 / 64.010.3
CoEM90.1 / 89.4 / 89.11.083.0 / 81.5 / 81.02.074.4 / 73.6 / 72.32.1

ReMemR1† and CoEM are from Table 1. Native ReMemR1 values are the per-length results of the runs summarized in Table 4.

Revised first contribution (Section 1): We identify a limitation that memory callbacks cannot address, since a detail omitted when a chunk is first compressed never enters any saved memory. This is the first work to learn when to compress source evidence into memory facts under a fixed context-memory budget.

Added to Section 2: ReMemR1 notes that early evidence may be pruned before its importance appears and adds callbacks over historical memory snapshots (Shi et al., 2025). [...] A callback returns only what an earlier memory recorded, so a detail omitted when that memory was written remains lost.

AuthorsOpenReview ↗

Response to Reviewer ta1s (3/5): W2 & Q1

W2 & Q1. Do the fixed-step baseline and CoEM get the same chances to retain and retry?

Reviewer ta1s, Weakness 2

The ablations do not fully isolate adaptive commitment timing because fixed-step pending also changes retention and retry rules, while the eager variant underperforms stronger existing baselines. Matching retry behavior and adding competitive baselines that retain verbatim excerpts alongside compact facts would clarify whether CoEM’s gains arise from learned timing, source retention, or other controller choices.

Reviewer ta1s, Question 1

Does the fixed-step baseline receive the same opportunities as full CoEM to retain and retry rejected or capacity-infeasible promotions? If not, can the authors provide a matched comparison, or qualify the claim that its performance gap isolates adaptive commitment timing?

Response.

Short answer: no. As defined in Appendix A.2, the Fixed-step pending variant attempts promotion once, two chunks after admission or at final resolution if that comes first, and it drops records whose promotion is rejected or capacity-infeasible. Full CoEM can KEEP such a record and retry in later chunks under the same limits H, K, and J and the same capacity rules. The 3.2 / 3.9-point gap in Table 2 therefore mixes commitment timing with the retry rule, and our original claim that it isolates adaptive commitment timing was too strong. We have qualified the claim in Section 4.4 and added the matched comparison you suggested.

Setup. We compare CoEM with two groups of new variants, and each group tests a different question. Both groups use the same budget split as CoEM (BPB_P = 256, BMB_M = 768). Table R8 also lists two component ablations of Table 2 for reference.

  1. Matched retry behavior. These matched timing controls use the same frozen verifier (η = 0.90), the same retention and retry rules, and the same RL training as CoEM. A rule sets only the step of the first promotion attempt: two chunks after admission, a delay tuned on the development set over 1, 2, 4, and 8 chunks (selected: 4 chunks), or an entity-triggered wait that attempts promotion when the current chunk mentions an entity that also appears in the pending excerpt. After this first attempt, each policy makes its own KEEP, retry, and DROP decisions under the same limits H, K, and J as CoEM. These controls test whether learned commitment timing matters.

  2. Competitive baselines that retain verbatim excerpts alongside compact facts. ReMemR1† + verbatim buffer reserves 256 of its 1,024 tokens for a first-in-first-out buffer of verbatim excerpts and keeps the memory updates of ReMemR1†, which are not verified. Extractive cache rescores source sentences at every chunk by their similarity to the question and to committed memory, keeps the top-scoring sentences within 256 tokens, and writes facts through the same verifier with the same RL training. These two verbatim-retention alternatives select verbatim text by fixed rules rather than by PROMOTE, KEEP, and DROP decisions, so they test whether keeping source text alone explains the gain.

Reviewer_xXBROpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes COEM, a recurrent memory method for long-context question answering. Instead of immediately compressing every potentially useful passage into memory, COEM keeps some source excerpts in a bounded pending set. As later chunks arrive, a learned policy decides whether to promote each excerpt into a compact fact, keep it pending, or drop it. A frozen NLI verifier checks proposed facts against available source text before they enter committed memory. The policy is trained using final-answer rewards and step-level penalties for rejected facts and invalid operations. On HotpotQA, 2WikiMultiHopQA, and MuSiQue, COEM outperforms the tested recurrent-memory baselines, with larger gains at longer input lengths.

Critical strengths

Premature compression is a meaningful problem: an early passage can contain a detail whose relevance becomes clear only after a later passage establishes the connection to the question. Keeping the original excerpt until that point is a sensible approach. The paper also provides useful ablations for pending retention, verification, and the evidence reward, as well as a controlled experiment varying the gap between related passages.

Critical weaknesses
  1. The cost comparison is incomplete. COEM repeatedly includes pending excerpts in policy inputs. The paper reports end-to-end latency against recurrent-memory baselines and gives token and call counts for one COEM example, so it would be inaccurate to say that it provides no efficiency comparison. However, it does not provide comparable aggregate input-token, generated-token, and call counts for every method tested. Those measurements are needed to understand the cost of repeatedly reconsidering pending evidence, alongside the reported F1 gains.
  2. All three evaluation benchmarks are Wikipedia-based multi-hop QA datasets. Evaluation on more recent and diverse sources is needed to establish whether the memory policy generalizes beyond this setting. The reported out-of-distribution results change the QA dataset, but do not resolve this source-domain limitation.
  3. The comparisons focus on full-context inference and recurrent-memory agents. A comparison with a strong agentic RAG approach that can retrieve or reopen source passages would help establish when carrying verbatim excerpts in a small pending set is preferable to retrieving evidence when needed. Such a comparison should report answer quality together with retrieval and inference costs, while making the different storage assumptions explicit.
  4. The paper varies COEM’s allocation between pending and committed memory but does not compare COEM and the existing methods across different total memory budgets.
Questions for authors
  1. Can you report aggregate policy input tokens, generated tokens, and model-call counts for COEM and each baseline at the tested context lengths, in addition to latency?
  2. How does COEM perform on long-context tasks using more recent, non-Wikipedia sources?
  3. How does COEM compare with an agentic RAG baseline that can retrieve or reopen source passages, when answer quality and total cost are both considered?
  4. How do the relative results change as the total retained-memory budget varies?
5 comments
AuthorsOpenReview ↗

Response to Reviewer xXBR (1/4): W1 & Q1

Thank you for the careful review. You point out that an early passage can hold a detail whose relevance becomes clear only after a later passage links it to the question, and that keeping the original excerpt until then is sensible. This is exactly the problem CoEM targets. Below we answer your four questions in turn. Because the discussion period is short, all new experiments use Qwen3.5-4B, and there was not enough time to repeat them with Qwen3.5-9B. All new results are in the revised PDF (new Appendix F), with changes marked in red.

W1 & Q1. Aggregate input tokens, generated tokens, and call counts for every method

Reviewer xXBR, Weakness 1

The cost comparison is incomplete. COEM repeatedly includes pending excerpts in policy inputs. The paper reports end-to-end latency against recurrent-memory baselines and gives token and call counts for one COEM example, so it would be inaccurate to say that it provides no efficiency comparison. However, it does not provide comparable aggregate input-token, generated-token, and call counts for every method tested. Those measurements are needed to understand the cost of repeatedly reconsidering pending evidence, alongside the reported F1 gains.

Reviewer xXBR, Question 1

Can you report aggregate policy input tokens, generated tokens, and model-call counts for COEM and each baseline at the tested context lengths, in addition to latency?

Response.

Setup. We counted policy calls, policy input tokens, and generated tokens per question for all five methods in the same setting as our latency study: Qwen3.5-4B, HotpotQA, the same 128 questions, one H800 GPU, batch size one, and no prefix caching across requests. Table R1 gives the results at 6,400 documents and Table R2 at every timed length. Latency and F1 are taken from Tables 8 and 1 of the paper.

Table R1. Mean cost per question at 6,400 documents (Qwen3.5-4B, HotpotQA). New Table 13 in Appendix F.1.

MethodPolicy callsInput tokensGenerated tokensVerifier pairsLatency (s)F1
MemAgent129.0792K112.7K072071.9
GRU-Mem129.0744K41.0K047475.4
ReMemR1† (budget-matched)129.0822K155.6K087074.3
ReMemR1 (native)129.0901K169.2K095176.5
CoEM140.91,018K21.4K35754884.2

Policy calls include the final-answer call. A single-pass baseline makes one call per chunk plus the answer call, so its count is the mean number of chunks plus one and is the same for all four baselines. GRU-Mem runs with early exit disabled, so it also reads every chunk. ReMemR1 generates its callback query in the same call as its memory update and recalls past memories without a policy call, so callbacks add input and generated tokens but no calls. For native ReMemR1, input tokens include 94.7K recalled tokens held outside the 1,024-token budget.

Table R2. Mean cost per question at 800 / 1,600 / 3,200 / 6,400 documents (Qwen3.5-4B, HotpotQA). New Table 14 in Appendix F.1.

MethodPolicy callsInput tokensGenerated tokens
MemAgent17.1 / 33.0 / 65.0 / 129.098K / 196K / 393K / 792K12.4K / 25.8K / 54.1K / 112.7K
GRU-Mem17.1 / 33.0 / 65.0 / 129.093K / 185K / 371K / 744K4.0K / 8.4K / 18.9K / 41.0K
ReMemR1†17.1 / 33.0 / 65.0 / 129.0103K / 205K / 410K / 822K17.5K / 35.8K / 74.8K / 155.6K
ReMemR1 (native)17.1 / 33.0 / 65.0 / 129.0107K / 218K / 443K / 901K18.9K / 38.6K / 80.8K / 169.2K
CoEM19.5 / 36.7 / 71.5 / 140.9129K / 253K / 507K / 1,018K2.7K / 5.2K / 10.3K / 21.4K

Because each stream draws thousands of distractors from the same pool, the number of chunks varies little across questions with the same document count. On HotpotQA, the mean number of chunks is 16.1, 32.0, 64.0, and 128.0 at 800, 1,600, 3,200, and 6,400 documents.

AuthorsOpenReview ↗

Response to Reviewer xXBR (2/4): W1 & Q1 (continued), W2 & Q2

Findings.

  1. Carrying pending evidence adds little input. Pending excerpts and committed facts share one 1,024-token context-memory budget, so each CoEM call carries no more memory tokens than a baseline call. At 6,400 documents, the mean memory occupancy of CoEM is 770 tokens, compared with 900 for MemAgent, and the pending set holds only 138 tokens, about 1.9% of the input of one call. Of the 226K extra input tokens relative to MemAgent, 87K come from 11.9 extra calls per question for capacity handling and final resolution (8.4% of all calls). The rest comes from the longer action format, source offsets, and short controller feedback.

  2. Generated tokens explain the latency. CoEM generates 5.3× fewer tokens than MemAgent and 7.3× fewer than ReMemR1†. CoEM outputs compact actions, namely source offsets, short facts, and PROMOTE, KEEP, or DROP decisions, and the controller copies source text by offsets. MemAgent and ReMemR1 instead rewrite their whole memory at every step. Decoding runs token by token, so one generated token costs far more time than one input token. This is why CoEM is 24% to 42% faster than MemAgent and both ReMemR1 variants even though it reads more input.

  3. Comparison with the fastest baseline. Compared with GRU-Mem, CoEM reads 37% more input tokens and generates 48% fewer tokens. Its latency is 15.6% higher, and its F1 is 8.8 points higher. The frozen verifier checks 357 source-fact pairs per question, which take about 1.5 s or 0.3% of the latency. All costs grow linearly with the number of documents. This matches Theorem 1 in Appendix D.2, which shows that CoEM needs several policy and verifier calls proportional to the number of chunks.

Revision. Tables R1 and R2 are added as Appendix F.1, and the efficiency paragraph of Section 4.3 now explains where the cost comes from.

W2 & Q2. Performance on more recent, non-Wikipedia sources

Reviewer xXBR, Weakness 2

All three evaluation benchmarks are Wikipedia-based multi-hop QA datasets. Evaluation on more recent and diverse sources is needed to establish whether the memory policy generalizes beyond this setting. The reported out-of-distribution results change the QA dataset, but do not resolve this source-domain limitation.

Reviewer xXBR, Question 2

How does COEM perform on long-context tasks using more recent, non-Wikipedia sources?

Response.

Our benchmark choice follows the recurrent memory agents we build on. MemAgent and GRU-Mem evaluate on HotpotQA, SQuAD, and synthetic RULER tasks, and ReMemR1 evaluates on HotpotQA and 2WikiMultiHopQA, so this line of work has so far used Wikipedia-based QA and synthetic tasks. We agree that this leaves the source domain open, and that our out-of-distribution results change the QA dataset but not the source domain. We therefore evaluated CoEM on two long-context benchmarks with non-Wikipedia sources.

Setup. All memory agents use the same checkpoints trained on HotpotQA, with no additional training and the same chunk size and memory budgets (C = 5,000, B = 1,024, BPB_P = 256, BMB_M = 768). Reasoning mode is disabled for every method, as in all our experiments.

  • LongBench v2 (Bai et al., 2025). 503 multiple-choice questions written in 2024 over financial reports, legal documents, government reports, academic papers, novels, code repositories, dialogue histories, and structured data. It is split by length into Short (180), Medium (215), and Long (108) questions, and the metric is accuracy.

  • ∞Bench En.QA (Zhang et al., 2024). Questions over full novels with about 192K tokens on average. Key entities are replaced so that a model cannot answer from memorized knowledge. The metric is answer F1, as in our paper.

The full-context model truncates the middle of inputs longer than its 262,144-token window, following the LongBench v2 protocol.

Table R3. Zero-shot results on non-Wikipedia sources (Qwen3.5-4B). LongBench v2 reports accuracy (%). New Table 15 in Appendix F.2.

MethodLongBench v2ShortMediumLong∞Bench En.QA (F1)
Full context (middle truncation)41.447.839.534.321.5
MemAgent38.943.937.233.819.0
GRU-Mem39.744.438.135.020.2
ReMemR1†40.645.039.136.121.0
CoEM44.748.343.241.725.6
AuthorsOpenReview ↗

Response to Reviewer xXBR (3/4): W2 & Q2 (continued), W3 & Q3

Findings. CoEM gives the best result on both benchmarks. On LongBench v2 it exceeds the strongest memory baseline by 4.1 accuracy points (95% CI [1.2, 7.0]), and on ∞Bench En.QA by 4.6 F1 points (95% CI [2.0, 7.3]). The gain is largest on the Long subset (5.6 points), where the evidence for a question is spread over the longest inputs and is most likely to be compressed before its relevance is clear. CoEM also exceeds the full-context model by 3.3 points overall and by 7.4 points on the Long subset, where the full-context model must drop the middle of the input. These results show that the memory policy learned on HotpotQA carries over to sources that differ in domain, writing style, and length.

Revision. New Appendix F.2, and a summary sentence at the end of Section 4.2.

W3 & Q3. Comparison with an agentic RAG baseline that can retrieve or reopen source passages

Reviewer xXBR, Weakness 3

The comparisons focus on full-context inference and recurrent-memory agents. A comparison with a strong agentic RAG approach that can retrieve or reopen source passages would help establish when carrying verbatim excerpts in a small pending set is preferable to retrieving evidence when needed. Such a comparison should report answer quality together with retrieval and inference costs, while making the different storage assumptions explicit.

Reviewer xXBR, Question 3

How does COEM compare with an agentic RAG baseline that can retrieve or reopen source passages, when answer quality and total cost are both considered?

Response.

Prior recurrent memory agents, namely MemAgent, ReMemR1, and GRU-Mem, compare with long-context LLMs and other memory agents rather than with RAG, since this line of work studies reading a long input once with a bounded memory. We agree that a direct comparison is useful for showing when each approach fits, and we added one.

Setup. We implemented two methods that can still access the full source after reading and compared them with CoEM on the same questions, with Qwen3.5-4B at 6,400 documents.

  • Agentic RAG. The full document stream is stored and indexed with BM25 plus dense retrieval (BGE-M3). Following Search-R1, the policy issues up to 4 search rounds of 5 documents each and can reopen any document by its ID. It is trained with RL on the same 32,768 HotpotQA questions, answer reward, and 500 updates as the memory agents. We also report a larger budget of 8 rounds with 20 documents each, which raises the maximum number of retrieved documents per question from 20 to 160. On average, the two budgets retrieve 15.0 and 89.8 distinct documents per question, because later search rounds often return documents seen in earlier rounds.

  • ReadAgent (Lee et al., 2024). ReadAgent splits a long input into pages and writes a gist for each page, which is a short summary of that page written by the LLM itself. At answer time it reads all gists, reopens the original pages it selects, and answers from them. We use each 5,000-token chunk as one page.

Table R4. Comparison with methods that keep the full source (Qwen3.5-4B, 6,400 documents). F1 on three datasets, cost on HotpotQA. New Table 16 in Appendix F.3.

MethodKept after readingHotpotQA2WikiMuSiQueCallsInputGeneratedLatency (s)
ReadAgentFull source + gists73.663.052.4130.0807K128.4K779
Agentic RAG (≤4 rounds × 5 docs)Full source + index79.867.557.64.711.2K0.9K11.5
Agentic RAG (≤8 rounds × 20 docs)Full source + index81.068.958.87.964.5K1.6K39.2
CoEM1,024-token state84.273.964.5140.91,018K21.4K548

RAG latency includes index construction (3.1 s) and retrieval.

Storage assumptions. RAG and ReadAgent keep the full source after reading, which is about 614K to 819K tokens at 6,400 documents, plus an index of about 10 MB or about 126K gist tokens. CoEM and the other recurrent memory agents keep only a 1,024-token state and cannot reread earlier chunks.

Comments on the paper

6 comments
AuthorsOpenReview ↗

General Response to All Reviewers and the AC (1/3)

We thank all reviewers for their careful reading and constructive comments. The reviewers find that premature compression is a meaningful problem (xXBR, AyfX), that separating unresolved source material from compact committed facts is useful (ta1s), and that the gains are substantial, consistent, and statistically supported (ta1s, AyfX), and that the paper is well-presented and easy to follow (ta1s). Below we summarize the main concerns, the new experiments, and the revisions. All changes in the revised PDF are marked in red. The main text stays within 10 pages, and all new results are collected in a new Appendix F, so the numbering of existing sections, tables, and figures is unchanged.

Because the discussion period is short, all new experiments use Qwen3.5-4B, and there was not enough time to repeat them with Qwen3.5-9B. Results are means over the three policy seeds (17, 29, 43) unless stated otherwise. The audit of final retained entries in Section 5 covers the existing checkpoints of both backbones.

1. What CoEM adds beyond prior recurrent memory agents (ta1s)

ReMemR1 first noted that early evidence can be pruned before later context shows its importance, and the revision credits this explicitly. CoEM addresses a different loss that memory callbacks cannot fix. A recurrent memory agent decides what to write when it first compresses a chunk. A detail left out at that moment never enters any saved memory, so neither later rewriting nor a callback over saved memories can bring it back. CoEM keeps such evidence verbatim in a pending set within the same context-memory budget and learns when to compress it into a committed fact. To our knowledge, no prior recurrent memory agent treats the timing of compression as a learned decision under a fixed budget.

New analyses support this distinction (Appendix F.8). For native ReMemR1 with its full archive of past memories, 73.6% of the supporting relations missing at answer time never appeared in any archived memory, so a callback had nothing to return. On the 49 evaluation questions with delayed evidence relevance, CoEM leads native ReMemR1 by 12.6 F1 points, compared with 4.8 points on the other questions. As the context grows from 50 to 6,400 documents, native ReMemR1 loses 8.4 to 10.3 F1 points with Qwen3.5-9B, whereas CoEM loses only 1.0 to 2.1 points.

2. Is the gain due to learned commitment timing? (ta1s, AyfX)

The original Fixed-step pending variant drops rejected records, and the eager variants move the pending budget BPB_P to committed memory. We therefore trained matched timing controls that keep BPB_P = 256, the same frozen verifier, the same retention and retry rules, and the same RL training as CoEM. In these controls, a rule sets only the step of the first promotion attempt, and each policy still makes its own later KEEP, retry, and DROP decisions. We also added two verbatim-retention alternatives that keep verbatim excerpts alongside compact facts by fixed rules. Table G1 groups all variants by the conclusion each group supports.

AuthorsOpenReview ↗

General Response to All Reviewers and the AC (2/3)

Table G1. Matched timing controls, verbatim-retention alternatives, and component ablations (revised Table 2 and Table 18).

Variant (Qwen3.5-4B, answer F1)HotpotQA 8002Wiki 800HotpotQA 6,400Pending tokens
Component ablations (Table 2)
Eager + verifier77.266.070.60
Fixed-step pending (original)82.372.278.6188
Verbatim-retention alternatives
ReMemR1† + verbatim buffer82.171.976.0251
Extractive cache81.871.676.9247
Matched timing controls
Fixed-step + matched retry82.872.879.2203
Tuned delay + matched retry83.073.179.6221
Entity-triggered wait + matched retry83.273.480.1196
CoEM85.576.184.2138

Pending tokens is the mean occupancy of verbatim text at 6,400 documents on HotpotQA. Matched timing controls share the budget split, verifier, retention and retry rules, and RL training of CoEM, and a rule sets only their first promotion attempt. The verbatim-retention alternatives keep up to 256 tokens of verbatim text by fixed rules, and ReMemR1† + verbatim buffer keeps the unverified memory updates of ReMemR1†. The component ablations come from Table 2, and Eager + verifier has no pending set.

Matching the retry rules explains only 0.5 / 0.6 points of the original 3.2 / 3.9-point gap (HotpotQA / 2WikiMultiHopQA). Under identical retention and retry rules, learned timing beats the best tuned rule by 2.3 / 2.7 points at 800 documents and by 4.1 points at 6,400 documents. It also beats both verbatim-retention alternatives by at least 3.4 / 4.2 points while using the fewest pending tokens. Diagnostics explain why. Fixed delays compress most supporting excerpts long before the linking passage arrives, and similarity-based caching evicts delayed evidence because it looks unrelated to the question on arrival. CoEM keeps 89.8% of supporting excerpts until the linking passage arrives and commits 81.3% of them at that step or the next one (Appendix F.5).

3. Cost, other sources, retrieval, and memory budgets (xXBR)

  • Cost. We report policy calls, input tokens, and generated tokens for every method at every timed length (Appendix F.1). At 6,400 documents, CoEM reads 28% more input tokens than MemAgent but generates only about one fifth as many tokens, because it outputs compact actions while the controller copies source text by offsets. Since decoding dominates latency, CoEM is 24% to 42% faster than MemAgent and both ReMemR1 variants.

  • Non-Wikipedia sources. Without additional training, CoEM exceeds the strongest memory baseline by 4.1 accuracy points on LongBench v2 and by 4.6 F1 points on ∞Bench En.QA (Appendix F.2).

  • Agentic RAG and ReadAgent. With the storage assumptions stated explicitly, CoEM is 4.4 to 6.9 F1 points more accurate than an RL-trained agentic RAG baseline. Raising its maximum number of retrieved documents eightfold, from 20 to 160, increases its latency about 3.4 times but closes only 1.2 to 1.4 points, because bridge evidence is hard to retrieve before the linking passage is read. RAG has a much lower online cost when the full source can be stored and indexed, while CoEM targets single-pass reading where the source cannot be archived (Appendix F.3).

  • Total memory budget. With all memory agents retrained at 512, 1,024, and 2,048 tokens, CoEM leads at every budget, and CoEM with 512 tokens beats every baseline with 2,048 tokens (Appendix F.4).

4. Capacity competition and evaluation scale (AyfX)

We added answer F1 and the loss rate of critical evidence to the pending-set stress tests, and evaluated more supporting passages (two-, three-, and four-hop MuSiQue) and several questions sharing one document stream. Every method loses F1 as more evidence competes for memory, but the lead of CoEM grows, from 8.8 to 12.7 points with four HotpotQA questions per stream and to 10.9 points on four-hop MuSiQue. Its critical-evidence loss stays at 40% to 57% of that of the strongest baseline. With 512 questions per dataset, the leads agree with the 128-question results within 0.2 points, and all 95% intervals exclude zero (Appendices F.6 and F.9).

AuthorsOpenReview ↗

General Response to All Reviewers and the AC (3/3)

5. Verifier audit and faithfulness after policy optimization (ta1s)

The 2,000 audited attempts come from the selected Qwen3.5-4B CoEM checkpoints of seeds 17, 29, and 43, on HotpotQA and 2WikiMultiHopQA development questions disjoint from evaluation (1,000 each), half at 800 and half at 6,400 documents. A new blinded audit of 1,200 final retained entries at 6,400 documents gives 97.5% (4B) and 97.9% (9B) faithfulness, and 94.6% and 95.4% for multi-source claims (Appendix F.7).

Summary of the new comparisons

Table G2. Summary of the new comparisons. All settings use 6,400 documents except LongBench v2 and ∞Bench.

Setting (Qwen3.5-4B)MetricStrongest alternativeCoEMLead
LongBench v2, non-Wikipedia sourcesAccuracy40.6 (ReMemR1†)44.7+4.1
∞Bench En.QA, novelsF121.0 (ReMemR1†)25.6+4.6
Agentic RAG, larger budget, HotpotQAF181.0 (RAG)84.2+3.2
Best tuned waiting rule, HotpotQA 6,400F180.1 (entity trigger)84.2+4.1
Total budget 512 tokens, HotpotQAF169.1 (GRU-Mem)80.1+11.0
Total budget 2,048 tokens, HotpotQAF178.9 (GRU-Mem)85.9+7.0
Four questions per stream, HotpotQAF157.6 (GRU-Mem)70.3+12.7
Four-hop MuSiQueF140.3 (ReMemR1†)51.2+10.9

Revisions to the paper (all marked in red)

LocationChangeReviewers
Section 1Credits ReMemR1 for premature pruning. Restates the first contribution around details omitted at first compression and learned commitment timing. Summarizes the new experiments.ta1s
Section 2States the storage assumptions and use cases of retrieval and CoEM, and the advantage of CoEM on bridge evidence. Credits ReMemR1 and contrasts its reward with the evidence reward of CoEM.xXBR, ta1s
Sections 4.1 and 4.2Points to the new experiments and summarizes the non-Wikipedia results.xXBR
Section 4.3Adds answer F1 under capacity competition, the total-budget sweep, and the token and call breakdown.xXBR, AyfX
Section 4.4, Table 2Adds matched timing controls and verbatim-retention alternatives in separate blocks, and qualifies the claim on commitment timing.ta1s, AyfX
Section 5States the use cases relative to retrieval and eviction under many concurrent chains.xXBR, AyfX
Appendices A.1, A.2, A.4, A.5Defines the matched controls, gives the provenance of the 2,000 audited attempts, and states how the number of chunks varies by question.xXBR, ta1s, AyfX
New Appendix F.1 to F.9All new experiments: cost (F.1), non-Wikipedia sources (F.2), retrieval (F.3), budgets (F.4), matched timing (F.5), capacity (F.6), faithfulness (F.7), omitted evidence (F.8), and larger cohorts (F.9).All

We hope these results address the main concerns. We are happy to continue the discussion and to provide further details on any of these results.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.