acceptodds
Under review as a conference paper at ICLR 2027

DeferMem: Query-Time Evidence Distillation via Reinforcement Learning for Long-Term Agent Memory

Abstract

Large language model (LLM) agents still struggle to effectively use long-term memory, with answer-supporting evidence often scattered across long conversational histories and buried in substantial irrelevant content. Existing memory systems commonly process memory into dedicated units before future queries are known and retrieve these preconstructed units at query time. Because the contents and granularity of these units are determined before the current query is known, the retrieved units can remain coarse and noisy for that query, leaving downstream answerers to further denoise them and uncover query-specific evidence. We present DeferMem, a long-term memory framework that decouples this problem into high-recall candidate retrieval and query-conditioned evidence distillation. A memory distiller then distills these high-recall but highly noisy candidates into a set of faithful, self-contained, and query-conditioned evidence. To train this distiller, we introduce DistillPO, a reinforcement learning algorithm that formulates post-retrieval evidence distillation as a structured action comprising message selection and evidence rewriting. It optimizes this action with a decomposed-and-gated reward pipeline and structure-aligned advantage assignment, gating reward components from validity to quality checks while exposing answerability-related feedback early and assigning each reward to its responsible output span. On LoCoMo and LongMemEval-S, DeferMem surpasses strong baselines in QA accuracy and memory-system efficiency, achieving the highest QA accuracy and fastest runtime while consuming no commercial-API tokens for memory operations.

Primary area:
foundation or frontier models, including LLMs
Keywords:
long-term memory, agent memory, large language models, reinforcement learning

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_sTMJOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

DeferMem addresses question answering over long conversational histories. Instead of directly passing retrieved messages to an answerer, it first retrieves a broad set of candidates and then uses a separate model to select relevant messages and rewrite them into compact, query-specific evidence. The proposed DistillPO algorithm trains this distiller using rewards for valid selection, evidence quality, and downstream answerability, with different rewards assigned to selection and rewriting. Experiments on LoCoMo and LongMemEval report improved answer accuracy and memory-system runtime, accompanied by component ablations and cross-source training evaluations.

Critical strengths
  1. The framework addresses a practical mismatch between retrieval and answering. A retrieved message may contain relevant evidence together with substantial unrelated information. DeferMem explicitly separates finding potentially useful messages from constructing the evidence needed for the current question. This provides a clear interface between memory retrieval and downstream answering and allows the distiller to be reused with different answerers.

  2. The ablations provide useful evidence about the system’s components. The paper separately examines the retrieval structure, distiller training, reward gating, advantage assignment, and teacher anchors. It also reports LoCoMo results excluding the training conversation and evaluates cross-source training. These controls help distinguish improvements from individual design choices and assess dependence on the training source.

Critical weaknesses
  1. The contribution beyond existing learned compression methods is unclear. DeferMem’s central pipeline retrieves candidate text, compresses it according to the current question, and passes the result to a separate answerer. RECOMP [1] and ECoRAG [3] already study related post-retrieval compression, while CORE [2] trains a query-conditioned compressor using downstream QA rewards with a frozen answerer. These works are not discussed. Consequently, the current comparisons establish improvements over memory systems more clearly than they establish the additional value of DistillPO over an existing learned compressor. Table 3 shows why this comparison matters. Passing segment-link candidates directly to the answerer already gives 84.11 on LoCoMo, close to the strongest baseline (EverMemOS, 85.79), and a prompted GPT-4o-mini distiller with no training reaches 86.35, above every baseline. Most of the margin over memory systems therefore comes from the retrieve-then-compress pipeline that these compression methods already use, and the value of DistillPO itself can only be judged against a learned compressor.

  2. Message selection is trained to match annotated evidence, without direct answerability feedback. The distiller first outputs selected message IDs and then rewrites their contents. DistillPO scores the selection against annotated supporting messages, while downstream answerability rewards are assigned to the rewriting span. However, a failed answer may result from selecting the wrong messages rather than rewriting them poorly. Removing the selection rewards reduces LongMemEval-S accuracy from 70.0 to 8.2, showing how strongly the current training design depends on this supervision. Its behavior with incomplete annotations or alternative valid evidence therefore remains uncertain.

  3. Correct answers do not establish that the distilled evidence is faithful. The distiller generates new text rather than only copying source passages, so it can potentially introduce unsupported details. The main evaluation measures final QA accuracy, and the fidelity reward is applied during training. Neither directly establishes the factual support of held-out evidence statements. A separate evaluation is needed to distinguish unsupported rewriting from omission of necessary information.

  4. The efficiency comparison does not fully account for differences in execution resources. DeferMem performs distillation with a local 8B model, whereas several baselines use commercial APIs for memory operations. The reported zero API-token usage therefore describes how computation is provided, not its total cost. Hardware, batching, and API latency can also affect the runtime ranking. More explicit resource accounting is needed to establish a reproducible efficiency advantage. Table 1 itself suggests the runtimes are not comparable: NaiveRAG makes no API calls and only performs embedding retrieval, yet it takes 867.38 s on LongMemEval-S and 173.87 s on LoCoMo, far longer than DeferMem (94.77 s and 65.43 s), which performs the same kind of retrieval plus segmentation and 8B-model distillation.

  5. The cross-source gains are too small to interpret without uncertainty estimates. The paper evaluates distillers trained on a different data source to test transfer beyond the training benchmark. These variants exceed the strongest reported baselines by only 0.20 points on LongMemEval-S and 0.28 points on held-out LoCoMo conversations. Although the full-training results are stronger, these small transfer margins do not yet establish reliable cross-source superiority.

  6. The training signal includes distillation from a much stronger teacher that sees the answer. Appendix C.2 states that every training group contains one anchor completion written by GPT-5.2 from the query, the gold answer and the gold supporting messages, and the main text describes it only as "a strong teacher". This mixes teacher imitation into the RL objective, and the paper does not separate how much of DistillPO's gain comes from this anchor versus its reward design. The GPT-5.2 calls and the LLM-judged rewards used in training are also commercial-API costs that the claim of "no commercial-API tokens" leaves out.

[1] RECOMP: Improving Retrieval-Augmented LMs with Context Compression and Selective Augmentation. 2024.

[2] CORE: Lossless Compression for Retrieval-Augmented LLMs via Reinforcement Learning. 2025.

[3] ECoRAG: Evidentiality-guided Compression for Long Context RAG. 2025.

Questions for authors

Please address the weaknesses above.

4 comments
AuthorsOpenReview ↗

Response to Reviewer sTMJ [1/4]

Thank you for your detailed review. We have conducted additional comparisons with learned compression methods, trained output-formulation ablations, and an independent assessment of distilled-evidence quality. We also provide further analyses of selection supervision, full-reward anchors, memory-system efficiency, and training-time resource usage. We address your concerns below, with the corresponding manuscript revisions highlighted in blue.

sTMJ W1. DistillPO's Contribution Beyond Learned Post-Retrieval Compression

Positioning relative to learned compression.

RECOMP and ECoRAG investigate retrieval-conditioned compression, while CORE trains a query-conditioned compressor using downstream QA rewards. We have expanded Sec.2 (Related Work) to discuss these methods and clarify DeferMem's two complementary contributions.

First, DistillPO is a reinforcement learning algorithm designed for structured query-time evidence distillation. It jointly optimizes source-message selection and source-aligned evidence rewriting through decomposed rewards, leaky hierarchical gating, and structure-aligned advantages. Second, DeferMem decouples lightweight, high-recall memory retrieval from query-conditioned evidence formation, allowing question-specific evidence to be constructed at query time rather than fixed during memory construction.

Gains beyond retrieval and prompted distillation.

The original Table 3 already provides relevant comparisons. On LoCoMo w/o conv-26, directly passing Segment-Link candidates to the answerer achieves 84.11%, below EverMemOS (85.79%). A prompted GPT-4o-mini distiller achieves 86.35%, only 0.56 points above EverMemOS, whereas the DistillPO-trained 8B distiller achieves 89.93%, improving by 3.58 points over prompted distillation and 4.14 points over EverMemOS. On LongMemEval-S, the prompted GPT-4o-mini distiller achieves 63.80%, compared with 70.00% for DistillPO.

The original Table 3(a) also includes an SFT baseline trained on verified full-reward anchor completions (80.97/63.40) and a DAPO baseline (84.89/60.60), compared with 89.93/70.00 for DistillPO on LoCoMo/LongMemEval-S. These results demonstrate the effectiveness of DistillPO's training design beyond supervised learning on verified evidence-distillation demonstrations and standard sequence-level policy optimization.

Output-formulation ablations.

We further trained two memory-distiller variants with different output formulations: one outputs only selected useful source messages, while the other outputs only rewritten evidence. Both variants were evaluated using exactly the same Segment-Link retrieved candidates and downstream answerer/judge protocol as DeferMem.

The selection-only variant achieves 85.28/66.80, and the evidence-only variant achieves 86.35/67.00, compared with 89.93/70.00 for the joint selection–rewriting formulation on LoCoMo/LongMemEval-S. These results demonstrate the benefit of explicitly coupling source-message selection with query-conditioned evidence rewriting.

Comparisons with reranking and learned compression.

We additionally evaluate the BGE cross-encoder reranker and extractive/abstractive RECOMP compressors, using as input exactly the same Segment-Link–retrieved candidates provided to DeferMem's memory distiller. For RECOMP, we use public checkpoints trained on Natural Questions (NQ) and HotpotQA. All evaluated methods use the same GPT-4o-mini downstream answerer and dataset-specific judge. We evaluate two token budgets, 1,024 and 2,048, for the memory information passed to the answerer.

The strongest RECOMP results across these settings are 73.14% on LoCoMo and 63.60% on LongMemEval-S, while BGE reranking reaches 74.03% and 64.60%, respectively. Both DeferMem (89.93/70.00) and DeferMem† (86.07/67.20), whose distillers use a maximum completion length of 1,024 tokens, outperform all these alternatives. DeferMem† is trained exclusively on the other benchmark's QA data, further demonstrating the effectiveness of the learned distillation policy across data sources.

Manuscript revisions.

We expanded the related-work discussion in Sec.2, added the output-formulation ablations to Table 3(a), and included representative BGE/RECOMP comparisons in Table 3(b). The complete reranking/compression comparisons under both token budgets are reported in Appendix D.3.

AuthorsOpenReview ↗

Response to Reviewer sTMJ [2/4]

sTMJ W2. Selection Supervision and Answerability Credit

Selection supervision and answerability.

In DistillPO, the selection span receives answerability-related supervision through the gold-support coverage reward r2r_2, together with the conciseness reward r3r_3. By contrast, r1r_1 checks the validity of selected message IDs, while r4r_4 evaluates the alignment between selected messages and rewritten evidence. Satisfying these validity and alignment requirements (r1r_1, r4r_4) does not necessarily mean that the selected messages contain the information needed to answer the query.

This helps explain the substantial accuracy reductions observed when removing r2,r3r_2,r_3 in the original Table 3(a): 42.92/61.80 points on LoCoMo w/o conv-26 / LongMemEval-S. The results highlight the importance of providing selection-specific rewards that directly encourage the identification of answer-supporting messages, beyond ensuring structural validity and selection–evidence alignment.

Reward-assignment ablation.

Although final-answer correctness depends on both selection and rewriting, an incorrect answer does not necessarily reveal which component is responsible. To examine whether assigning downstream answerability rewards to selection would improve training, we further trained a variant that also assigns r6,r7r_6,r_7 to the selection span. This reduces accuracy from 89.93/70.00 to 87.69/67.60 on LoCoMo w/o conv-26 / LongMemEval-S, a decrease of 2.24/2.40 points, providing further empirical support for DistillPO's span-specific reward assignment.

The training-process analysis also reveals distinct optimization signals for selection and rewriting. Among 19,841 on-policy completions containing both spans, their advantages have opposite signs in 3,095 cases (15.60%), while a unified sequence-level advantage conflicts with at least one span-specific advantage in 2,966 cases (14.95%). These observations further support the importance of distinguishing the optimization signals for the two components.

Supporting-evidence annotations.

Answer-supporting evidence annotations used by r2,r3r_2,r_3 are widely available in long-term conversational-memory QA benchmarks. For example, LoCoMo and LongMemEval provide annotations identifying supporting dialogue messages, while PersonaMem-v2 provides answer-supporting conversation snippets that can be mapped back to the original messages. In our experiments, DistillPO leverages existing supporting-evidence annotations without requiring additional manual evidence labeling. At query time, the distiller selects useful source messages without access to gold supporting-message annotations.

Manuscript revisions.

We added the r6,r7r_6,r_7 selection-span assignment variant to Table 3(a). The corresponding training-process analysis is reported in Appendix D.7.

sTMJ W3. Independent Assessment of Evidence Faithfulness and Answer-Critical Omissions

Independent evidence-quality audit.

We used Codex (GPT-6-Astra configuration) to independently assess DeferMem's distilled evidence on evaluation questions. The assessment examined whether individual rewritten evidence entries were faithful to their source messages and self-contained, as well as whether the complete evidence set was sufficient to answer the query or omitted answer-critical information. In particular, the faithfulness assessment identifies unsupported rewriting, while the sufficiency and omission assessments examine whether necessary information is preserved in the distilled evidence.

We evaluated 70 sampled questions per benchmark, including both correct and incorrect QA outcomes.

Rebuttal Table H. Independent model-based audit of distilled evidence (70 sampled QA per benchmark).

BenchmarkFaithful entriesSelf-contained entriesSufficient evidence setsAnswer-critical omissions
LoCoMo w/o conv-26139/158 (87.97%)143/158 (90.51%)58/70 (82.86%)10/70 (14.29%)
LongMemEval-S171/179 (95.53%)173/179 (96.65%)60/70 (85.71%)8/70 (11.43%)

These results provide separate assessments of the factual grounding of rewritten evidence and the sufficiency of the complete distilled-evidence sets for answering the queries, with explicit checks for unsupported rewriting and answer-critical omissions. Further details are provided in Appendix D.4.

AuthorsOpenReview ↗

Response to Reviewer sTMJ [3/4]

sTMJ W4. Memory-System Efficiency

Memory-operation efficiency metrics.

As defined in Sec.4.1, our reported runtime measures the end-to-end memory-system processing time, from history construction through query-time memory utilization, until information is returned to the downstream answerer. For DeferMem, this includes Segment-Link construction and retrieval, as well as the full local 8B evidence-distillation inference, including prompt tokenization, GPU transfer, prefill, generation, and output post-processing. Final answer generation and judging are excluded from the memory-operation metrics.

We report commercial LLM API token usage as a separate efficiency metric, capturing the reliance on paid API services during memory operations and the associated token-based expenditure. Thus, DeferMem's zero commercial-API token usage means that memory construction and query-time evidence distillation require no commercial LLM API calls, while their local computation is fully included in the reported runtime.

Execution setup and efficiency breakdown.

The runtime comparisons in Table 1 use the same execution resources (queries were processed sequentially with batch size one, four NVIDIA RTX 3090 GPUs) and a consistent memory-operation measurement scope across methods, covering both memory construction and query-time memory utilization. To further clarify the sources of runtime differences, we break down the reported memory-system efficiency metrics into write-time memory construction and query-time memory utilization.

Rebuttal Table I. Breakdown of memory-system efficiency metrics (write per history; query per QA).

BenchmarkMethodWrite (s)Query (s)Write API tokens (k)Query API tokens (k)
LongMemEval-SDeferMem79.2915.4800
LongMemEval-SLightMem279.160.00431.850
LoCoMoDeferMem54.8710.5600
LoCoMoEverMemOS325.0225.37312.173.63

On LongMemEval-S, DeferMem achieves substantially lower memory-construction time than LightMem (79.29s vs. 279.16s). On LoCoMo, DeferMem achieves lower runtime than EverMemOS in both memory construction (54.87s vs. 325.02s) and query-time utilization (10.56s vs. 25.37s). These results show how the different processing stages contribute to the end-to-end runtime differences reported in Table 1, and DeferMem requires zero commercial LLM API tokens for both memory construction and query-time utilization.

NaiveRAG runtime provenance.

The NaiveRAG runtime results in Table 1 are based on the NaiveRAG baseline results reported in LightMem's main experiments, which use the same execution resources, backbone LLM, and judge LLM as our evaluation. We retained the available results and supplemented the missing memory-construction measurements and LoCoMo's fifth (adversarial) QA category under the same experimental settings.

Manuscript revisions.

The evaluation setup is detailed in Appendix B. The write- and query-time breakdown of memory-system efficiency metrics, cumulative runtime analysis across query counts, and training-time API usage and GPU-hours are reported in Appendix D.6.

sTMJ W5. Cross-Source Transfer of Learned Distillation

The cross-source variants in the original Table 1 evaluate whether a distiller trained exclusively on one benchmark's QA data can transfer its learned evidence-distillation behavior to another benchmark. They achieve 67.20% on LongMemEval-S and 86.07% on LoCoMo, closely matching the strongest memory-system baselines. The main evidence for cross-source transfer comes from the improvements over the untrained distiller, rather than the small numerical margins over these baselines.

As shown in the original Table 3(a), the untrained 8B distiller achieves 52.20% on LongMemEval-S and 72.02% on LoCoMo under the same evaluation setting. Training exclusively on the other benchmark's QA data increases accuracy to 67.20% (+15.00 points) and 86.07% (+14.05 points), respectively. These substantial improvements provide direct evidence of cross-source transfer of the learned distillation policy, while the fully trained DeferMem achieves the strongest overall results.

AuthorsOpenReview ↗

Response to Reviewer sTMJ [4/4]

sTMJ W6. Teacher-Anchor Contributions and Training-Time API Usage

Anchor contribution and original ablations.

As shown in the original Table 3(a), DistillPO without full-reward anchors achieves 86.60/66.80 on LoCoMo w/o conv-26 / LongMemEval-S, compared with 89.93/70.00 with anchors, quantifying their contribution as 3.33/3.20 points. Importantly, even without teacher-generated anchors, DistillPO outperforms DAPO (84.89/60.60) and SFT trained on verified anchor completions (80.97/63.40). These comparisons demonstrate the effectiveness of DistillPO's structured optimization beyond teacher-derived demonstrations, with additional improvements from full-reward anchors.

How anchors contribute to optimization.

The verified full-reward anchors are generated by GPT-5.2 using the training query, gold answer, and gold supporting messages, and are reused throughout training. Each anchor participates in both group-relative advantage normalization and the clipped policy optimization objective, allowing its selection and rewriting tokens to contribute directly to parameter updates. Our analysis of 4,962 recorded training groups shows that including the anchor changes 66.38%/73.35% of selection/rewriting advantages and restores nonzero on-policy advantages in 11.83%/10.00% of selection/rewriting groups. These observations further characterize the anchor's contribution to DistillPO's training process.

Training resource accounting and deployment API usage.

As defined in Sec.4.1, the commercial-API token metric in Table 1 measures API usage during deployment-stage memory operations, spanning write-time memory management and query-time memory utilization until information is returned to the downstream answerer.

We additionally quantify the resources used for training: preparing 329 verified GPT-5.2 full-reward anchors involved 1,160 recorded generation and validation calls, consuming approximately 2.80M input and 0.078M output API tokens. During DistillPO training, LLM-based reward evaluation was invoked 27,134 times, including 3,369 cache hits (12.42%) that did not require external API requests. The recorded API usage for reward evaluation during training totaled approximately 69.60M input and 2.71M output tokens. GPU resource usage for DistillPO training was approximately 167.95 A100 GPU-hours.

These resources are incurred during the training of the memory distiller. Once trained, the same distiller can be reused throughout deployment, with no commercial LLM API calls required for either write-time memory management or query-time evidence distillation.

Manuscript revisions.

We clarified the anchor's participation in group normalization and clipped policy optimization in Appendix C.2, added the training-process analyses in Appendix D.7, and reported training-time API usage and GPU-hours in Appendix D.6.

Reviewer_W9w3OpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

DeferMem keeps raw conversational history in a segment-link structure, retrieves a broad candidate set, and uses a trained 8B distiller to produce query-conditioned evidence. DistillPO models message selection and evidence rewriting as one structured action, combining decomposed rewards, leaky hierarchical gates, span-specific advantages, and teacher anchors. The paper reports 70.00% on LongMemEval-S and 89.93% on LoCoMo (after excluding the training conversation), together with lower deployment-time memory-operation runtime and zero commercial-API tokens within that scope.

Critical strengths

S1. A practical, well-ablated retrieval/distillation decomposition. Bypassing the distiller lowers accuracy by 5.82 points on LoCoMo and 8.00 on LongMemEval-S, so the distillation stage clearly contributes. The paper also ablates SFT on the same verified anchor targets, DAPO, prompted GPT-4o-mini distillation, gating, span assignment, anchors and individual reward components, which makes the training design inspectable rather than asserted.

S2. Careful evaluation hygiene and transparency. The paper excludes conv-26, separates LongMemEval training histories from evaluation, and provides cross-training-source and cross-answerer checks. It also reports the million-token experiment, category-level weaknesses (DeferMem is not best on multi-hop or open-domain LoCoMo questions), stage-level errors, and the training-time API requirement.

Critical weaknesses

W1. Closest-method positioning is incomplete, and the mechanistic explanations exceed the process evidence. PRCA already trains a post-retrieval adapter with downstream answer rewards, and RECOMP learns textual compression of retrieved content, so the defensible novelty is the structured selection/rewriting formulation and training design for conversational memory. No matched comparison against these methods (same candidates, backbone, supervision, output budget) is given, so the ablations show effectiveness but not the incremental algorithmic contribution. Likewise, the gating, span-assignment and anchor ablations report only final accuracy; they do not show earlier answerability activation, better credit assignment or more stable optimization. Appendix C.2 acknowledges off-policy anchors, but the current/old likelihood ratio is not a standard correction for the teacher sampling distribution and the mixed clipped objective needs a precise interpretation. Normalization-only anchors, a separate supervised anchor term, matched DAPO/full-reward budgets and multi-seed training diagnostics are missing.

W2. Statistical robustness, evidence quality and efficiency accounting are not established. The three-point LongMemEval-S margin is 15 net correct answers among 500 questions, and LoCoMo questions are correlated within conversations; paired predictions, conversation-level bootstrap intervals and multiple training seeds are needed. The permissive same-topic criterion in the LoCoMo judge prompt is shared across methods and is not evidence of favoritism, but it motivates an independent audit of disputed predictions and a direct measurement of evidence faithfulness, omissions and self-containedness; training rewards and two successful examples are not an independent quality evaluation. On efficiency, excluding answer generation and judging from the memory-operation metrics is reasonable, but zero deployment API tokens does not remove local 8B inference, embedding and compression costs, and construction vs per-query cost, teacher/reward calls and amortization over query counts are not reported. The rise from 94.77s to 804.19s on LongMemEval-M is useful scalability evidence but does not by itself establish acceptable latency or superiority over matched long-history baselines.

Questions for authors
  1. How do PRCA-style outcome RL and RECOMP-style learned compression perform with the same 8B backbone, candidates, supervision and output budget?
  2. Are DAPO and DistillPO matched in rewards, anchor information, rollouts and updates? How much of the anchor benefit comes from normalization versus imitation?
  3. Do the gains survive paired statistics, conversation clustering, multiple seeds and an independent blind audit of the evidence?
  4. Over what number of subsequent queries does the cost advantage persist after separating construction, inference, training and local compute?
  5. How are r2 and r5 in Table 10 defined when gold support or evidence is empty?
4 comments
AuthorsOpenReview ↗

Response to Reviewer W9w3 [1/4]

Thank you for the detailed review. We have added post-retrieval baselines, an independent model-based audit of held-out evidence, paired statistical analyses, deployment/training cost accounting, and training-process analyses based on the actual DistillPO training trajectory. We also refined the claims about closest methods and off-policy anchors. Below we address the five questions together with the corresponding weaknesses.

W9w3 W1 & Q1. Comparison with Related Methods and DistillPO's Algorithmic Contribution

Positioning relative to prior methods.

PRCA trains a post-retrieval adapter using downstream answer rewards, while RECOMP learns to compress retrieved information. DeferMem makes two complementary contributions. First, DistillPO is a reinforcement learning algorithm for structured query-time evidence distillation, jointly optimizing source-message selection and source-aligned evidence rewriting through decomposed rewards, leaky hierarchical gating, and structure-aligned advantages. Second, DeferMem decouples lightweight, high-recall memory retrieval from query-conditioned evidence formation, allowing question-specific evidence to be constructed at query time rather than fixed during memory construction. We have expanded Sec.2 (Related Work) to clarify the relationship to PRCA and RECOMP.

Output-formulation ablations.

We further trained two memory-distiller variants with different output formulations: one outputs only selected useful source messages, while the other outputs only rewritten evidence. Both variants were evaluated using exactly the same Segment-Link retrieved candidates and downstream answerer/judge protocol as DeferMem.

Rebuttal Table C. Output-formulation ablations (QA accuracy, %).

Distiller outputLoCoMo w/o conv-26Δ (pp)LongMemEval-SΔ (pp)
Selected useful messages only85.28−4.6566.80−3.20
Rewritten evidence only86.35−3.5867.00−3.00
Joint selection and rewriting (DeferMem)89.93—70.00—

The joint selection–rewriting formulation consistently outperforms both alternatives, demonstrating the benefit of explicitly coupling source-message selection with query-conditioned evidence rewriting.

The original Table 3(a) also includes supervised training on the verified full-reward anchor completions and a DAPO baseline using the same 8B initialization. SFT achieves 80.97/63.40, and DAPO achieves 84.89/60.60, compared with 89.93/70.00 for DistillPO on LoCoMo w/o conv-26 / LongMemEval-S. These results further demonstrate the effectiveness of DistillPO's training design beyond supervised learning on verified evidence-distillation demonstrations and standard sequence-level policy optimization.

Comparisons with reranking and learned compression.

We additionally evaluate the BGE cross-encoder reranker [1] and extractive/abstractive RECOMP compressors [2], using as input exactly the same Segment-Link–retrieved candidates provided to DeferMem's memory distiller. For RECOMP, we use public checkpoints trained on Natural Questions (NQ) and HotpotQA. We could not identify a publicly available PRCA implementation or trained checkpoint for direct evaluation. All evaluated methods use the same GPT-4o-mini downstream answerer and dataset-specific judge. We evaluate two token budgets, 1,024 and 2,048, for the memory information passed to the answerer.

Rebuttal Table D. QA accuracy (%) of reranking and compression methods under two output budgets.

Reranking/compression methodLoCoMo w/o conv-26: 1,024LoCoMo w/o conv-26: 2,048LME-S: 1,024LME-S: 2,048
BGE reranker (v2-m3)73.9274.0362.6064.60
RECOMP extractive, NQ73.0372.3053.2056.40
RECOMP extractive, HotpotQA73.1472.6961.4063.60
RECOMP abstractive, NQ57.1957.1319.0019.00
RECOMP abstractive, HotpotQA50.5350.6439.6040.20

Both DeferMem and DeferMem† use a maximum distiller completion length of 1,024 tokens. Under its original structured-output setting, DeferMem achieves 89.93% on LoCoMo and 70.00% on LongMemEval-S, while DeferMem†, trained exclusively on the other benchmark's QA data, achieves 86.07% and 67.20%, respectively. Both outperform all evaluated reranking and compression alternatives, whose best accuracies are 74.03% and 64.60%, respectively.

Manuscript revisions.

We expanded the discussion of PRCA and RECOMP in Sec.2, added the output-formulation ablations to Table 3(a), and included representative BGE/RECOMP results in Table 3(b). The complete reranking/compression comparisons and implementation details are reported in Appendix D.3.

AuthorsOpenReview ↗

Response to Reviewer W9w3 [2/4]

W9w3 W1 & Q2. Reward Gating, Structure-Aligned Credit Assignment, and Anchor Optimization

Early answerability feedback.

Under leaky hierarchical gating, the answerability-related rewards are computed as soon as a completion is format-valid. According to the training records, these answerability-related rewards were computed for 91.3%, 97.7%, and 98.3% of on-policy completions during early, middle, and late training, respectively. By contrast, the quality-related faithfulness reward r5r_5 was computed for 76.0%, 95.8%, and 97.9%, while the conciseness reward r3r_3 was computed for 29.1%, 55.0%, and 75.5%, respectively. This demonstrates how the gating design makes answerability-related feedback available early in training. The corresponding ablations in Table 3(a) further show that replacing leaky gating with strict gating or no gating reduces QA accuracy.

Structure-aligned credit assignment.

To examine the optimization signals for selection and rewriting, we analyzed the actual DistillPO training trajectory. Among 19,841 on-policy completions containing both selection and rewriting spans, the two span-level advantages have opposite signs in 3,095 cases (15.60%). A unified sequence-level advantage conflicts with at least one of the two span-specific advantages in 2,966 cases (14.95%). These observations provide direct evidence of the distinct optimization signals addressed by structure-aligned advantage assignment. Additionally, assigning r6,r7r_6,r_7 to the selection span as well reduces accuracy by 2.24/2.40 points on LoCoMo/LongMemEval-S, providing further empirical support for DistillPO's span-specific reward assignment (Table 3(a)).

Full-reward anchor: normalization and direct optimization.

We analyzed 4,962 recorded training groups, each containing four on-policy rollouts and one verified full-reward anchor. Comparing group-relative advantages for the same on-policy rollouts with and without the anchor in normalization, we found that including the anchor changes 66.38%/73.35% of selection/rewriting advantages. It also turns all-zero on-policy advantage groups into groups with nonzero signals in 587/4,962 (11.83%) of selection groups and 496/4,962 (10.00%) of rewriting groups. The verified full-reward anchor also participates directly in the clipped policy loss, allowing its selection and rewriting tokens to contribute to the distiller's parameter updates. Removing the anchor reduces QA accuracy by 3.33/3.20 points on LoCoMo w/o conv-26/LongMemEval-S (Table 3(a)). The no-anchor ablation demonstrates the anchor's overall contribution to QA accuracy, while the training-process analysis quantifies the anchor's effect on group-relative normalization.

Comparison with DAPO.

The original Table 3(a) already includes both the DAPO baseline and a DistillPO variant without full-reward anchors. DAPO and DistillPO use the same 8B initialization and underlying reward signals, with matched numbers of on-policy rollouts and optimizer updates. However, DAPO combines the reward signals into a single sequence-level reward and applies a unified advantage to the entire completion, whereas DistillPO employs leaky hierarchical gating and structure-aligned advantages for selection and rewriting. Importantly, even without full-reward anchors, DistillPO achieves 86.60/66.80 on LoCoMo w/o conv-26 / LongMemEval-S, outperforming DAPO (84.89/60.60) by 1.71/6.20 points. With full-reward anchors, DistillPO further improves to 89.93/70.00. These results demonstrate the effectiveness of DistillPO's structured optimization design beyond standard sequence-level policy optimization, with additional gains from full-reward anchors.

Off-policy anchor objective.

Inspired by DaGRPO's off-policy data augmentation, DistillPO incorporates a verified teacher-generated anchor into each group of on-policy completions. Span-level advantages are computed over the combined group, and both on-policy and anchor tokens participate in the same clipped policy optimization objective. For anchor tokens, the current/old-policy likelihood ratio measures changes in their probabilities relative to the previous distiller policy, with clipping regulating their contribution to the optimization. Since these tokens are teacher-generated, the ratio is not an importance-sampling correction for the teacher's sampling distribution. We have clarified this objective in Appendix C.2, with the training-process analyses reported in Appendix D.7.

AuthorsOpenReview ↗

Response to Reviewer W9w3 [3/4]

W9w3 W2 & Q3. Paired Statistics and Independent Evidence-Quality Assessment

Paired statistical analysis.

We conducted paired statistical analyses using the per-question correctness results of DeferMem and the strongest non-DeferMem baseline on each benchmark.

Rebuttal Table E. Paired QA accuracy analysis.

Benchmark / baselineNDeferMem Acc. (%)Baseline Acc. (%)Δ (pp)95% CI (pp)Statistical test
LoCoMo w/o conv-26 / EverMemOS1,78789.9385.79+4.14[+2.39, +6.15]Cluster bootstrap (9 conversations)
LongMemEval-S / LightMem50070.0067.00+3.00[−2.20, +8.20]Exact McNemar, p = 0.287

We computed 95% confidence intervals using 10,000 paired bootstrap resamples. For LoCoMo, we resampled the nine entire conversation histories as clusters to account for the possible dependencies among questions from the same conversation. The LoCoMo accuracy advantage remains positive under conversation-level cluster resampling, with a 95% confidence interval of [+2.39, +6.15] percentage points.

Additionally, three evaluations using the same trained checkpoint yield 89.93/90.60/89.42 on LoCoMo and 70.00/71.40/70.40 on LongMemEval-S, respectively, showing consistent QA accuracy across repeated evaluations. The statistical procedures and evaluation details are reported in Appendix D.5.

Independent evidence-quality audit.

We used Codex (GPT-6-Astra configuration) to independently assess DeferMem's distilled evidence on evaluation questions. The assessment examined whether individual evidence entries were faithful to their source messages and self-contained, as well as whether the complete evidence set was sufficient to answer the query or omitted answer-critical information.

We evaluated 70 sampled questions per benchmark, including both correct and incorrect QA outcomes.

Rebuttal Table F. Independent model-based audit of distilled evidence (70 sampled QA per benchmark).

BenchmarkFaithful entriesSelf-contained entriesSufficient evidence setsAnswer-critical omissions
LoCoMo w/o conv-26139/158 (87.97%)143/158 (90.51%)58/70 (82.86%)10/70 (14.29%)
LongMemEval-S171/179 (95.53%)173/179 (96.65%)60/70 (85.71%)8/70 (11.43%)

The audit provides separate assessments of source faithfulness, evidence self-containedness, and answer sufficiency. Further details are provided in Appendix D.4.

AuthorsOpenReview ↗

Response to Reviewer W9w3 [4/4]

W9w3 W2 & Q4. Memory-System Efficiency, Repeated-Query Costs, and Training Resources

Memory-operation efficiency metrics.

As defined in Sec.4.1, our reported runtime measures the end-to-end memory-system processing time, from history construction through query-time memory utilization, until information is returned to the downstream answerer. For DeferMem, this includes Segment-Link construction and retrieval, as well as the full local 8B evidence-distillation inference, including prompt tokenization, GPU transfer, prefill, generation, and output post-processing. Final answer generation and judging are excluded from the memory-operation metrics. Commercial-API tokens are reported separately to quantify paid LLM API usage during memory operations.

Write- and query-time breakdown.

To examine efficiency over repeated queries, we further break down the reported memory-system efficiency metrics into write-time memory construction and query-time memory utilization.

Rebuttal Table G. Breakdown of memory-system efficiency metrics (write per history; query per QA).

DatasetMethodWrite (s)Query (s)Write API tokens (k)Query API tokens (k)
LongMemEval-SDeferMem79.2915.4800
LongMemEval-SLightMem279.160.00431.850
LoCoMoDeferMem54.8710.5600
LoCoMoEverMemOS325.0225.37312.173.63

For a history constructed once and subsequently queried NN times without further memory updates, the cumulative memory-operation time can be estimated from the measured stage averages as T(N)=Twrite+NTqueryT(N)=T_{\mathrm{write}}+NT_{\mathrm{query}}.

On LongMemEval-S, DeferMem yields 79.29+15.48N79.29+15.48N seconds, compared with 279.16+0.004N279.16+0.004N for LightMem. DeferMem maintains lower cumulative runtime for up to 12 queries. On LoCoMo, the corresponding expressions are 54.87+10.56N54.87+10.56N and 325.02+25.37N325.02+25.37N for DeferMem and EverMemOS, respectively. DeferMem maintains lower cumulative runtime for any positive number of queries.

Importantly, DeferMem requires zero commercial LLM API tokens for both memory construction and query-time utilization. In comparison, LightMem consumes 31.85k commercial-API tokens per constructed history on LongMemEval-S, while EverMemOS consumes 312.17k per history plus 3.63k per query on LoCoMo.

Training costs before deployment.

We separately account for the resources used to train the DeferMem‘s memory distiller. Preparing 329 verified GPT-5.2 full-reward anchors involved 1,160 recorded generation and validation calls, consuming approximately 2.80M input and 0.078M output tokens. Anchors are pre-generated once and reused throughout training. During DistillPO training, LLM-based reward evaluation was invoked 27,134 times, including 3,369 cache hits (12.42%) that did not require external API requests. The recorded API usage totaled approximately 69.60M input and 2.71M output tokens. GPU resource usage for DistillPO training was approximately 167.95 A100 GPU-hours.

These costs are incurred during anchor preparation and distiller training. Once trained, DeferMem performs memory construction and query-time evidence distillation locally, without requiring any commercial LLM API calls for memory operations.

Manuscript revisions.

The write- and query-time breakdown of memory-system efficiency metrics, cumulative runtime analysis across query counts, and training-time API usage and GPU-hours are reported in Appendix D.6.

W9w3 Q5. Definitions of r2r_2 and r5r_5 for Empty Gold Support or Distilled Evidence

We have clarified the reward edge cases in revised Table 17 (Table 10 in the original submission). When the gold supporting-message set is empty, selection coverage reward r2=2r_2=2 if the selected-message set is empty and r2=0r_2=0 otherwise. For the faithfulness reward, an empty distilled-evidence list receives r5=1r_5=1 because it contains no unsupported statements.

Reviewer_iTjxOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes DeferMem, a query-time evidence distillation framework for long-term conversational memory. It first preserves historical interactions using a segment-link structure and retrieves high-recall but noisy candidates through message retrieval and semantic expansion. A distillation model then selects key messages and rewrites them into faithful, self-contained, and query-specific evidence. The authors further introduce DistillPO, which trains the distiller using decomposed rewards, hierarchical gating, structure-aligned advantages, and high-quality anchor completions. Experiments show that DeferMem improves both question-answering accuracy and runtime efficiency on LoCoMo and LongMemEval-S.

Critical strengths
  1. Coherent system design. The segment-link retriever and structured distiller form a sensible pipeline. Separating message selection from evidence rewriting makes the training objectives easier to interpret.
  2. Useful ablation coverage. The paper studies the retriever, distiller, reward components, reward gating, structure-aligned advantages, full-reward anchor, alternative retrieval schemes, and different downstream answerers. This is considerably stronger than presenting only an end-to-end comparison.
  3. Promising empirical performance. The reported improvements are consistent across LoCoMo and LongMemEval-S. The cross-dataset variants reduce, although do not eliminate, concerns that the method merely memorizes the training benchmark.
Critical weaknesses

The main weakness of this paper is its limited methodological novelty. DeferMem mainly combines several existing ideas into a unified pipeline: its segment construction follows LightMem, query-time evidence distillation is closely related to reranking and query-focused summarization, the policy optimization is based on DAPO, and the full-reward anchor is adapted from DaGRPO. Reward decomposition, hierarchical gating, and span-level advantage assignment are reasonable engineering modifications, but they do not constitute a fundamentally new learning paradigm. Therefore, the contribution appears to be primarily an effective integration of established techniques rather than a substantial algorithmic advance, and the paper needs stronger comparisons and analyses to demonstrate what DistillPO uniquely enables beyond simpler retrieval-plus-SFT or reranking baselines.

Questions for authors

See weakness

2 comments
AuthorsOpenReview ↗

Response to Reviewer iTjx [1/2]

Thank you for your review. We would like to clarify the two complementary contributions of DeferMem.

First, DistillPO is a reinforcement learning algorithm designed for structured query-time evidence distillation. It jointly optimizes source-message selection and source-aligned evidence rewriting through decomposed rewards, leaky hierarchical gating, and structure-aligned advantages.

Second, DeferMem decouples lightweight, high-recall memory retrieval from query-conditioned evidence formation, allowing the evidence needed for a particular question to be constructed at query time rather than fixed during memory construction.

We have expanded Sec.2 (Related Work) to clarify the relationship to PRCA, RECOMP, CORE, and ECoRAG. The additional comparisons, ablations, and training-process analyses below further demonstrate the contributions of this formulation and the DistillPO algorithm.

iTjx W1. DistillPO's Algorithmic Contribution and Empirical Comparisons

1. DistillPO's optimization design and component ablations.

DistillPO is designed to optimize a structured evidence-distillation action that involves two interdependent decisions: selecting useful source messages and rewriting their content into query-conditioned evidence. Its reward decomposition, leaky hierarchical gating, and structure-aligned advantage assignment are designed around this action structure.

The original Table 3(a) provides direct ablations of these algorithmic components. Removing structure-aligned advantages reduces accuracy by 3.08/4.40 points on LoCoMo w/o conv-26 / LongMemEval-S, while removing the full-reward anchor reduces accuracy by 3.33/3.20 points. Replacing leaky hierarchical gating with strict gating or no gating reduces accuracy by 4.14/7.40 and 2.52/5.20 points, respectively. Additionally, assigning r6,r7r_6,r_7 to the selection span as well reduces accuracy by 2.24/2.40 points, providing further empirical support for DistillPO's span-specific reward assignment.

The reward-component ablations further demonstrate the contributions of the decomposed training signals. Removing the selection rewards r2,r3r_2,r_3 reduces accuracy by 42.92/61.80 points, while removing the LLM-judged rewards r5r_5–r7r_7 reduces accuracy by 4.82/6.20 points. Together, these results demonstrate the importance of DistillPO's component-specific training design.

2. Comparison with SFT and standard policy optimization.

The original Table 3(a) already includes supervised training on the verified full-reward anchor completions, as well as a DAPO baseline using the same 8B initialization. SFT achieves 80.97/63.40, and DAPO achieves 84.89/60.60, compared with 89.93/70.00 for DistillPO on LoCoMo w/o conv-26 / LongMemEval-S. These results demonstrate the effectiveness of DistillPO's training design beyond supervised learning on verified evidence-distillation demonstrations and standard sequence-level policy optimization.

3. Contribution of the structured selection–rewriting action.

We further trained two memory-distiller variants with different output formulations: one outputs only selected useful source messages, while the other outputs only rewritten evidence. Both variants were evaluated using exactly the same Segment-Link retrieved candidates and downstream answerer/judge protocol as DeferMem.

Rebuttal Table A. Output-formulation ablations (QA accuracy, %).

Distiller outputLoCoMo w/o conv-26Δ (pp)LongMemEval-SΔ (pp)
Selected useful messages only85.28−4.6566.80−3.20
Rewritten evidence only86.35−3.5867.00−3.00
Joint selection and rewriting (DeferMem)89.93—70.00—

The joint selection–rewriting formulation consistently outperforms both alternatives, demonstrating the benefit of explicitly coupling source-message selection with query-conditioned evidence rewriting.

AuthorsOpenReview ↗

Response to Reviewer iTjx [2/2]

4. Comparisons with reranking and learned compression.

We additionally evaluate the BGE cross-encoder reranker [1] and extractive/abstractive RECOMP compressors [2], using as input exactly the same Segment-Link–retrieved candidates provided to DeferMem's memory distiller. For RECOMP, we use public checkpoints trained on Natural Questions (NQ) and HotpotQA. All methods use the same GPT-4o-mini downstream answerer and dataset-specific judge. We evaluate two token budgets, 1,024 and 2,048, for the memory information passed to the answerer.

Rebuttal Table B. QA accuracy (%) of reranking and compression methods under two output budgets.

Reranking/compression methodLoCoMo w/o conv-26: 1,024LoCoMo w/o conv-26: 2,048LME-S: 1,024LME-S: 2,048
BGE reranker (v2-m3)73.9274.0362.6064.60
RECOMP extractive, NQ73.0372.3053.2056.40
RECOMP extractive, HotpotQA73.1472.6961.4063.60
RECOMP abstractive, NQ57.1957.1319.0019.00
RECOMP abstractive, HotpotQA50.5350.6439.6040.20

Both DeferMem and DeferMem† use a maximum distiller completion length of 1,024 tokens. Under its original structured-output setting, DeferMem achieves 89.93% on LoCoMo and 70.00% on LongMemEval-S, while DeferMem†, trained exclusively on the other benchmark's QA data, achieves 86.07% and 67.20%, respectively. Both outperform all evaluated reranking and compression alternatives, whose best accuracies are 74.03% and 64.60%, respectively.

5. Training-process evidence for DistillPO's optimization design.

Beyond final QA accuracy, we analyzed the actual DistillPO training trajectory. Among 19,841 on-policy completions containing both selection and rewriting spans, the two span-level advantages have opposite signs in 3,095 cases (15.60%). A unified sequence-level advantage conflicts with at least one of the two span-specific advantages in 2,966 cases (14.95%). These observations provide direct evidence of the distinct optimization signals addressed by structure-aligned advantage assignment.

We also examined reward activation throughout training. Under leaky hierarchical gating, the answerability-related rewards are computed for 91.3%/97.7%/98.3% of on-policy completions during early/middle/late training, compared with 76.0%/95.8%/97.9% for the quality-related faithfulness reward and 29.1%/55.0%/75.5% for the quality-related conciseness reward. This demonstrates how the gating design makes answerability-related feedback available early in training. These training-process observations complement the performance ablations in the original Table 3(a), which show that removing structure-aligned advantages or replacing leaky hierarchical gating consistently reduces QA accuracy.

Manuscript revisions.

We expanded the related-work discussion in Sec.2, added the output-formulation ablations and the additional reward-assignment variant to Table 3(a), and included representative BGE/RECOMP results in Table 3(b). The complete reranking/compression comparisons and training-process analyses are reported in Appendices D.3 and D.7, respectively. All manuscript additions are highlighted in blue.

References

[1] BAAI. BGE Reranker v2-m3.

[2] Xu, F., Shi, W., and Choi, E. RECOMP: Improving Retrieval-Augmented LMs with Context Compression and Selective Augmentation. ICLR 2024.

Taken together, these comparisons show that DeferMem's gains go beyond straightforward combinations of existing retrieval, distillation, and training techniques. The component ablations and training-process analyses further substantiate the effectiveness of DistillPO's algorithmic design for structured selection–rewriting.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary of Key Contributions and Supporting Evidence

Dear AC and Reviewers,

We thank the reviewers for recognizing the strengths of DeferMem's framework design, component ablations, and empirical results, and for acknowledging the additional evidence and clarifications provided during rebuttal. Here, we summarize the key evidence supporting DeferMem's contributions, particularly the methodological novelty of DistillPO.

1. Contributions and evidence already in the original submission.

DeferMem makes two complementary contributions: (1) DistillPO, a reinforcement learning algorithm that jointly optimizes source-message selection and evidence rewriting through decomposed rewards, leaky hierarchical gating, and structure-aligned advantages; and (2) a long-term memory framework that decouples high-recall retrieval from query-conditioned evidence formation.

The original Table 3(a) already compared DistillPO with SFT and DAPO. All three use the same Llama-3.1-8B initialization. SFT was trained on the verified full-reward anchor completions also used by DistillPO. DAPO and DistillPO use the same underlying reward signals, with matched numbers of on-policy rollouts and optimizer updates, while DAPO applies a unified sequence-level advantage. DistillPO achieves 89.93/70.00%, compared with 80.97/63.40% for SFT and 84.89/60.60% for DAPO on LoCoMo w/o conv-26 / LongMemEval-S. Even without full-reward anchors, DistillPO achieves 86.60/66.80%, outperforming DAPO on both benchmarks.

The original Table 3(a) also presented extensive ablations of DistillPO's algorithmic design, including decomposed reward components, leaky hierarchical gating, structure-aligned advantages, and full-reward anchors. Removing or replacing these components reduced accuracy on both benchmarks. For example, removing structure-aligned advantages reduces accuracy by 3.08/4.40 points, replacing leaky gating with strict gating by 4.14/7.40 points, and removing the LLM-judged rewards by 4.82/6.20 points.

Together, the original submission already provided comparisons against simpler training alternatives and component-level evidence supporting DistillPO's optimization design.

2. Additional experimental evidence provided during rebuttal.

  • Structured selection–rewriting formulation and reward assignment. Newly trained output-formulation ablations show that joint selection–rewriting (89.93/70.00%) outperforms both selection-only (85.28/66.80%) and rewriting-only (86.35/67.00%) variants. An additional reward-assignment ablation shows that also assigning downstream answerability rewards to the selection span reduces accuracy by 2.24/2.40 points. These results support the joint action formulation and DistillPO's design for assigning optimization signals.
  • Comparisons with reranking and learned compression. Using the same Segment-Link candidates and downstream evaluation protocol, we compared DeferMem with BGE reranking and public RECOMP extractive/abstractive checkpoints (Appendix D.3). On LoCoMo w/o conv-26 / LongMemEval-S, DeferMem (89.93/70.00%) and cross-source-trained DeferMem† (86.07/67.20%) both outperform the best BGE (74.03/64.60%) and RECOMP (73.14/63.60%) results, supporting the DistillPO-trained distiller's effectiveness.
  • Training-process analyses. Among 19,841 on-policy completions, selection and rewriting advantages have opposite signs in 15.60% of cases, while a unified sequence-level advantage conflicts with at least one span-specific signal in 14.95%. These observations support the motivation for structure-aligned advantage assignment. The training analyses also show that full-reward anchors restore nonzero on-policy advantages in otherwise all-zero groups, while leaky gating exposes answerability-related feedback early.

Together, these additional results further substantiate the value of DistillPO's structured selection–rewriting formulation and the effectiveness of its optimization design.

3. Evidence quality and efficiency accounting.

An independent model-based audit assessed the quality of DeferMem's distilled evidence (Appendix D.4). Regarding deployment-stage efficiency, the original Table 1 already showed that DeferMem achieved the shortest end-to-end memory-system runtime (65.43/94.77 seconds) on LoCoMo / LongMemEval-S while requiring zero commercial LLM API tokens for memory operations. We added write-time/query-time breakdowns and distiller training-resource accounting (Appendix D.6). Distiller training incurs one-time costs before deployment; once trained, the same distiller can be reused without commercial LLM API calls for memory operations.

We are confident in the contributions of our work. We hope this summary, our rebuttal responses, and the revised manuscript receive careful consideration during the reviewer–AC discussion and in the overall assessment of our paper.

Best regards,

The Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.