- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
DeferMem addresses question answering over long conversational histories. Instead of directly passing retrieved messages to an answerer, it first retrieves a broad set of candidates and then uses a separate model to select relevant messages and rewrite them into compact, query-specific evidence. The proposed DistillPO algorithm trains this distiller using rewards for valid selection, evidence quality, and downstream answerability, with different rewards assigned to selection and rewriting. Experiments on LoCoMo and LongMemEval report improved answer accuracy and memory-system runtime, accompanied by component ablations and cross-source training evaluations.
Critical strengths
-
The framework addresses a practical mismatch between retrieval and answering. A retrieved message may contain relevant evidence together with substantial unrelated information. DeferMem explicitly separates finding potentially useful messages from constructing the evidence needed for the current question. This provides a clear interface between memory retrieval and downstream answering and allows the distiller to be reused with different answerers.
-
The ablations provide useful evidence about the system’s components. The paper separately examines the retrieval structure, distiller training, reward gating, advantage assignment, and teacher anchors. It also reports LoCoMo results excluding the training conversation and evaluates cross-source training. These controls help distinguish improvements from individual design choices and assess dependence on the training source.
Critical weaknesses
-
The contribution beyond existing learned compression methods is unclear. DeferMem’s central pipeline retrieves candidate text, compresses it according to the current question, and passes the result to a separate answerer. RECOMP [1] and ECoRAG [3] already study related post-retrieval compression, while CORE [2] trains a query-conditioned compressor using downstream QA rewards with a frozen answerer. These works are not discussed. Consequently, the current comparisons establish improvements over memory systems more clearly than they establish the additional value of DistillPO over an existing learned compressor. Table 3 shows why this comparison matters. Passing segment-link candidates directly to the answerer already gives 84.11 on LoCoMo, close to the strongest baseline (EverMemOS, 85.79), and a prompted GPT-4o-mini distiller with no training reaches 86.35, above every baseline. Most of the margin over memory systems therefore comes from the retrieve-then-compress pipeline that these compression methods already use, and the value of DistillPO itself can only be judged against a learned compressor.
-
Message selection is trained to match annotated evidence, without direct answerability feedback. The distiller first outputs selected message IDs and then rewrites their contents. DistillPO scores the selection against annotated supporting messages, while downstream answerability rewards are assigned to the rewriting span. However, a failed answer may result from selecting the wrong messages rather than rewriting them poorly. Removing the selection rewards reduces LongMemEval-S accuracy from 70.0 to 8.2, showing how strongly the current training design depends on this supervision. Its behavior with incomplete annotations or alternative valid evidence therefore remains uncertain.
-
Correct answers do not establish that the distilled evidence is faithful. The distiller generates new text rather than only copying source passages, so it can potentially introduce unsupported details. The main evaluation measures final QA accuracy, and the fidelity reward is applied during training. Neither directly establishes the factual support of held-out evidence statements. A separate evaluation is needed to distinguish unsupported rewriting from omission of necessary information.
-
The efficiency comparison does not fully account for differences in execution resources. DeferMem performs distillation with a local 8B model, whereas several baselines use commercial APIs for memory operations. The reported zero API-token usage therefore describes how computation is provided, not its total cost. Hardware, batching, and API latency can also affect the runtime ranking. More explicit resource accounting is needed to establish a reproducible efficiency advantage. Table 1 itself suggests the runtimes are not comparable: NaiveRAG makes no API calls and only performs embedding retrieval, yet it takes 867.38 s on LongMemEval-S and 173.87 s on LoCoMo, far longer than DeferMem (94.77 s and 65.43 s), which performs the same kind of retrieval plus segmentation and 8B-model distillation.
-
The cross-source gains are too small to interpret without uncertainty estimates. The paper evaluates distillers trained on a different data source to test transfer beyond the training benchmark. These variants exceed the strongest reported baselines by only 0.20 points on LongMemEval-S and 0.28 points on held-out LoCoMo conversations. Although the full-training results are stronger, these small transfer margins do not yet establish reliable cross-source superiority.
-
The training signal includes distillation from a much stronger teacher that sees the answer. Appendix C.2 states that every training group contains one anchor completion written by GPT-5.2 from the query, the gold answer and the gold supporting messages, and the main text describes it only as "a strong teacher". This mixes teacher imitation into the RL objective, and the paper does not separate how much of DistillPO's gain comes from this anchor versus its reward design. The GPT-5.2 calls and the LLM-judged rewards used in training are also commercial-API costs that the claim of "no commercial-API tokens" leaves out.
[1] RECOMP: Improving Retrieval-Augmented LMs with Context Compression and Selective Augmentation. 2024.
[2] CORE: Lossless Compression for Retrieval-Augmented LLMs via Reinforcement Learning. 2025.
[3] ECoRAG: Evidentiality-guided Compression for Long Context RAG. 2025.
Questions for authors
Please address the weaknesses above.
4 comments
Response to Reviewer sTMJ [2/4]
sTMJ W2. Selection Supervision and Answerability Credit
Selection supervision and answerability.
In DistillPO, the selection span receives answerability-related supervision through the gold-support coverage reward , together with the conciseness reward . By contrast, checks the validity of selected message IDs, while evaluates the alignment between selected messages and rewritten evidence. Satisfying these validity and alignment requirements (, ) does not necessarily mean that the selected messages contain the information needed to answer the query.
This helps explain the substantial accuracy reductions observed when removing in the original Table 3(a): 42.92/61.80 points on LoCoMo w/o conv-26 / LongMemEval-S. The results highlight the importance of providing selection-specific rewards that directly encourage the identification of answer-supporting messages, beyond ensuring structural validity and selection–evidence alignment.
Reward-assignment ablation.
Although final-answer correctness depends on both selection and rewriting, an incorrect answer does not necessarily reveal which component is responsible. To examine whether assigning downstream answerability rewards to selection would improve training, we further trained a variant that also assigns to the selection span. This reduces accuracy from 89.93/70.00 to 87.69/67.60 on LoCoMo w/o conv-26 / LongMemEval-S, a decrease of 2.24/2.40 points, providing further empirical support for DistillPO's span-specific reward assignment.
The training-process analysis also reveals distinct optimization signals for selection and rewriting. Among 19,841 on-policy completions containing both spans, their advantages have opposite signs in 3,095 cases (15.60%), while a unified sequence-level advantage conflicts with at least one span-specific advantage in 2,966 cases (14.95%). These observations further support the importance of distinguishing the optimization signals for the two components.
Supporting-evidence annotations.
Answer-supporting evidence annotations used by are widely available in long-term conversational-memory QA benchmarks. For example, LoCoMo and LongMemEval provide annotations identifying supporting dialogue messages, while PersonaMem-v2 provides answer-supporting conversation snippets that can be mapped back to the original messages. In our experiments, DistillPO leverages existing supporting-evidence annotations without requiring additional manual evidence labeling. At query time, the distiller selects useful source messages without access to gold supporting-message annotations.
Manuscript revisions.
We added the selection-span assignment variant to Table 3(a). The corresponding training-process analysis is reported in Appendix D.7.
sTMJ W3. Independent Assessment of Evidence Faithfulness and Answer-Critical Omissions
Independent evidence-quality audit.
We used Codex (GPT-6-Astra configuration) to independently assess DeferMem's distilled evidence on evaluation questions. The assessment examined whether individual rewritten evidence entries were faithful to their source messages and self-contained, as well as whether the complete evidence set was sufficient to answer the query or omitted answer-critical information. In particular, the faithfulness assessment identifies unsupported rewriting, while the sufficiency and omission assessments examine whether necessary information is preserved in the distilled evidence.
We evaluated 70 sampled questions per benchmark, including both correct and incorrect QA outcomes.
Rebuttal Table H. Independent model-based audit of distilled evidence (70 sampled QA per benchmark).
| Benchmark | Faithful entries | Self-contained entries | Sufficient evidence sets | Answer-critical omissions |
|---|---|---|---|---|
| LoCoMo w/o conv-26 | 139/158 (87.97%) | 143/158 (90.51%) | 58/70 (82.86%) | 10/70 (14.29%) |
| LongMemEval-S | 171/179 (95.53%) | 173/179 (96.65%) | 60/70 (85.71%) | 8/70 (11.43%) |
These results provide separate assessments of the factual grounding of rewritten evidence and the sufficiency of the complete distilled-evidence sets for answering the queries, with explicit checks for unsupported rewriting and answer-critical omissions. Further details are provided in Appendix D.4.
Response to Reviewer sTMJ [3/4]
sTMJ W4. Memory-System Efficiency
Memory-operation efficiency metrics.
As defined in Sec.4.1, our reported runtime measures the end-to-end memory-system processing time, from history construction through query-time memory utilization, until information is returned to the downstream answerer. For DeferMem, this includes Segment-Link construction and retrieval, as well as the full local 8B evidence-distillation inference, including prompt tokenization, GPU transfer, prefill, generation, and output post-processing. Final answer generation and judging are excluded from the memory-operation metrics.
We report commercial LLM API token usage as a separate efficiency metric, capturing the reliance on paid API services during memory operations and the associated token-based expenditure. Thus, DeferMem's zero commercial-API token usage means that memory construction and query-time evidence distillation require no commercial LLM API calls, while their local computation is fully included in the reported runtime.
Execution setup and efficiency breakdown.
The runtime comparisons in Table 1 use the same execution resources (queries were processed sequentially with batch size one, four NVIDIA RTX 3090 GPUs) and a consistent memory-operation measurement scope across methods, covering both memory construction and query-time memory utilization. To further clarify the sources of runtime differences, we break down the reported memory-system efficiency metrics into write-time memory construction and query-time memory utilization.
Rebuttal Table I. Breakdown of memory-system efficiency metrics (write per history; query per QA).
| Benchmark | Method | Write (s) | Query (s) | Write API tokens (k) | Query API tokens (k) |
|---|---|---|---|---|---|
| LongMemEval-S | DeferMem | 79.29 | 15.48 | 0 | 0 |
| LongMemEval-S | LightMem | 279.16 | 0.004 | 31.85 | 0 |
| LoCoMo | DeferMem | 54.87 | 10.56 | 0 | 0 |
| LoCoMo | EverMemOS | 325.02 | 25.37 | 312.17 | 3.63 |
On LongMemEval-S, DeferMem achieves substantially lower memory-construction time than LightMem (79.29s vs. 279.16s). On LoCoMo, DeferMem achieves lower runtime than EverMemOS in both memory construction (54.87s vs. 325.02s) and query-time utilization (10.56s vs. 25.37s). These results show how the different processing stages contribute to the end-to-end runtime differences reported in Table 1, and DeferMem requires zero commercial LLM API tokens for both memory construction and query-time utilization.
NaiveRAG runtime provenance.
The NaiveRAG runtime results in Table 1 are based on the NaiveRAG baseline results reported in LightMem's main experiments, which use the same execution resources, backbone LLM, and judge LLM as our evaluation. We retained the available results and supplemented the missing memory-construction measurements and LoCoMo's fifth (adversarial) QA category under the same experimental settings.
Manuscript revisions.
The evaluation setup is detailed in Appendix B. The write- and query-time breakdown of memory-system efficiency metrics, cumulative runtime analysis across query counts, and training-time API usage and GPU-hours are reported in Appendix D.6.
sTMJ W5. Cross-Source Transfer of Learned Distillation
The cross-source variants in the original Table 1 evaluate whether a distiller trained exclusively on one benchmark's QA data can transfer its learned evidence-distillation behavior to another benchmark. They achieve 67.20% on LongMemEval-S and 86.07% on LoCoMo, closely matching the strongest memory-system baselines. The main evidence for cross-source transfer comes from the improvements over the untrained distiller, rather than the small numerical margins over these baselines.
As shown in the original Table 3(a), the untrained 8B distiller achieves 52.20% on LongMemEval-S and 72.02% on LoCoMo under the same evaluation setting. Training exclusively on the other benchmark's QA data increases accuracy to 67.20% (+15.00 points) and 86.07% (+14.05 points), respectively. These substantial improvements provide direct evidence of cross-source transfer of the learned distillation policy, while the fully trained DeferMem achieves the strongest overall results.
Response to Reviewer sTMJ [4/4]
sTMJ W6. Teacher-Anchor Contributions and Training-Time API Usage
Anchor contribution and original ablations.
As shown in the original Table 3(a), DistillPO without full-reward anchors achieves 86.60/66.80 on LoCoMo w/o conv-26 / LongMemEval-S, compared with 89.93/70.00 with anchors, quantifying their contribution as 3.33/3.20 points. Importantly, even without teacher-generated anchors, DistillPO outperforms DAPO (84.89/60.60) and SFT trained on verified anchor completions (80.97/63.40). These comparisons demonstrate the effectiveness of DistillPO's structured optimization beyond teacher-derived demonstrations, with additional improvements from full-reward anchors.
How anchors contribute to optimization.
The verified full-reward anchors are generated by GPT-5.2 using the training query, gold answer, and gold supporting messages, and are reused throughout training. Each anchor participates in both group-relative advantage normalization and the clipped policy optimization objective, allowing its selection and rewriting tokens to contribute directly to parameter updates. Our analysis of 4,962 recorded training groups shows that including the anchor changes 66.38%/73.35% of selection/rewriting advantages and restores nonzero on-policy advantages in 11.83%/10.00% of selection/rewriting groups. These observations further characterize the anchor's contribution to DistillPO's training process.
Training resource accounting and deployment API usage.
As defined in Sec.4.1, the commercial-API token metric in Table 1 measures API usage during deployment-stage memory operations, spanning write-time memory management and query-time memory utilization until information is returned to the downstream answerer.
We additionally quantify the resources used for training: preparing 329 verified GPT-5.2 full-reward anchors involved 1,160 recorded generation and validation calls, consuming approximately 2.80M input and 0.078M output API tokens. During DistillPO training, LLM-based reward evaluation was invoked 27,134 times, including 3,369 cache hits (12.42%) that did not require external API requests. The recorded API usage for reward evaluation during training totaled approximately 69.60M input and 2.71M output tokens. GPU resource usage for DistillPO training was approximately 167.95 A100 GPU-hours.
These resources are incurred during the training of the memory distiller. Once trained, the same distiller can be reused throughout deployment, with no commercial LLM API calls required for either write-time memory management or query-time evidence distillation.
Manuscript revisions.
We clarified the anchor's participation in group normalization and clipped policy optimization in Appendix C.2, added the training-process analyses in Appendix D.7, and reported training-time API usage and GPU-hours in Appendix D.6.
Response to Reviewer sTMJ [1/4]
Thank you for your detailed review. We have conducted additional comparisons with learned compression methods, trained output-formulation ablations, and an independent assessment of distilled-evidence quality. We also provide further analyses of selection supervision, full-reward anchors, memory-system efficiency, and training-time resource usage. We address your concerns below, with the corresponding manuscript revisions highlighted in blue.
sTMJ W1. DistillPO's Contribution Beyond Learned Post-Retrieval Compression
Positioning relative to learned compression.
RECOMP and ECoRAG investigate retrieval-conditioned compression, while CORE trains a query-conditioned compressor using downstream QA rewards. We have expanded Sec.2 (Related Work) to discuss these methods and clarify DeferMem's two complementary contributions.
First, DistillPO is a reinforcement learning algorithm designed for structured query-time evidence distillation. It jointly optimizes source-message selection and source-aligned evidence rewriting through decomposed rewards, leaky hierarchical gating, and structure-aligned advantages. Second, DeferMem decouples lightweight, high-recall memory retrieval from query-conditioned evidence formation, allowing question-specific evidence to be constructed at query time rather than fixed during memory construction.
Gains beyond retrieval and prompted distillation.
The original Table 3 already provides relevant comparisons. On LoCoMo w/o conv-26, directly passing Segment-Link candidates to the answerer achieves 84.11%, below EverMemOS (85.79%). A prompted GPT-4o-mini distiller achieves 86.35%, only 0.56 points above EverMemOS, whereas the DistillPO-trained 8B distiller achieves 89.93%, improving by 3.58 points over prompted distillation and 4.14 points over EverMemOS. On LongMemEval-S, the prompted GPT-4o-mini distiller achieves 63.80%, compared with 70.00% for DistillPO.
The original Table 3(a) also includes an SFT baseline trained on verified full-reward anchor completions (80.97/63.40) and a DAPO baseline (84.89/60.60), compared with 89.93/70.00 for DistillPO on LoCoMo/LongMemEval-S. These results demonstrate the effectiveness of DistillPO's training design beyond supervised learning on verified evidence-distillation demonstrations and standard sequence-level policy optimization.
Output-formulation ablations.
We further trained two memory-distiller variants with different output formulations: one outputs only selected useful source messages, while the other outputs only rewritten evidence. Both variants were evaluated using exactly the same Segment-Link retrieved candidates and downstream answerer/judge protocol as DeferMem.
The selection-only variant achieves 85.28/66.80, and the evidence-only variant achieves 86.35/67.00, compared with 89.93/70.00 for the joint selection–rewriting formulation on LoCoMo/LongMemEval-S. These results demonstrate the benefit of explicitly coupling source-message selection with query-conditioned evidence rewriting.
Comparisons with reranking and learned compression.
We additionally evaluate the BGE cross-encoder reranker and extractive/abstractive RECOMP compressors, using as input exactly the same Segment-Link–retrieved candidates provided to DeferMem's memory distiller. For RECOMP, we use public checkpoints trained on Natural Questions (NQ) and HotpotQA. All evaluated methods use the same GPT-4o-mini downstream answerer and dataset-specific judge. We evaluate two token budgets, 1,024 and 2,048, for the memory information passed to the answerer.
The strongest RECOMP results across these settings are 73.14% on LoCoMo and 63.60% on LongMemEval-S, while BGE reranking reaches 74.03% and 64.60%, respectively. Both DeferMem (89.93/70.00) and DeferMem† (86.07/67.20), whose distillers use a maximum completion length of 1,024 tokens, outperform all these alternatives. DeferMem† is trained exclusively on the other benchmark's QA data, further demonstrating the effectiveness of the learned distillation policy across data sources.
Manuscript revisions.
We expanded the related-work discussion in Sec.2, added the output-formulation ablations to Table 3(a), and included representative BGE/RECOMP comparisons in Table 3(b). The complete reranking/compression comparisons under both token budgets are reported in Appendix D.3.