- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
Previous:
SLICE-Mem gates memory writes with a salience score from LLM teachers, keeping only high-salience entries. It reports 92.4%–99.5% token reductions with largely unchanged task metrics. Main concerns: the gating threshold τg=0.2 was selected on the test benchmarks; the Mem0 baseline (13.8%) is far below public numbers; and the offline consolidation variant yields negative marginal gains.
Update:
This paper, SLICE-Mem, propose a compact salience classifier with the extraction operation to construct memories without online generative LLM calls. Optional offline consolidation handles cross-entry updates. Besides, the paper studies the trade-off between memory quality and inference cost on several benchmarks.
Critical strengths
Previous:
Large, direct efficiency gains:1–2 orders of magnitude token reduction with flat task metrics — clear engineering value. Simple and reproducible:Intuitive mechanism with a fully disclosed teacher-annotation pipeline (Appendix A.5).
Update:
Moreover, avoiding generative calls during online construction offers clear practical value. The revised evaluation and analysis prove that the trade-off between quality–cost.
Critical weaknesses
Previous:
Threshold tuned on test data:τg=0.2 is selected on the evaluation benchmarks without an independent validation split, risking evaluation overfitting. Abnormal baseline and unclear attribution:1Mem0 at 13.8% diverges from public results; the offline variant's negative gain obscures which component is effective.
Update:
The threshold sweep shows a relatively stable range, but the validation for selecting the threshold is not stable and solid. Besides, the extractor’s value is also limited. Teacher annotation adds about 8M tokens ( one-time cost), which is not included.
Questions for authors
- How should τg be chosen without touching the test set? Please provide a sensitivity curve over τg.
- Please explain the Mem0 13.8% baseline (version, configuration, protocol).
- Is teacher-annotation cost counted in the total budget, and how does it compare to the saved tokens?
- What are typical failure cases where the gate drops critical memories, and how do they affect long-horizon tasks?
2 comments
Thanks authors for correcting the Mem0 evaluation and adding the threshold and cost analyses. I have updated and withdraw the broad criticism based on the original 13.8% score.
Besides, the sweep partialy solves my concern about threshold sensitivity, but the threshold selection os still unclear. The consolidation results also clarify a task-dependent trade-off rather than a uniformly negative effect.
Overall, My remaining concerns are the threshold-selection protocol and the extractor’s limited benefit. I will keep my score of 2.
Thank you for drawing our attention to the unusually low Mem0 score, which we have corrected in Table 2 and Appendix D.2 (Table 7) after rerunning both model sizes. Appendix C.3 (Table 5) reports the full threshold sweep, and Section 5 discusses how missed memories can affect later answers.
R1. How can the threshold be chosen before testing, and how sensitive is performance to it? (W1, Q1)
Table 5: Sensitivity of memory curation performance to Salience Gate threshold
R2. Why did Mem0 score 13.8%, and what does offline consolidation contribute to SLICE-Mem? (W1, Q2)
R3. Is teacher annotation included in the token budget? (Q3)
R4. What are typical failure cases and how missed turns matter for long-term use? (Q4)