- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 2
Read review
Paper summary
MemFit removes the LLM from the write path and from retrieval ranking, relying on verbatim turn storage, dual BM25 and dense vector indexes, segment summaries used as an index, and cross-encoder reranking to achieve low-cost memory construction and precise retrieval. It achieves competitive results on the LoCoMo, MemGallery, and LongMemEval-S benchmarks across multiple backbone models, while showing significantly efficiency than comparable baselines.
Critical strengths
MemFit achieves competitive performance against multiple baselines on the LoCoMo, MemGallery, and LongMemEval-S benchmarks, while requiring significantly fewer LLM token cost / inference time than those baselines.
Critical weaknesses
- The font in Fig. 1 is too small, especially in the “memory insertion append only” part, hard to follow.
- Although the authors claim “LLM free retrieval,” summary expansion clearly depends on LLM generated summaries. The quality of these summaries also appears to have a significant impact on the candidate pool. Moreover, authors emphasize “no consolidation,” the summary layer in Memfit is essentially a form of consolidation.
- The authors do not clearly explain why consolidation in iterative retrieval baselines such as Mem0, SeCom, and MemTree is unnecessary (in related work). The discussion in this part is unclear.
- Segmenting still requires an LLM call, so this is not “no LLM call per turn.” Also, why is no consolidation necessarily a reasonable design choice?
- The 'additive context window' design is naive. After retrieving an episode, the method adds neighboring turns from the same session to the pack with . However, is fixed and does not adapt to session length, turn type, or query. No sensitivity analysis about this parameter in the experiments.
- Overclaim. Neither authors claimed “LLM free write path” nor the “hybrid retrieval and cross encoder reranking” seems novel. The motivation for the former overlaps with that of many existing memory works, while the latter is a standard practice in information retrieval.
- The authors claim the “summary is an index, not compression,” but do not explain why using the summary as an index is better than using it as a memory unit. In my opinion, the 'index' in this paper is also a type of summary.
Questions for authors
Please see Weaknesses
Thank you for the review.
(4) Segmented does not require an LLM call. Segmentation uses TextTiling over the stored embeddings (Sec. 3.2), instead of an LLM. The only write-time LLM cost is one summary per segment, which costs about 437 calls for LoCoMo's 5,882 turns.
Consolidation decides at write time which version of a fact survives, before any question is known, and it cannot recover the discarded evidence. Many questions need exactly what consolidation discards. MemFit keeps all interactions, with their dates, and lets the LLM process them correctly when the question is known. Four results support this:
(5) Regarding "No sensitivity analysis of w.” The paper does include a sensitivity analysis:
The fixed context window is intentionally simple because making it adaptable requires additional LLM calls or a learned or thresholded rule, which requires tuning per dataset and reduces practical applicability.
(2) Regarding "LLM-free retrieval" and summary quality, "LLM-free retrieval" means search and ranking do not use an LLM call. The single call per question is the answer. We write summaries once at storage time. On summary quality, the paper contains a direct test: replacing summaries written by Qwen3-8B with ones written by Gemma3-27B did not change accuracy (judge 64.4 vs. 64.6, p=0.77; Appendix C). The new experiment below shows that on LoCoMo summaries matter only as an index.
Regarding "the summary layer is consolidation”, we use "consolidation" in the same way the memory literature does. “Consolidation” means operations that rewrite, merge, or delete stored memories (Mem0's UPDATE/DELETE, A-MEM's note evolution, MemoryBank's forgetting). MemFit never does this. Summaries are added beside the turns. We agree that summarizing is a form of abstraction. The case against destructive consolidation is that it must choose what to keep before the question is known, and errors are unrecoverable (see Response to R1, W2).
(7) Regarding why we use summaries as an index rather than as memory units, we present new results using the same summaries, same retrieval, and same reader (LoCoMo, Gemma3-27B):
Used as memory units, the summaries locate the right stretch of conversation for 75.6% of questions, yet judge accuracy falls by 45 points because the answer itself (a date, a name, a count) does not survive compression. Retrieving whole segments verbatim loses 26 points and uses more tokens than MemFit, because segment-level search is too coarse to place the evidence turn in context. Turn-level search with summaries as an index recovers both. This is the paper's distinction: the summary finds a stretch of conversation, and the raw turn answers the question.
(3) Regarding related work, we agree that the paragraph was unclear. Mem0, SeCom, and MemTree are not iterative-retrieval methods: they consolidate or restructure memory at write time (Mem0, MemTree), or compress segments into the retrieval unit (SeCom). We have rewritten the paragraph.
(6) Regarding novelty, we agree that BM25, dense retrieval, and cross-encoders are standard IR components. The contribution is a memory design and its evidence:
This design matches or beats LLM-managed memory systems with 81–87% fewer LLM calls.
(1) Figure 1 is larger, with a larger font, in the revision.