- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This work proposes Mem, which is a test-time memory construction framework that defers memory construction to inference and progressively builds memory along the reasoning process. The experiments support the effectiveness of Mem. The work plans to release the code and datasets via GitHub.
Critical strengths
- The figure is well-presented.
- It seems that the method is relatively easy to rpeoduce.
- The experiment results support the effectiveness.
- The author propose to provide code and data to reproduce the results.
Critical weaknesses
- It there any pseudocode to directly introduce the method? The main Figure 1 is relatively complex, and I do not capture the core idea.
- Can the method could be used to math datasets? Or whether the method is limiated to LongMemEval Dataset?
- Can the evaluation be to more common datasets, such as the MATH dataset or other?
- What is the core difference between this work and previous RAG?
Questions for authors
N/A
3 comments
Thank you for the response. I will keep the positive score for this work, as the main concerns are addressed.
Thank you for your time and efforts again in reviewing our paper. If there are any remaining points or if you require further clarification, please feel free to reach out. We sincerely appreciate your attention and valuable feedback.
Thank you for your detailed review. Please find our point-by-point responses below.
Reply to W1:
Thank you for your comments. T²Mem first extracts lightweight episode-level metadata offline while preserving links to the original interactions. At test time, starting from initially retrieved metadata, the LLM iteratively distills relevant evidence, updates the query-specific memory state, and adaptively decides whether to retrieve additional metadata, revisit raw interactions, or terminate construction based on the current information needs. Finally, the accumulated memory is used to generate the answer. We will incorporate a pseudocode into the revised manuscript to make the overall workflow more accessible.
Reply to W2 and W3:
Thank you for your suggestion. First, on the evaluation setting: T²Mem addresses memory construction over long-term interaction histories, and we follow the standard setting of existing memory systems, evaluating on the widely adopted LoCoMo and LongMemEval benchmarks, the same evaluation suite used by nearly all recent work in this area, including our primary baselines LightMem and AnchorMem.
Regarding extension to math datasets such as MATH: we believe this is a promising direction, but a direct transfer is premature, because what constitutes "memory" in mathematical reasoning is itself undefined. In memory-oriented tasks, factual knowledge, user preferences, and temporal dependencies are well-recognized as content to memorize, whereas in mathematical reasoning, which mathematical knowledge should be memorized, and whether building and reusing such memory helps, remains an open question without established definitions or studies. MATH evaluates standalone problem-solving ability, which differs from dynamically acquiring evidence from interaction histories. We believe exploring whether memory mechanisms benefit mathematical reasoning, and what should be memorized there, is a promising future direction, and thank you for raising this discussion.
Reply to W4:
The key distinction between T²Mem and traditional RAG lies in reasoning-driven, query-specific memory construction at test time, rather than simply retrieving relevant documents to support answer generation. Traditional RAG primarily retrieves external documents as context for answering questions. However, in long-term conversational scenarios, relevant evidence is often scattered across multiple sessions, and fragmented retrieval may fail to preserve temporal relationships and cross-session dependencies. In contrast, T²Mem explicitly organizes evidence accumulated during test-time reasoning into an evolving memory state, which guides subsequent decisions to retrieve additional metadata, revisit raw interactions, or terminate construction. Therefore, compared with traditional RAG and general iterative retrieval, T²Mem goes beyond multi-step evidence acquisition by interleaving evidence retrieval, memory construction, and reasoning decisions, enabling memory to evolve progressively with the reasoning process. This design is further supported by the progressive memory construction ablations in Tables 3 and 5.