- Rating
- 2
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper propose 3D-Mem, a framework that learns how to organize conversational evidence for future retrieval. 3D-Mem employs a PPO controller to select graph-edit operators based on task feedback, an LLM designer to evolve the operator library from QA failures, and provenance anchors to project graph relevance back to source text chunks. Coupled with hierarchical reranking, utility gating, and diversity pruning, this framework effectively organizes and assembles evidence, achieving competitive results on the LoCoMo and LongMemEval-S benchmarks.
Critical strengths
- The paper proposes a 3D-Mem, a framework that learns how to organize conversational evidence for future retrieval.
- 3D-Mem achieves competitive performance against multiple baselines on the LoCoMo and LongMemEval-S benchmarks.
Critical weaknesses
- 3D-Mem use PPO for skill selection has already been explored in MemSkill. The authors merely narrow the scope from general memory skills to graph-operation skills, without providing a comparative study of the two strategies within 3D-Mem, which makes the novelty unconvincing. Furthermore, the combination of a multi-scale temporal axis and a semantic topology axis has been extensively discussed in existing works such as MAGMA, MemOS, and TiMem. 3D-Mem essentially stacks existing hierarchical summarization and graph retrieval mechanisms without introducing fundamental architectural innovations.
- Authors claim that 3D-Mem can recover evidence within a "compact context" in abstract. However, according to the results in Table 4, the average retrieved token cost of 3D-Mem is 1259, which is significantly higher than the baseline MemSkill (605.60). This raises my concern of whether the higher F1 performance is merely achieved at the expense of consuming substantially more tokens. Additionally, the authors should provide a comprehensive cost profiling, such as comparing the token and time costs of different baselines across the memory construction, retrieval, and generation stages, to give a fair evaluation of efficiency.
- 3D-Mem appears to be a patchwork of stacked modules that lack overall logical coherence. Essentially, the system relies heavily on fixed heuristic rules. 4.Eq.8 introduces an embedding-based diversity threshold δ, but no experiments demonstrating how different values of this threshold affect the trade-off between F1 performance and token cost.
- Retrieval budgets (Sec.4.1) for the graph and L1–L5 layers are hard-coded as '4, 16, 8, 4, 2, 1'. What is the rationale behind these specific numbers? The paper provides no explanation for this design, nor are there any ablation studies to demonstrate the effectiveness or sensitivity of these hard-coded values.
Questions for authors
Please see weakness