- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper introduces LifeMemBench, a benchmark for evaluating long-term memory under evolving user states through memory QA, personalized planning, and replanning. It constructs synthetic longitudinal interactions spanning 8K–64K tokens and proposes CoMem, a state-aware memory system that separates historical evidence from current user states. Experiments show that CoMem outperforms six memory baselines on QA, planning, and replanning using a shared Qwen3.5-4B backbone.
Critical strengths
- The paper addresses an important problem by connecting long-term memory evaluation with downstream personalized decision-making rather than focusing solely on information retrieval.
- The experiments compare multiple memory systems and include dynamic-state evaluations, ablations, and statistical analyses. CoMem demonstrates consistent improvements across the three main tasks.
Critical weaknesses
- Limited generalization. The benchmark contains only 16 synthetic user trajectories, with downstream planning evaluated exclusively in the travel domain. This limits confidence in its applicability to more diverse users and real-world scenarios.
- Incomplete isolation of performance gains. CoMem uses an explicit current-state snapshot that is not provided to competing methods. Although the ablation study supports its usefulness, it remains unclear how much of the improvement comes from this additional representation rather than the overall memory architecture.
Questions for authors
- How well do the findings generalize beyond synthetic user trajectories and travel planning?
- Would CoMem's improvements remain substantial if competing methods were given comparable current-state representations?
2 comments
Kind Reminder
Dear Reviewer Ubyo,
We sincerely appreciate the time and effort you have dedicated to reviewing our work. As the discussion deadline is approaching, we would greatly appreciate it if you could kindly review our rebuttal and additional experimental results.
During the rebuttal period, we carefully addressed your concerns by expanding persona diversity, incorporating real-participant profiles and human dialogues, extending LifeMemBench to workplace scenarios, and evaluating CoMem on independent memory benchmarks. We also conducted additional experiments by incorporating comparable current-state representations into competing memory systems to better isolate the contributions of CoMem.
If you have any further questions or concerns, we would be grateful for the opportunity to address them. Moreover, if you find our responses satisfactory, we would sincerely appreciate it if you could kindly consider updating your rating.
Thank you again for your valuable feedback and consideration. We look forward to your response.
Best regards,
The Authors
Response to Reviewer Ubyo
We sincerely thank the reviewer for the constructive feedback. We agree that data diversity and generalization are important for evaluating LifeMemBench, and we have conducted additional experiments to address these concerns.
List of Changes in the Manuscript:
Note: All revisions are highlighted in red in the updated manuscript.
A1 About limited users. We have expanded LifeMemBench from 16 to 40 user profiles, including 8 profiles and corresponding dialogue datasets constructed from real participant information, covering a wider range of ages, occupations, budgets, preferences, and Big Five personality traits. The supplementary results below cover 4 original synthetic profiles, 2 newly generated synthetic profiles, and 2 profiles based on real participants. CoMem outperforms the strongest baseline for each task by 9.69, 3.37, and 7.28 percentage points on QA, Planning, and Replanning, respectively, providing preliminary support for its effectiveness across different profile sources.
Table 1. Performance comparison on user profiles from different sources. Improvements are measured against the strongest baseline for each task.
About limited downstream planning. We have introduced LifeMemBench-Office, extending downstream tasks beyond travel planning to document writing, presentation creation, task prioritization, and work plan adjustment, while retaining long-term user state evolution and the QA, Planning, and Replanning evaluation framework. CoMem achieves Planning and Replanning scores of 67.50% and 60.50%, respectively, in this setting, broadening the benchmark’s coverage of long-term memory and personalized decision-making.
A2 We thank the reviewer for highlighting the contribution of explicit user state representations. To distinguish this contribution from the benefits of the overall architecture, we provided five baselines with the same current user state representations extracted by CoMem’s state maintenance module, while preserving their original memory mechanisms. State augmentation improved Replanning scores for all baselines, yet CoMem still outperformed the strongest state-augmented baseline on each task by 14.98 and 6.67 percentage points on Planning and Replanning, respectively. These results suggest that CoMem’s gains extend beyond explicit state representations to the coordinated organization of interaction evidence, state change events, and currently valid user states, together with query-conditioned state derivation and evidence-aware retrieval. The full description can be found in Appendix Q.7.