- Rating
- 1
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes AMBER, a multimodal conversational memory framework that builds temporal, procedural, complementary, and conflicting associations across episodes. At retrieval time, it selects an association type, expands evidence chains from semantic seeds, and ranks them under a fixed budget. The paper also introduces MemStream, a streaming evaluation derived from Mem-Gallery and H2HMem with state-level metrics for continuity and degradation. AMBER improves average F1 over the strongest baseline by 2.68% and improves evidence recall on evolution-related queries.
Critical strengths
The motivation is reasonable: independent similarity retrieval can miss evidence that is procedurally, temporally, or contradictorily related to the current query. The proposed association taxonomy is intuitive, and MemStream is a useful attempt to evaluate process-level memory behavior rather than only final answers. The paper evaluates two answer backbones and reports both answer and evidence-retrieval metrics.
Critical weaknesses
The method is fairly complex while the demonstrated gains are modest. The average F1 improvement over the strongest baseline is 2.68%, and key ablations provide limited support for the necessity of the proposed machinery: removing specific association reduces F1 by only 0.47%, while removing anchor generation reduces it by only 0.27%. The method also relies heavily on LLM-assisted association construction and anchor generation without sufficient analysis of cost, latency, reproducibility, or failures with weaker/open models. MemStream's annotation agreement and metric robustness are also under-validated, and the small judge-based gains need statistical significance analysis.
Questions for authors
- What is the total memory-construction and retrieval cost of AMBER, including all LLM calls?
- How does AMBER perform if association construction and anchor generation use an open or weaker model?
- Are the answer-quality gains statistically significant under paired tests?
- What is the annotation agreement for MemStream state labels and supporting evidence?
Post-rebuttal follow-up
I have not seen a substantive response or additional experiment addressing the questions in my review. My concerns therefore remain unchanged. In particular, the framework introduces considerable complexity while showing limited average improvement over the strongest baselines, and the reported ablations for association routing and anchor generation have relatively small effects. The paper also lacks sufficient analysis of total construction cost, retrieval latency, association construction with weaker or open models, statistical significance, and MemStream annotation consistency. I maintain my rating of 1 (Clear Rejection).