- Rating
- 1
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper introduces A-MBER, a framework that evaluates a model's ability to leverage cross-session history to explain current emotional or relational states through judgment, evidence retrieval, and explanation tasks. The dataset comprises 34 synthetic interaction trajectories and 2,047 items; experiments compare the performance of ten models under conditions involving local context, full history, and retrieved history.
Critical strengths
Distinguishing between sentiment judgments, the identification of historical evidence, and explanations enables an analysis that is richer and more accurate than one based on simple sentiment labels.
The authors conducted trajectory-level manual inspections, spot checks, and calibration of explanation scoring to provide initial quality assurance for the synthetic benchmark.
Critical weaknesses
Section 4.3 stipulates that the overall score is the average of the scores across all items. Table 6 further indicates that "content type" and "memory level" each constitute a complete partition of the total 2,047 items. Therefore, calculating the weighted average score for the groups within either complete partition—based on the number of items—should yield the overall score found in the main table. However, for the "Retrieved" condition, Claude-Opus yields a result of approximately 0.590 when weighted by the number of items and scores across memory levels (Tables 6 and 9), and approximately 0.601 when weighted by content type; yet, Table 2 reports a score of 0.613. These two complete partitions differ not only from the main table but also from each other.
There is already direct precedent for memory evaluation in long-term emotional support. ES-MemEval (WWW 2026) has investigated implicit and scattered information as well as evolving user states within this context, covering aspects such as information extraction, temporal reasoning, conflict detection, refusal to answer, and user modeling, while evaluating memory utilization through question answering, summarization, and dialogue generation. Its EvoEmo dataset combines existing emotional support conversations with synthetic cross-session trajectories. While the overlap between this work and A-MBER is evident, the current manuscript fails to cite or discuss it. Consequently, characterizing the intersection of long-term memory and emotional understanding as lacking evaluation is insufficient to substantiate the paper's claims of novelty. A-MBER must clearly demonstrate that its approaches to current utterance interpretation, relationship dynamics, and evidence chain design introduce challenges not addressed by ES-MemEval, rather than merely rebranding the task.
The use of historical implicit states to explain subsequent behavior has also been evaluated. LoCoMo-Plus assesses whether models can apply implicit user states, goals, and value constraints by leveraging the semantic disconnect between historical cues and subsequent requests. Its appendix explicitly includes examples where "prior emotional states influence subsequent behavior," and it employs a constraint-consistency evaluation that allows for multiple valid responses; the experiments cover standard models, RAG, and specialized memory systems. Although A-MBER cites this work, it fails to demonstrate—through task-aligned experiments—what additional insights its emotion-related items capture. Performance gains derived from accessing history do not, in themselves, prove the existence of a novel emotional memory capability.
Clearer methods now exist to distinguish between retrieval failures and evidence integration failures. Beyond reporting retrieval performance, LongMemEval employs an "oracle" condition, providing only the correct evidence—to examine the reading and reasoning errors a model makes even when the necessary evidence is already available. While A-MBER utilizes the top-10 results from BGE-M3, it lacks a corresponding control using correct evidence and fails to analyze upstream retrieval coverage. Residual errors under retrieval conditions could stem from missed retrievals, missing segments, or interpretation failures; current experiments do not adequately differentiate among these causes. Since existing work has already demonstrated this diagnostic approach, the present paper should either adopt it or explain why it is inapplicable.
The “multimodal” claim, however, shows a significant discrepancy with the actual input: tone, pauses, and expressive style are described via text, yet the paper provides no experiments using the corresponding raw audio or video inputs. Comprehending a textual description of tone is a different task from recognizing emotion based on acoustic or visual signals. While manual inspection addresses concerns regarding synthetic data quality, the kappa value of 0.82 measures consistency in decisions to retain, modify, or rewrite content; it does not directly demonstrate consistency in the ground-truth emotional interpretations. Furthermore, the 2,047 items originate from only 34 trajectories, and item-level bootstrapping fails to adequately account for the statistical dependencies arising from these shared trajectories.
Questions for authors
see weakness. If the authors can address the issues raised in the "Weakness" section, I will raise my rating.