- Rating
- 1
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
EvoMem stores dated versions of facts and uses query types and development feedback to choose retrieval rules for questions about changing information.
Critical strengths
The dated version chains keep old facts available for questions about earlier states.
Critical weaknesses
-
It specifies 400 test questions, and Appendix B.4 describes averaged binary scores. Accuracy must therefore change in steps of 0.25. Table 1 reports 82.6 and 88.4, neither obtainable after rounding those outcomes. Another aggregation rule might explain this, but the description leave it unresolved. Provide actual denominators and per-question verdicts before interpreting the reported gains.
-
It can accept a count of 1 instead of 2, or a week instead of an exact date. The pre-check also accepts high keyword overlap. These rules can reward errors in abilities tested by LongMemEval, compare verdicts with human judgments. The current rules makes memory accuracy difficult to trust; rescore using question-specific factual requirements.
-
It train EDPL on known development answers, then freeze it before testing. This supports offline tuning, but not continued learning from noisy user feedback. EvoMem needs a comparable feedback–update–evaluation sequence. The development curve alone do not establish continued improvement; report later gains and new errors.
-
It places a wish to adopt a child and an interest in counseling within one career history. These may be separate attributes, rather than changing versions of one fact. This example raises a core extraction concern, without establishing its frequency. checks claims against sources. EvoMem’s version chains needs similar checks of attribute identity and update relations.
-
It promises the same first 30,000 characters, but Figure 4 gives EvoMem earlier facts outside FullText’s window. Clarify whether this example follows the scored protocol. Extra information could explain its advantage.The comparison need exact input records to separate versioning benefits from differences in evidence access.
-
These results demonstrate that, In contrast, a broken sentence.
Questions for authors
Please refer to the weaknesses.