- Rating
- 3
- Confidence
- 2
- Informativeness
- 3
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper studies how a single memory-eviction action can remove evidence needed by multiple future queries. It defines this effect as the action’s blast radius and shows that action-level tail risk can be non-monotone in the deletion threshold. The proposed framework uses independent conversations to calibrate a threshold that jointly controls large-blast-radius events and the overall rate of evidence loss. Experiments on LoCoMo, LongMemEval, and Mem0-extracted memories evaluate the risk–deletion tradeoff, with separate QA experiments checking the effect on answers.
Critical strengths
-
The theoretical argument is clear and internally coherent. The paper connects risk definitions, attribution drift, conversation-level calibration, and threshold selection into a complete methodological argument. The sample-complexity analysis also explains which risk targets can realistically be certified with limited data.
-
The evaluation connects statistical certification with practical outcomes. The paper reports memory deletion, evidence loss, QA changes, and abstention, making the tradeoffs explicit. The additional scorers, Mem0 experiments, and replay audits provide useful checks beyond the main configuration.
Critical weaknesses
The practical need to control blast-radius concentration is not sufficiently established. The experiments show that CAT and overall evidence loss capture different properties, but provide limited evidence that more concentrated harm causes worse downstream outcomes when overall evidence loss is comparable. The QA evaluation mainly links evidence deletion to answer degradation. Table 1 also shows that a tighter harmed-query constraint achieves similar observed deletion and CAT rates to CAT certification, despite providing a different guarantee. Why controlling concentration deserves an additional constraint therefore remains insufficiently justified.
Questions for authors
What concrete downstream failure motivates controlling CAT in addition to the harmed-query rate? Do any existing results show that concentrating a comparable amount of evidence loss in fewer eviction actions leads to worse task outcomes?