- Rating
- 1
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper studies how an agent’s memory can cause mistakes even when it improves average performance, and how those mistakes can spread when they are stored and reused. The authors introduce NMR-LOOP, a benchmark that compares decisions with and without retrieved memories and tests whether newly stored records harm later tasks. They also propose LoopGuard, which controls both what the agent retrieves and what it saves, using training signals from these interventions. Across tasks, LoopGuard reduces memory-caused failures while preserving similar benefits from useful memories.
Critical strengths
-
The paper goes beyond average performance to examine when retrieved memories cause failures and whether those failures produce harmful records that affect later decisions.
-
LoopGuard reduces harmful reversals compared with competing methods and its read-only variant while preserving similar positive-transfer rates, with modest reported deployment token overhead.
Critical weaknesses
-
The models, controller architecture, and training settings are not clearly specified, making the results difficult to reproduce.
-
Lower memory-induced harm at similar positive-transfer rates does not necessarily imply better end-to-end performance, since each policy is compared against its own no-memory reference. Absolute task-success results would strengthen the practical claim.
-
LoopGuard is compared mainly on controlled streams, while the evaluation only measures dense retrieval. An ablation removing descendant-risk prediction is also needed to establish its contribution beyond immediate-utility prediction and write filtering.
Questions for authors
See weakness.