- Rating
- 2
- Confidence
- 3
- Informativeness
- 3
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper studies the problem of dependency-mediated memory revision in LLM agents. In this problem, an upstream update may invalidate downstream beliefs, even if these beliefs are never explicitly contradicted and can still be retrieved. To address this problem, the authors propose Cascade Memory, which combines an explicit dependency graph, assumption-based truth maintenance, SAT-based verification, and ordered belief-base contraction. In this framework, an LLM first maps natural-language observations into structured state changes, and then the revision is handled symbolically. The authors evaluate Cascade Memory on STALE and MEME, and it achieves 66.7% and 93.52% overall accuracy, respectively. The improvements are particularly large on MEME Deletion, Cascade, and Absence. The method also consistently improves over LLM-only configurations across four model backbones.
Critical strengths
The paper addresses an important gap between retrieving a stored belief and determining whether it is still justified. Its distinction between direct updates and dependency-mediated invalidation gives the problem a clear focus. Tracking alternative support also offers a sensible way to avoid withdrawing every downstream belief after an update. This is a useful contribution because it identifies what a memory system needs to maintain beyond the facts themselves.
The MEME results provide meaningful evidence that the approach helps with memory revision. Compared with LLM-only, Deletion improves from 45.0% to 96.0%, and Cascade from 67.7% to 96.3%. These are precisely the tasks the method is intended to address. Overall accuracy also improves across all four tested backbones. This gives the proposal empirical value beyond its conceptual motivation.
Critical weaknesses
Some central steps of the revision procedure remain unclear, making its correctness difficult to assess. Since () includes current commitments, the paper should explain whether the target belief is excluded before testing its entailment; otherwise, its own presence could make the test trivial. The algorithm also applies contraction after a belief is judged not entailed, whereas Appendix D.5 defines contraction for an entailed target. The source of propagated replacement values is similarly unclear: Figure 2 changes the commute from subway to car after a health update without showing what licenses that value. These gaps concern the core claim that symbolic support determines which beliefs are retained, withdrawn, or replaced.
The experiments do not establish the added value of the complete formal pipeline over simpler support tracking. The MEME Cascade-only configuration uses the plain-LLM answering path, so it does not isolate the symbolic engine. Appendix B.1 also acknowledges that the STALE configurations do not fully share extraction and execution paths. The full system reaches 66.7%, only 1.8 percentage points above Graph+ATMS, with a single seed. The results therefore leave substantial uncertainty about whether SAT and kernel contraction provide a reliable benefit. This limits how much of the reported improvement can be attributed to the paper’s proposed combination of formal mechanisms.
Questions for authors
How was the dependency ontology constructed, and which benchmark examples were inspected during its design? Were any edges, strengths, or thresholds adjusted based on evaluation results? Evidence from held-out relations or new combinations of dependencies would help establish how well the method generalizes beyond the current ontology.
Can Graph+ATMS and the complete system be compared using identical extracted facts and otherwise shared execution paths? Including a simple deterministic dependency-propagation baseline under the same conditions would help establish what SAT and kernel contraction add beyond support tracking.
How accurately does the extractor identify the relevant slots and values, and how much would performance improve with oracle extraction? For MEME Cascade, what rule or module determines the new downstream value? Dependency edges identify affected slots but do not, by themselves, specify replacement values. A worked example showing the extracted update, transformation rule, and resulting state would clarify this.
When computing (), is the current commitment to (b) excluded from ()? If not, what prevents the belief from being entailed by its own presence? What formula is subsequently contracted when (b) is judged no longer entailed? Please also specify where and how the edge strength () affects the revision procedure.
Could the authors reconcile the token counts and costs in Table 7? For example, Supersede uses 27.49 million tokens but costs 4.35 USD, whereas the stated rates imply at least 27.49 USD. A breakdown by extraction, revision, and answering would also help explain the different cost profiles on STALE and MEME. How do runtime and accuracy change as memory grows, particularly when the six-assumption and 24-label caps are reached?
Could the authors manually check a sample of judge decisions, especially cases where Graph+ATMS and the complete system receive different scores? Their overall gap is only 1.8 percentage points from a single seed. How sensitive is this difference to judging errors and variation across runs?