- Rating
- 1
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper proposes Structured Policy Memory (SPM), an external-memory framework that represents agent experience as a typed, weighted graph containing facts, states, actions, outcomes, rules, and constraints. Retrieved subgraphs guide answer generation, while knowledge extracted after each interaction is evaluated by a critic and incorporated into memory. Experiments cover LoCoMo, StrategyQA, LongMemEval, and ALFWorld. The paper presents these results as evidence for improved memory performance, continual policy learning, and favorable computational efficiency.
Critical strengths
The paper addresses a relevant problem in experience reuse. Converting accumulated interactions into reusable guidance for future decisions is an important objective for long-running LLM agents. A representation that combines factual information with behavioral constraints is a plausible direction, particularly when future queries require relationships between multiple pieces of experience.
The appendix provides useful visibility into the proposed system. The submission includes algorithmic pseudocode, extraction and reasoning prompts, graph-retrieval ablations, critic ablations, and failure examples. These materials help identify how memory updates operate and where the approach can fail. The authors also acknowledge ordering bias and the limitations of critic filtering, although these qualifications are not adequately reflected in the main conclusions.
Critical weaknesses
Essential experimental specifications are missing, and central claims conflict with the reported results.
Section 5.1 (p. 6, lines 306–311) states that all experiments use the same underlying LLM but does not identify its name, size, checkpoint, or version. Neither the main text nor the appendix explicitly identifies the actual backbone, embedding model, or model used for critic evaluation, and important generation settings are not specified. These omissions do not permit reproduction from the manuscript alone and make it difficult to assess baseline comparability or interpret the latency measurements. A statement that methods share a model does not replace identifying the model and explaining each method’s configuration.
There are also substantive claim–evidence conflicts. Table 6 (p. 14) reports SPM at 0.3325, below Mem0 at 0.3372 and KG-RAG at 0.3329, while its accompanying text explicitly claims superiority over these methods. Both baselines are absent from Table 1, which supports the headline claim of best overall LoCoMo performance. Table 8 (p. 15) reports ALFWorld Action Match of 95.78% for SPM, below BM25 (99.40%), Reflexion (98.20%), and A-Mem (99.80%). These results do not support the broad statements of consistent superiority. Additionally, Table 4 and Tables 10/14 all specify the first 100 LoCoMo questions, but report conflicting results: Table 4 gives 8.0% EM for both SPM and RAG (BM25), whereas Tables 10 and 14 consistently report 6.0% for SPM and 5.0% for RAG (BM25). KG-RAG likewise changes from 8.0% in Table 4 to 4.0% in both Tables 10 and 14. Table 1 labels the overall measure Semantic Similarity, whereas Table 6 reproduces matching values for many methods under “Success Rate.” The manuscript does not reconcile these differences.
The concern is not a requirement to outperform every baseline. The experimental conditions and reported outcomes are insufficiently specified and internally inconsistent, preventing a dependable assessment of the claimed contribution.
The evaluation does not establish continual improvement through policy evolution.
Section 5.3 and Table 7 evaluate cumulative scores over changing question prefixes. The fixed-index BM25 baseline improves by 10.50 percentage points, demonstrating a substantial question-composition effect. Subtracting this change from SPM’s 13.17-point gain does not identify a learning effect: different systems can respond differently to changing questions even without memory evolution. The within-conversation early-versus-late comparison in §5.7 has the same limitation. A same-system frozen-memory control or fixed held-out probes is needed to attribute improvement to the update mechanism.
Algorithm 1, line 10, and Table 11 (p. 17) explicitly provide the ground-truth answer to the extraction module after prediction. This can be a legitimate supervised online protocol, but equivalent feedback access across baselines is not established. The paper therefore does not adequately distinguish acquisition of reusable policies from accumulation of answer-derived factual information. This is not evidence that the current answer was exposed before prediction; it is a concern about information available for subsequent questions and the comparability of the evaluation.
The available mechanism ablations do not resolve the issue. Removing policy evolution reduces Semantic Similarity by only 0.19 points on the 500-question subset (Tables 2/9). Removing the critic improves EM from 17.60% to 18.00% and Semantic Similarity from 31.53 to 31.86 (Table 13, p. 21). The authors acknowledge this subset result and assert that filtering becomes beneficial at larger scales, but do not provide the corresponding full-length quantitative comparison. Consequently, the central claim of sustained improvement through critic-guided policy evolution remains unsubstantiated.
Questions for authors
Actual models and configurations: What exact backbone, embedding model, and critic model were used? Please provide checkpoint/API versions, decoding configurations, module assignments, and the corresponding baseline settings. Where are these specified in the submitted materials? How are graph construction, extraction, and critic calls included in the efficiency accounting?
Reconciliation of the experimental record: Please reconcile Tables 1 and 6, the stated superiority over Mem0/KG-RAG, and the first-100-question EM discrepancies between Tables 4 and 10/14. Are these different metrics, subsets, or protocols? Which results support the abstract and conclusion after reconciliation?
Feedback and information boundaries: Which baselines receive ground-truth answers after each query? When is memory reset, and how are repeated or closely related questions handled? Is the initial graph built from the same permitted corpus for every method? Please reconcile the empty-graph description in §5.1 with BuildInitialGraph(M) in Algorithm 1.
Controlled evidence for evolution and critic filtering: Can the authors compare the same SPM system with evolving memory, frozen memory, and no critic over the full evaluation length under identical feedback? Fixed held-out probes or multiple randomized question orders would help separate memory updates from question-composition effects. Please provide uncertainty across independent runs or orders.
Meaning of cross-benchmark results: Please define ALFWorld Action Match and the LongMemEval temporal reasoning score, including their denominators, aggregation, and evaluation procedures. Does ALFWorld involve closed-loop execution, and what is the episode-level task success rate? Do the additional benchmarks evaluate transfer of an existing policy memory or separately initialized systems? These distinctions materially affect the claimed generalization and decision-making conclusions.
Response to Reviewer nGmU
Thank you for these questions. We address the reported results and the limits of the submitted record.
1. Models and efficiency. Section 5.1 states that methods share an underlying LLM, but the submission does not give its checkpoint or API version. It also does not identify the embedding checkpoint, a separate Critic model, decoding settings, module assignments, or complete baseline configurations. Algorithm 1 specifies the roles of answering, extraction, and
CriticEvaluate, but not their exact implementations.Table 4 reports 46,691 ms average latency and 535 input plus 24 output tokens per question for SPM on the first 100 LoCoMo questions. It does not itemize graph construction, retrieval, post-answer extraction, Critic calls, or graph updates. We therefore cannot establish from that table that the reported latency and tokens include the full cost of maintaining an evolving memory.
2. Experimental record. Appendix Table 6 gives the LoCoMo category values on a 0–1 scale and adds methods omitted from Table 1. Although its caption calls them “Success Rate,” the corresponding main-table values are labeled Semantic Similarity. SPM scores 0.3325 overall, below Mem0 (0.3372) and KG-RAG (0.3329). “Best overall” applies only to the methods displayed in Table 1; superiority over Mem0 and KG-RAG is not supported by Appendix Table 6.
Table 4 reports 8.0% EM for SPM on the first 100 LoCoMo questions, while Tables 10 and 14 report 6.0%. The submission documents no difference in subset, metric, or protocol that explains the two-question discrepancy. We cannot reconcile it by assuming a different evaluation run.
The supported conclusion is narrower than the abstract’s: SPM reaches 33.25% overall LoCoMo Semantic Similarity and performs competitively across the reported tasks. The complete set of reported comparisons does not establish best LoCoMo performance or consistent superiority across benchmarks.
3. Feedback and information boundaries. Algorithm 1 pairs each query with its ground-truth answer. SPM predicts first; extraction then receives the question, prediction, and ground truth. Accepted triples can enter memory and be retrieved for later questions. The submitted protocol does not establish which baselines receive equivalent post-prediction feedback. It also does not specify a reset between conversations or controls for repeated facts and closely related questions. Consequently, the observed learning curve may include benefits from earlier answer feedback and question ordering.
Section 5.1 describes an empty initial graph, whereas Algorithm 1 calls
BuildInitialGraph(M). These descriptions agree if M is empty at initialization, but the submission does not specify that condition or demonstrate that every method starts from the same permitted corpus. Equal information access and conversation isolation cannot be verified from the reported protocol.4. Evolution and Critic controls. The 500-question ablation compares full SPM with policy evolution disabled: Semantic Similarity is 31.53 versus 31.34. A separate ablation finds that removing the Critic increases EM from 17.60% to 18.00% and Semantic Similarity from 31.53 to 31.86%. The submitted results do not compare evolving, frozen, and no-Critic memory over the full 1,500-question stream under identical feedback.
SPM’s score rises by 13.17 percentage points between 50 and 1,500 processed questions, but a fixed-index BM25 baseline also rises by 10.50 points. This indicates that question order or composition affects the trend. Without fixed held-out probes, randomized orders, and uncertainty across independent runs, the additional increase cannot be assigned confidently to graph evolution. The reported 500-question ablation does not show an accuracy benefit from Critic filtering.
5. Cross-benchmark interpretation. Figure 5 labels the ALFWorld evaluation n=100, and Table 8 reports 95.78% Action Match. The submission does not specify whether its denominator counts actions, action sequences, or another unit, nor does it give the matching procedure or episode-level success rate. Action Match does not establish closed-loop task completion.
Figure 5 labels LongMemEval n=133 and reports a temporal reasoning score of 5.26, but does not sufficiently define its denominator, aggregation, or scoring procedure. The submission also does not state whether these benchmarks reuse a policy memory learned elsewhere or initialize separate systems. Their results therefore cannot establish cross-benchmark transfer of an existing memory or successful embodied decision making.