- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
The paper proposes PersonaMap, a personal memory agent that stores atomic claims in a graph with four typed relations (same, complements, unresolved conflict, distinct-by-qualifier) and navigates it with tool calls. One LoRA policy on Qwen2.5-7B-Instruct does all stages and is initialized by SFT on GPT-5.1 teacher traces that were converted into the tool protocol. RL is done with RICO, a GRPO variant that gives credit only to actions at "natural forks" (matched prefix and state, different actions, different mean terminal reward). When a rollout group has no such fork it falls back to a small counterfactual replay. The reward is a binary evidence-sufficiency verdict from a frozen Qwen3.8-27B judge on the claims the agent observed. A deterministic structure-aware pruning rule is also proposed. Results: 50.81% on SubtleMemory (personas 7 to 9), 56.41% zero-shot on PersonaMem, RICO 50.81 vs GRPO 48.12 from the same SFT checkpoint, and about 80% relational evidence retention at 50% claim compression.
Critical strengths
S1: Credit assignment matched to the structure of memory actions. Restricting credit to decisions where matched-prefix branches actually disagree, and replaying a counterfactual only when needed, is a reasonable and cheap way to avoid smearing one outcome over extraction, relation and navigation steps. The coverage analysis (Fig. 4, 33.2% of questions get usable credit, 23.7% from natural forks) is useful because it shows how sparse the signal really is.
S2: Policy/executor split. The policy picks content and labels while the executor binds endpoints and mutates the graph, and teacher traces are replay-validated (A.6). This makes the system easier to reason about than most LLM-memory pipelines.
Critical weaknesses
W1: The RICO vs GRPO evidence is thin. The method claim rests on 50.81 vs 48.12 in Table 2. That is a single run, with no seeds or intervals, on three held-out personas whose question count I could not find. The compute is also not matched. "Same rollout budget" in Fig. 5 seems to count only the K=16 base rollouts, while RICO adds counterfactual replays and suffix samples. With M=2 suffixes per branch, the counterfactual credit takes values like plus or minus 0.25 or 0.5 and is very noisy. I would want several seeds, GRPO with matched total samples, and a variant that turns off the counterfactual fallback (natural forks only). Fig. 5 (smaller advantages and gradient norms) describes the update. It does not tell me the update is better.
W2: Main comparison is confounded. PersonaMap is distilled from GPT-5.1 teacher traces built from SubtleMemory personas 0 to 6, i.e. the same distribution as the test personas. The paper does not say which LLM backs the training-free baselines or what data the trainable ones were trained on (I may have missed this, but Table 1 gives no backbone column). Some numbers look off: Full Context at 4.27% and most baselines near 0 to 4% on contradictory questions. That makes me suspect a protocol mismatch more than a real gap. The evaluation answer judge for SubtleMemory is also not described. The training reward never scores the generated answer at all (only evidence sufficiency), so I would like to see how well reward and test accuracy line up. The ablation also shows SFT alone (45.88) below the fixed structured pipeline (46.99), so most of the gain over baselines comes from the teacher-derived graph design rather than from RL.
One more: the forgetting result is evaluated on personas 0 to 6 (training personas), only as a graph-level proxy, with no end-to-end accuracy (Appendix C says this is pending). The proxy requires the gold relations to survive, and the method protects anything reachable through relations, so it is partly favored by construction. The abstract claim should be softened.
Questions for authors
Q1. How many test questions are in SubtleMemory personas 7 to 9? Can you report mean and std of RICO and GRPO over at least three seeds, and GRPO at matched total rollout count including the counterfactual samples?
Q2. Which backbone LLM and which checkpoints were used for each baseline in Table 1? How is answer accuracy judged at evaluation time? What is the Structured Graph row exactly (GPT-5.1 or the 7B model running the fixed pipeline)?
Q3. In practice, at what depth do natural forks occur? I would guess that exact prefix plus state fingerprint matching at K=16 mostly gives forks at the first divergent action. If so, how much of the credit actually lands on relation and navigation decisions rather than early AddClaim calls?
Q4. How often does the evidence-sufficiency reward disagree with final answer correctness on held-out data?
2 comments
Response to reviewer HrTx
This continues our response to Reviewer HrTx from Part 1/2.
3) Natural-fork depth and credited action types (Q3).
Exact prefix matching means a matched branch point is the first divergence of the compared trajectories. It can occur at different stages, depending on how long their prefixes agree.
Fig. 4 includes credited nodes for ClassifyRelation, Neighbors, and Inspect, showing that credit reaches relation and navigation decisions as well as extraction. These are credited-node counts, not unique-question counts or a depth distribution. We do not currently have a depth-by-stage breakdown. Repeated-action averaging and matched-context CF comparisons improve comparability, but the small continuation budget leaves finite-sample uncertainty.
4) Evidence reward versus answer correctness (W2, Q4).
The frozen Qwen3.8-27B judge evaluates whether the observed evidence supports answering the question; it does not score the generated answer (Appendix B.3). Final QA scoring is performed separately. Sufficient evidence can still lead to an incorrect answer, so these signals need not agree. We have not measured their held-out disagreement rate and cannot report a verified percentage.
5) Forgetting scope and downstream QA (W2).
The original approximately 80% result measures complete mapped relational-evidence retention on eligible questions from personas 0–6 at 50% claim compression. It is a graph-level result, not held-out QA accuracy. The pruning operator uses graph structure and does not access gold evidence mappings, although the structural metric alone cannot establish downstream utility.
We additionally evaluated QA after removing 50% of claims on the same 422 held-out questions. Accuracy decreases from 53.08% (224/422) to 39.10% (165/422), retaining 73.7% of its unpruned value. This directly quantifies the downstream cost of compression.
Forgetting is currently an offline, deterministic operation; longitudinal memory-growth and maintenance-cost curves have not been evaluated. We will revise the abstract to state the scope of the evidence-retention result explicitly.
Response to reviewer HrTx
Thank you for the constructive feedback. We appreciate your recognition: our credit-assignment design is a reasonable and inexpensive way to avoid conflating outcomes across memory operations; the coverage analysis is useful; and the policy/executor split makes the system easier to reason about than most LLM-memory pipelines. We address your concerns below.
1) Question counts, RL comparisons, and sampling budgets (W1, Q1).
We apologize for the aggregation and reporting error. Earlier SubtleMemory results used incomplete aggregates that did not consistently cover all three held-out personas. We have completed evaluation on all 422 questions from personas 7–9: 107 complementary, 207 nuanced, and 108 contradictory. Missing or invalid predictions count as incorrect.
These results replace the earlier aggregates. RICO improves over the shared SFT initialization by 4.50 percentage points and over GRPO by 3.55 points in this run.
“Same rollout budget” refers to the K=16 base rollouts. RICO first reuses natural comparisons within these rollouts, then selectively adds CF continuations only for groups without usable natural credit. This increases usable-credit coverage from 23.7% to 33.2% of questions. This additional signal requires additional sampling: with BCF=1 and M=2 continuations per branch, an eligible group can require up to four extra suffix continuations. Thus, equal base-rollout budgets do not imply equal total sampling cost.
The reported QA results are single-run comparisons. Additional seeds and total-sampling-matched GRPO controls are currently underway. These comparisons require multiple independent RL training runs and substantial GPU resources, making them time-consuming. We will supplement the results and analysis once the runs are complete and verified.
We agree that smaller advantages and gradient norms alone do not establish better updates. Fig. 5 illustrates RICO’s update characteristics; its downstream benefit is evaluated through the QA comparison.
2) Baseline models, evaluation, and the structured pipeline (W2, Q2).
All Table 1 scores come from our own evaluation runs. Appendix D.1 describes the evaluation protocol, and Appendix D.2 specifies the baseline configurations. Training-free memory baselines—Mem0, Mem0-g, A-Mem, LightMem, MemoryOS, and GAM—use Qwen2.5-7B-Instruct for memory operations, with the 128K variant on SubtleMemory. MemAgent, Mem-α, UMA, and MEM1 use the RL-MemoryAgent-7B, Memalpha-4B, UMA-4B, and MEM1-Qwen2.5-7B checkpoints, respectively. Memory-R2 uses its own memory-management checkpoint. The shared answer-generation pipeline uses Qwen2.5-7B-Instruct at temperature zero (Appendices D.1–D.2).
The Structured Graph baseline uses Qwen2.5-7B-Instruct to run our fixed pipeline. For SFT data construction, GPT-5.1 runs the pipeline to generate teacher demonstrations (Appendix A). Thus, the Structured Graph score reports the Qwen-based baseline, rather than the GPT-5.1 teacher’s performance.
For SubtleMemory, answer correctness follows the benchmark’s evaluation procedure, using Gemini-2.5-Flash with majority voting over three calls when LLM judgment is required. The answer prompts and scoring procedures are detailed in Appendix D.3.
Responses to the remaining concerns continue in Part 2/2.